Test an agent

Write OrchaJS agent tests with real model calls, mocked actions, and contract assertions.

01{02  "name": "Delayed order response",03  "input": {04    "content": "Where is order ord_1001?"05  },06  "actions": {07    "lookup_order": {08      "responses": [09        {10          "output": {11            "orderId": "ord_1001",12            "status": "in_transit"13          }14        }15      ]16    }17  },18  "expect": {19    "status": "completed",20    "text": {21      "contains": ["ord_1001", "in transit"]22    },23    "actions": [24      {25        "name": "lookup_order",26        "arguments": {27          "equals": { "orderId": "ord_1001" }28        }29      }30    ]31  }32}

Protect the behavior that matters

This example is a complete order agent with one model-backed contract test. The test runs the real compiled agent and provider while returning a fixed response for lookup_order.

Mocking the action makes the data deterministic without replacing the model. The expectations verify that the agent completed, called the correct action with the exact order ID, and included confirmed facts in its response.

Each direct folder under tests/ is one case. Run this case with:

npx orcha test orderAgent/delayedOrder

Unmocked local actions execute live, so mock operations that should not touch real services during a test. Client actions must always be mocked.

Select any file in the editor to inspect the complete standalone project.

Understand each file

orcha/orderAgent/tests/delayedOrder/index.json

Defines one model-backed contract test.

input is passed to the real compiled agent and provider.
actions supplies deterministic mock responses.
expect checks status, response text, and ordered action calls.

orcha/orderAgent/actions/lookupOrder/index.json

Defines the action contract used by the agent and referenced by the mock.

orcha/orderAgent/actions/lookupOrder/index.js

Contains the live implementation. It is not executed because this test mocks lookup_order.

orcha/orderAgent/index.json

Configures the agent tested by the case.

orcha/orderAgent/instructions.md

Defines the behavior the test is protecting.

orcha/index.ts

Registers orderAgent. Test folders are discovered automatically.

Next: evaluate an agent.

Protect the behavior that mattersCopy Anchor Link

Understand each fileCopy Anchor Link

On this page

Protect the behavior that matters

Understand each file