## File: README.md

# Yao Agents

✨ **All your agents and workspaces in one place, on every device you own. Track tasks on a board, accessible from desktop, mobile, browser, or API. Self-hosted.**



**🏠 Homepage:** [https://yaoagents.com](https://yaoagents.com)

**📚 Docs:** [https://yaoagents.com/docs](https://yaoagents.com/docs)

**🖥️ Yao Desktop:** [https://yaoagents.com/download](https://yaoagents.com/download)


⚡⚡⚡ **DeepSeek Harness is now integrated — use it directly in the desktop app or browser, running across your devices.**

*This README was written with DeepSeek Harness in the Yao Agents Android app (beta) and published automatically.*

---

## How it works

Agents run on your own devices. Every machine you add is another place for them to work, all managed from one place.

## Workspaces

Every workspace is isolated. Work stays separate, managed across all your computers, and accumulates into documents that become your knowledge base.

While working, agents can read what they need from any node.

## Task Board

Say what you need in a conversation, and it becomes a task on your board.

Keep the conversation going, keep it running, and it improves with use, growing into an agent purpose-built for this kind of work.

Connect task agents to your apps, or share them with others.

## Open API

Expert and task agents integrate into your own apps and workflows. They're built on standard APIs, with both SSE and WebSocket support.

Let your agents become part of your business, wherever you need them.

---

## Links

- Homepage: https://yaoagents.com
- Docs: https://yaoagents.com/docs
- Yao Desktop: https://yaoagents.com/download
- Discord: https://discord.gg/yao

---

## File: agent/eval/README.md

# Agent Eval Framework

A comprehensive evaluation framework for Yao AI agents with support for E2E testing, dynamic (simulator-driven) testing, agent-driven assertions, script unit tests, and CI integration.

## Quick Start

```bash
# Send a single message
yao agent eval myagent "what is the weather"

# Run JSONL test cases (@ = read from file, like curl)
yao agent eval myagent @tests/inputs.jsonl

# JSON output for AI consumption
yao agent eval myagent "submit $500 expense" --json

# Run TypeScript unit tests
yao agent eval myagent --scripts tools

# Override connector and user identity
yao agent eval myagent "hello" -c deepseek.v3 -u user-001 -t team-alpha

# Stability test: 5 rounds
yao agent eval myagent @tests/inputs.jsonl --runs 5 --json
```

### Agent-Driven Input

```bash
# Generate test cases using an agent
yao agent eval -i "agents:tests.generator-agent?count=10" -n assistants.expense

# Preview generated tests without running (dry-run)
yao agent eval -i "agents:tests.generator-agent?count=5" -n assistants.expense --dry-run
```

### Dynamic Mode (Simulator)

```bash
# Run dynamic tests with simulator
yao agent eval -i tests/dynamic.jsonl --simulator tests.simulator-agent

# See detailed turn-by-turn output
yao agent eval -i tests/dynamic.jsonl -v
```

### Script Tests

```bash
# Test agent handler scripts (hooks, tools, setup functions)
yao agent eval -i scripts.expense.setup -v

# Run specific tests with regex filter
yao agent eval -i scripts.expense.setup --run "TestSystemReady" -v

# Run with custom context (authorization, metadata)
yao agent eval -i scripts.expense.setup --ctx tests/context.json -v
```

## Input Modes

The `-i` flag supports multiple input modes:

### 1. JSONL File Mode

Load test cases from a file:

```bash
yao agent eval -i tests/inputs.jsonl
```

Agent is auto-detected by traversing up from the input file to find `package.yao`.

### 2. Direct Message Mode

Test with a single message:

```bash
# Auto-detect agent from current working directory
cd assistants/keyword
yao agent eval -i "Extract keywords from this text"

# Or specify agent explicitly
yao agent eval -i "Hello" -n workers.system.keyword
```

### 3. Agent-Driven Input Mode

Generate test cases using a generator agent:

```bash
# Basic usage (-n specifies the target agent to test)
yao agent eval -i "agents:tests.generator-agent" -n assistants.expense

# With parameters
yao agent eval -i "agents:tests.generator-agent?count=10&focus=edge-cases" -n assistants.expense

# Dry-run to preview generated tests
yao agent eval -i "agents:tests.generator-agent?count=5" -n assistants.expense --dry-run
```

**Note**: The `-n` flag is **required** for agent-driven input mode to specify which agent to test. The generator agent creates test cases for the target agent.

### 4. Script Test Mode

Test agent handler scripts:

```bash
yao agent eval -i scripts.expense.setup -v
```

Script test input format: `scripts.<assistant>.<module>` (e.g., `scripts.expense.setup` → `assistants/expense/src/setup_test.ts`).

### 5. Script-Generated Input Mode

Generate test cases using a script:

```bash
yao agent eval -i "scripts:tests.gen.Generate" -n assistants.expense
```

**Note**: `scripts.xxx` (with dot) runs script tests, while `scripts:xxx` (with colon) generates test cases from a script.

## Test Modes

### Standard Mode

Single call to agent with optional message history. Each test is independent and stateless.

```jsonl
{
  "id": "T001",
  "input": "Hello",
  "assert": {
    "type": "contains",
    "value": "Hi"
  }
}
```

### Dynamic Mode

Simulator-driven testing with checkpoint validation. A simulator agent generates user messages while checkpoints verify agent behavior.

```jsonl
{
  "id": "T001",
  "input": "I want to order coffee",
  "simulator": {
    "use": "tests.simulator-agent",
    "options": {
      "metadata": {
        "persona": "Customer",
        "goal": "Order a latte"
      }
    }
  },
  "checkpoints": [
    {
      "id": "greeting",
      "assert": {
        "type": "regex",
        "value": "(?i)hello"
      }
    },
    {
      "id": "ask_size",
      "after": [
        "greeting"
      ],
      "assert": {
        "type": "regex",
        "value": "(?i)size"
      }
    }
  ],
  "max_turns": 10
}
```

## Command Line Options

| Flag          | Description                                              | Default                    |
| ------------- | -------------------------------------------------------- | -------------------------- |
| `-i`          | Input: JSONL file, message, `agents:xxx`, or `scripts:x` | (required)                 |
| `-o`          | Output file path                                         | `output-{timestamp}.jsonl` |
| `-n`          | Agent ID (optional, auto-detected)                       | auto-detect                |
| `-a`          | Application directory                                    | auto-detect                |
| `-e`          | Environment file                                         | -                          |
| `-c`          | Override connector                                       | agent default              |
| `-u`          | Test user ID                                             | `test-user`                |
| `-t`          | Test team ID                                             | `test-team`                |
| `-r`          | Reporter agent ID for custom report                      | built-in                   |
| `-v`          | Verbose output                                           | false                      |
| `--ctx`       | Path to context JSON file for custom authorization       | -                          |
| `--simulator` | Default simulator agent ID for dynamic mode              | -                          |
| `--before`    | Global BeforeAll hook (e.g., `env_test.BeforeAll`)       | -                          |
| `--after`     | Global AfterAll hook (e.g., `env_test.AfterAll`)         | -                          |
| `--runs`      | Runs per test (stability analysis)                       | 1                          |
| `--run`       | Regex pattern to filter which tests to run               | -                          |
| `--timeout`   | Timeout per test                                         | 2m                         |
| `--parallel`  | Parallel test cases                                      | 1                          |
| `--fail-fast` | Stop on first failure                                    | false                      |
| `--dry-run`   | Generate test cases without running them                 | false                      |

## Custom Context File

Create a JSON file for custom authorization:

```json
{
  "chat_id": "test-chat-001",
  "authorized": {
    "user_id": "test-user-123",
    "team_id": "test-team-456",
    "constraints": {
      "owner_only": true,
      "extra": { "department": "engineering" }
    }
  },
  "metadata": {
    "mode": "test"
  }
}
```

Use with `--ctx`:

```bash
yao agent eval -i scripts.expense.setup --ctx tests/context.json -v
```

## Input Format (JSONL)

Each line is a JSON object. Below are examples organized by scenario.

### Scenario 1: Simple Text Input

Basic test with string input:

```jsonl
{"id": "greeting-basic", "input": "Hello, how are you?"}
{"id": "greeting-chinese", "input": "你好，请问有什么可以帮助你的？"}
```

### Scenario 2: With Assertions

Validate response content:

```jsonl
{"id": "keyword-extract", "input": "Extract keywords from: AI and machine learning", "assert": {"type": "contains", "value": "AI"}}
{"id": "json-response", "input": "What's the weather?", "assert": {"type": "json_path", "path": "need_search", "value": true}}
{"id": "no-error", "input": "Help me", "assert": {"type": "not_contains", "value": "error"}}
```

### Scenario 3: Multiple Assertions

All assertions must pass:

```jsonl
{
  "id": "expense-submit",
  "input": "Submit $500 travel expense",
  "assert": [
    {
      "type": "contains",
      "value": "expense"
    },
    {
      "type": "not_contains",
      "value": "error"
    },
    {
      "type": "regex",
      "value": "(?i)(submitted|created|confirmed)"
    }
  ]
}
```

### Scenario 4: Conversation History

Test with multi-turn context:

```jsonl
{
  "id": "expense-confirm",
  "input": [
    {
      "role": "user",
      "content": "Submit an expense"
    },
    {
      "role": "assistant",
      "content": "What type of expense?"
    },
    {
      "role": "user",
      "content": "Travel, $500"
    },
    {
      "role": "assistant",
      "content": "Please confirm: $500 travel expense"
    },
    {
      "role": "user",
      "content": "Yes, confirm"
    }
  ],
  "assert": {
    "type": "regex",
    "value": "(?i)(submitted|created)"
  }
}
```

### Scenario 5: With File Attachments

Test with images or documents:

```jsonl
{
  "id": "receipt-analyze",
  "input": {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "Analyze this receipt"
      },
      {
        "type": "image",
        "source": "file://fixtures/receipt.jpg"
      }
    ]
  },
  "assert": {
    "type": "contains",
    "value": "amount"
  }
}
```

### Scenario 6: Agent-Driven Assertion

Use LLM to validate response semantics:

```jsonl
{
  "id": "helpful-response",
  "input": "How do I reset my password?",
  "assert": {
    "type": "agent",
    "use": "agents:tests.validator-agent",
    "value": "Response should provide clear step-by-step instructions"
  }
}
```

### Scenario 7: With Options

Override connector or skip features:

```jsonl
{"id": "fast-model", "input": "Quick question", "options": {"connector": "deepseek.v3", "skip": {"history": true, "trace": true}}}
{"id": "scenario-test", "input": "Query users", "options": {"metadata": {"scenario": "filter"}}, "assert": {"type": "json_path", "path": "from", "value": "users"}}
```

### Scenario 8: With Before/After Hooks

Setup and teardown for each test:

```jsonl
{
  "id": "with-user-data",
  "input": "Show my expenses",
  "before": "env_test.Before",
  "after": "env_test.After",
  "assert": {
    "type": "contains",
    "value": "expense"
  }
}
```

### Scenario 9: Skip Test

Temporarily disable a test:

```jsonl
{
  "id": "wip-feature",
  "input": "New feature test",
  "skip": true
}
```

### Scenario 10: Dynamic Mode (Simulator)

Multi-turn testing with user simulator:

```jsonl
{
  "id": "coffee-order",
  "input": "I want to order coffee",
  "simulator": {
    "use": "tests.simulator-agent",
    "options": {
      "metadata": {
        "persona": "Regular customer",
        "goal": "Order a medium latte"
      }
    }
  },
  "checkpoints": [
    {
      "id": "greeting",
      "assert": {
        "type": "regex",
        "value": "(?i)(hello|hi|help)"
      }
    },
    {
      "id": "ask-size",
      "after": [
        "greeting"
      ],
      "assert": {
        "type": "regex",
        "value": "(?i)size"
      }
    },
    {
      "id": "confirm",
      "after": [
        "ask-size"
      ],
      "assert": {
        "type": "regex",
        "value": "(?i)confirm"
      }
    }
  ],
  "max_turns": 10
}
```

### Scenario 11: Dynamic Mode with Optional Checkpoint

Some checkpoints are optional:

```jsonl
{
  "id": "expense-flow",
  "input": "Submit expense",
  "simulator": {
    "use": "tests.simulator-agent",
    "options": {
      "metadata": {
        "persona": "New employee",
        "goal": "Submit $500 travel expense"
      }
    }
  },
  "checkpoints": [
    {
      "id": "ask-type",
      "assert": {
        "type": "regex",
        "value": "(?i)type"
      }
    },
    {
      "id": "suggest-category",
      "required": false,
      "assert": {
        "type": "contains",
        "value": "category"
      }
    },
    {
      "id": "confirm",
      "after": [
        "ask-type"
      ],
      "assert": {
        "type": "regex",
        "value": "(?i)confirm"
      }
    }
  ],
  "max_turns": 15
}
```

### Standard Mode Fields

| Field      | Type                           | Required | Description                           |
| ---------- | ------------------------------ | -------- | ------------------------------------- |
| `id`       | string                         | Yes      | Test case ID                          |
| `input`    | string \| Message \| []Message | Yes      | Test input                            |
| `assert`   | Assertion \| []Assertion       | No       | Assertion rules                       |
| `expected` | any                            | No       | Expected output (exact match)         |
| `user`     | string                         | No       | Override user ID for this test        |
| `team`     | string                         | No       | Override team ID for this test        |
| `metadata` | map                            | No       | Additional metadata for hooks         |
| `options`  | Options                        | No       | Context options                       |
| `timeout`  | string                         | No       | Override timeout (e.g., "30s")        |
| `skip`     | bool                           | No       | Skip this test                        |
| `before`   | string                         | No       | Before hook (e.g., `env_test.Before`) |
| `after`    | string                         | No       | After hook (e.g., `env_test.After`)   |

### Dynamic Mode Fields

| Field                         | Type   | Required | Description                            |
| ----------------------------- | ------ | -------- | -------------------------------------- |
| `id`                          | string | Yes      | Test case ID                           |
| `input`                       | string | Yes      | Initial user message                   |
| `simulator`                   | object | Yes      | Simulator configuration                |
| `simulator.use`               | string | Yes      | Simulator agent ID (no prefix)         |
| `simulator.options`           | object | No       | Simulator options                      |
| `simulator.options.metadata`  | map    | No       | Metadata (persona, goal, etc.)         |
| `simulator.options.connector` | string | No       | Override simulator connector           |
| `checkpoints`                 | array  | Yes      | Checkpoints to verify                  |
| `checkpoints[].id`            | string | Yes      | Checkpoint identifier                  |
| `checkpoints[].description`   | string | No       | Human-readable description             |
| `checkpoints[].assert`        | object | Yes      | Assertion to validate                  |
| `checkpoints[].after`         | array  | No       | Checkpoint IDs that must occur first   |
| `checkpoints[].required`      | bool   | No       | Is checkpoint required (default: true) |
| `max_turns`                   | int    | No       | Maximum turns (default: 20)            |
| `timeout`                     | string | No       | Override timeout (e.g., "2m")          |

### Options

The `options` field allows per-test-case configuration:

| Field                    | Type   | Description                                |
| ------------------------ | ------ | ------------------------------------------ |
| `connector`              | string | Override connector (e.g., `"deepseek.v3"`) |
| `mode`                   | string | Agent mode (default: `"chat"`)             |
| `search`                 | bool   | Enable/disable search mode                 |
| `disable_global_prompts` | bool   | Temporarily disable global prompts         |
| `metadata`               | map    | Custom data passed to hooks                |
| `skip`                   | object | Skip configuration (see below)             |

### Options.skip

| Field     | Type | Description             |
| --------- | ---- | ----------------------- |
| `history` | bool | Skip history loading    |
| `trace`   | bool | Skip trace logging      |
| `output`  | bool | Skip output to client   |
| `keyword` | bool | Skip keyword extraction |
| `search`  | bool | Skip auto search        |

### Input Types

| Type        | Description          | Example                                               |
| ----------- | -------------------- | ----------------------------------------------------- |
| `string`    | Simple text          | `"Hello world"`                                       |
| `Message`   | Single message       | `{"role": "user", "content": "..."}`                  |
| `[]Message` | Conversation history | `[{"role": "user", ...}, {"role": "assistant", ...}]` |

## Assertions

Use `assert` for flexible validation. If `assert` is defined, it takes precedence over `expected`.

### Static Assertions

| Type           | Description                   | Example                                                   |
| -------------- | ----------------------------- | --------------------------------------------------------- |
| `equals`       | Exact match                   | `{"type": "equals", "value": {"key": "val"}}`             |
| `contains`     | Output contains value         | `{"type": "contains", "value": "keyword"}`                |
| `not_contains` | Output does not contain value | `{"type": "not_contains", "value": "error"}`              |
| `json_path`    | Extract JSON path and compare | `{"type": "json_path", "path": "$.field", "value": true}` |
| `regex`        | Match regex pattern           | `{"type": "regex", "value": "\\d+"}`                      |
| `type`         | Check output type             | `{"type": "type", "value": "object"}`                     |
| `tool_called`  | Check if a tool was called    | `{"type": "tool_called", "value": "setup"}`               |
| `tool_result`  | Check tool execution result   | `{"type": "tool_result", "value": {"tool": "setup", "result": {"success": true}}}` |

### Assertion Fields

| Field     | Type   | Description                                              |
| --------- | ------ | -------------------------------------------------------- |
| `type`    | string | Assertion type (required)                                |
| `value`   | any    | Expected value or pattern                                |
| `path`    | string | JSON path for `json_path` type                           |
| `script`  | string | Script name for `script` type                            |
| `use`     | string | Agent/script ID for `agent` type (with `agents:` prefix) |
| `options` | object | Options for agent assertions                             |
| `message` | string | Custom failure message                                   |
| `negate`  | bool   | Invert the assertion result                              |

### Agent-Driven Assertions

For semantic or fuzzy validation using an LLM:

```jsonl
{
  "id": "T001",
  "input": "Hello",
  "assert": {
    "type": "agent",
    "use": "agents:tests.validator-agent",
    "value": "Response should be friendly and helpful"
  }
}
```

The validator agent receives the output and criteria, then returns `{"passed": true/false, "reason": "..."}`.

**How it works:**

1. The framework builds a validation request with the agent's response (including tool result messages)
2. The validator agent evaluates the response against the criteria
3. The validator returns a JSON response with `passed` and `reason`

**Output in test report (for checkpoints):**

```json
{
  "agent_validation": {
    "passed": true,
    "reason": "Response explicitly confirms setup completion",
    "criteria": "Response should be friendly and helpful",
    "input": "Hello! How can I help you today?",
    "response": {
      "passed": true,
      "reason": "Response explicitly confirms setup completion"
    }
  }
}
```

- `input`: The content sent to the validator (agent response + tool result messages)
- `response`: The raw JSON response from the validator agent
- `criteria`: The validation criteria from the test case

### Tool Assertions

For validating that specific tools were called and their results:

#### tool_called

Check if a specific tool was called:

```jsonl
{
  "id": "T001",
  "input": "Set up my expense system",
  "assert": {
    "type": "tool_called",
    "value": "setup"
  }
}
```

**Value formats:**

- **String**: Tool name (supports suffix matching, e.g., `"setup"` matches `"agents_expense_tools__setup"`)
- **Array**: Any of the specified tools must be called
- **Object**: Match tool name and optionally arguments

```jsonl
// Match any of these tools
{"type": "tool_called", "value": ["setup", "init"]}

// Match tool with specific arguments
{"type": "tool_called", "value": {"name": "setup", "arguments": {"action": "init"}}}
```

#### tool_result

Check the result of a tool execution:

```jsonl
{
  "id": "T001",
  "input": "Set up my expense system",
  "assert": {
    "type": "tool_result",
    "value": {
      "tool": "setup",
      "result": {
        "success": true
      }
    }
  }
}
```

**Result matching:**

- If `result` is omitted, only checks that the tool executed without error
- Supports partial matching (only specified fields are checked)
- Supports regex patterns with `regex:` prefix for string values

```jsonl
// Just check tool executed without error
{"type": "tool_result", "value": {"tool": "setup"}}

// Check specific result fields
{"type": "tool_result", "value": {"tool": "setup", "result": {"success": true}}}

// Use regex for message matching
{"type": "tool_result", "value": {"tool": "setup", "result": {"message": "regex:(?i)setup.*complete"}}}
```

### Script Assertions

For custom validation logic:

```jsonl
{
  "id": "T001",
  "input": "Test",
  "assert": {
    "type": "script",
    "script": "scripts.test.Validate"
  }
}
```

### Multiple Assertions

All assertions must pass:

```jsonl
{
  "id": "T001",
  "input": "Hello",
  "assert": [
    {
      "type": "contains",
      "value": "Hi"
    },
    {
      "type": "not_contains",
      "value": "error"
    },
    {
      "type": "json_path",
      "path": "status",
      "value": "ok"
    }
  ]
}
```

## File Attachments

Test inputs support file attachments using the `file://` protocol:

```jsonl
{
  "id": "T001",
  "input": {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "Analyze this image"
      },
      {
        "type": "image",
        "source": "file://fixtures/receipt.jpg"
      }
    ]
  }
}
```

Supported types: images (jpg, png, gif, webp), audio (wav, mp3), documents (pdf, doc, txt).

## Before/After Hooks

Hooks allow you to run setup and teardown code before and after tests. Hook scripts must be placed in the agent's `src/` directory with `_test.ts` suffix.

### Hook Types

| Hook        | Scope    | When Called           | Use Case                        |
| ----------- | -------- | --------------------- | ------------------------------- |
| `Before`    | Per-test | Before each test case | Create test data, setup context |
| `After`     | Per-test | After each test case  | Cleanup test data, log results  |
| `BeforeAll` | Global   | Once before all tests | Database migration, init        |
| `AfterAll`  | Global   | Once after all tests  | Global cleanup, report          |

### Execution Order

```
BeforeAll (global)
  ├─ Before (test 1)
  │    └─ Test 1 execution
  │    └─ After (test 1)
  ├─ Before (test 2)
  │    └─ Test 2 execution
  │    └─ After (test 2)
  └─ ...
AfterAll (global)
```

### Per-Test Hooks

Defined in JSONL, scripts located in agent's `src/` directory:

```jsonl
{
  "id": "T001",
  "input": "Test",
  "before": "env_test.Before",
  "after": "env_test.After"
}
```

### Global Hooks

Via CLI flags:

```bash
yao agent eval -i tests/inputs.jsonl --before env_test.BeforeAll --after env_test.AfterAll
```

### Hook Function Signatures

```typescript
// assistants/expense/src/env_test.ts

/**
 * Before - Called before each test case
 * @param ctx - Agent context with user/team info
 * @param testCase - The test case about to run
 * @returns any - Data passed to After hook (optional)
 */
export function Before(ctx: Context, testCase: TestCase): any {
  const userId = Process("models.user.Create", { name: "Test User" });
  return { userId }; // This data is passed to After
}

/**
 * After - Called after each test case (pass or fail)
 * @param ctx - Agent context
 * @param testCase - The test case that ran
 * @param result - Test result with status, output, duration
 * @param beforeData - Data returned from Before hook
 */
export function After(
  ctx: Context,
  testCase: TestCase,
  result: TestResult,
  beforeData: any
) {
  if (beforeData?.userId) {
    Process("models.user.Delete", beforeData.userId);
  }
  if (result.status === "failed") {
    console.log(`Test ${testCase.id} failed: ${result.error}`);
  }
}

/**
 * BeforeAll - Called once before all tests
 * @param ctx - Agent context
 * @param testCases - Array of all test cases
 * @returns any - Data passed to AfterAll hook (optional)
 */
export function BeforeAll(ctx: Context, testCases: TestCase[]): any {
  Process("models.migrate");
  return { initialized: true, count: testCases.length };
}

/**
 * AfterAll - Called once after all tests complete
 * @param ctx - Agent context
 * @param results - Array of all test results
 * @param beforeData - Data returned from BeforeAll hook
 */
export function AfterAll(ctx: Context, results: TestResult[], beforeData: any) {
  const passed = results.filter((r) => r.status === "passed").length;
  console.log(`Tests completed: ${passed}/${results.length} passed`);
  Process("models.cleanup");
}
```

### Hook Parameters

**Context** - Agent execution context:

```typescript
interface Context {
  locale: string; // Locale (e.g., "en-us")
  authorized: {
    user_id: string; // Test user ID
    team_id: string; // Test team ID
    constraints?: object; // Access constraints
  };
  metadata: object; // Custom metadata from test case
}
```

**TestCase** - Test case definition:

```typescript
interface TestCase {
  id: string; // Test case ID
  input: any; // Test input (string, Message, or Message[])
  assert?: object; // Assertion rules
  expected?: any; // Expected output
  user?: string; // Override user ID
  team?: string; // Override team ID
  metadata?: object; // Custom metadata
  options?: object; // Context options
  timeout?: string; // Timeout (e.g., "30s")
  skip?: boolean; // Skip flag
  before?: string; // Before hook reference
  after?: string; // After hook reference
}
```

**TestResult** - Test execution result:

```typescript
interface TestResult {
  id: string; // Test case ID
  status: string; // "passed" | "failed" | "error" | "skipped" | "timeout"
  input: any; // Actual input sent
  output: any; // Agent response
  expected?: any; // Expected output (if defined)
  error?: string; // Error message (if failed)
  duration_ms: number; // Execution time in milliseconds
  assertions?: object[]; // Assertion results
}
```

### Common Use Cases

**Database Setup/Teardown**:

```typescript
export function Before(ctx: Context, testCase: TestCase): any {
  // Create test records
  const user = Process("models.user.Create", {
    name: "Test",
    email: "test@example.com",
  });
  const expense = Process("models.expense.Create", {
    user_id: user.id,
    amount: 100,
  });
  return { user, expense };
}

export function After(
  ctx: Context,
  testCase: TestCase,
  result: TestResult,
  data: any
) {
  // Clean up in reverse order
  if (data?.expense) Process("models.expense.Delete", data.expense.id);
  if (data?.user) Process("models.user.Delete", data.user.id);
}
```

**Conditional Setup Based on Metadata**:

```typescript
export function Before(ctx: Context, testCase: TestCase): any {
  const scenario = testCase.metadata?.scenario || "default";

  if (scenario === "empty_db") {
    Process("models.expense.DeleteAll");
  } else if (scenario === "with_data") {
    Process("scripts.tests.seed.LoadTestData");
  }

  return { scenario };
}
```

**Logging and Debugging**:

```typescript
export function After(
  ctx: Context,
  testCase: TestCase,
  result: TestResult,
  data: any
) {
  if (result.status === "failed") {
    console.log("=== Test Failed ===");
    console.log("Test ID:", testCase.id);
    console.log("Input:", JSON.stringify(testCase.input));
    console.log("Output:", JSON.stringify(result.output));
    console.log("Error:", result.error);
  }
}
```

## Script Testing

Test agent handler scripts with the `t.assert` API:

```typescript
// assistants/expense/src/setup_test.ts
import { SystemReady } from "./setup";

export function TestSystemReady(t: TestingT, ctx: Context) {
  const result = SystemReady(ctx);

  t.assert.True(result.success, "Should succeed");
  t.assert.Equal(result.status, "ready", "Status should be ready");
  t.assert.NotNil(result.data, "Data should not be nil");
}

export function TestWithAgentAssertion(t: TestingT, ctx: Context) {
  const response = Process("agents.expense.Stream", ctx, messages);

  // Static assertion
  t.assert.Contains(response.content, "confirm");

  // Agent-driven assertion
  t.assert.Agent(response.content, "tests.validator-agent", {
    criteria: "Response should ask for confirmation",
  });
}
```

### Available Assertions

| Method                           | Description                    |
| -------------------------------- | ------------------------------ |
| `t.assert.True(value, msg)`      | Assert value is true           |
| `t.assert.False(value, msg)`     | Assert value is false          |
| `t.assert.Equal(a, b, msg)`      | Assert a equals b              |
| `t.assert.NotEqual(a, b, msg)`   | Assert a not equals b          |
| `t.assert.Nil(value, msg)`       | Assert value is null/undefined |
| `t.assert.NotNil(value, msg)`    | Assert value is not nil        |
| `t.assert.Contains(s, sub, msg)` | Assert string contains substr  |
| `t.assert.Len(arr, n, msg)`      | Assert array/string length     |
| `t.assert.Agent(resp, id, opts)` | Agent-driven assertion         |

## Dynamic Mode

For testing complex conversation flows where the path is unpredictable:

```jsonl
{
  "id": "coffee-order",
  "input": "I want to order coffee",
  "simulator": {
    "use": "tests.simulator-agent",
    "options": {
      "metadata": {
        "persona": "Customer ordering a latte",
        "goal": "Complete the coffee order"
      }
    }
  },
  "checkpoints": [
    {
      "id": "greeting",
      "description": "Agent greets customer",
      "assert": {
        "type": "regex",
        "value": "(?i)(hello|hi|help)"
      }
    },
    {
      "id": "ask_size",
      "description": "Agent asks for size",
      "after": [
        "greeting"
      ],
      "assert": {
        "type": "regex",
        "value": "(?i)size"
      }
    },
    {
      "id": "confirm",
      "description": "Agent confirms order",
      "after": [
        "ask_size"
      ],
      "assert": {
        "type": "regex",
        "value": "(?i)confirm"
      }
    }
  ],
  "max_turns": 10
}
```

### Console Output (Dynamic Mode)

```
► [coffee-order] (dynamic, 3 checkpoints)
ℹ Dynamic test: coffee-order (max 10 turns)
ℹ   Turn 1: User: I want to order coffee
ℹ   Turn 1: Agent: Hello! What can I get for you?
ℹ     ✓ checkpoint: greeting
ℹ   Turn 2: User: A medium latte please
ℹ   Turn 2: Agent: What size would you like?
ℹ     ✓ checkpoint: ask_size
ℹ   Turn 3: User: Medium
ℹ   Turn 3: Agent: Let me confirm: Medium latte. Correct?
ℹ     ✓ checkpoint: confirm
  └─ PASSED (3 turns, 3 checkpoints, 8.5s)
```

### Dynamic Mode Output Structure

Each turn in the output includes:

```typescript
interface TurnResult {
  turn: number; // Turn number (1-based)
  input: string; // User message
  output: any; // Agent response summary (for display)
  response: {
    // Full agent response (for detailed analysis)
    content: string; // LLM text content
    tool_calls: [
      {
        // Tool calls made
        tool: string; // Tool name
        arguments: any; // Call arguments
        result: any; // Execution result
      }
    ];
    next: any; // Next hook data
  };
  checkpoints_reached: string[]; // Checkpoint IDs reached
  duration_ms: number; // Execution time
}
```

### Checkpoint Result Structure

Each checkpoint in the output includes:

```typescript
interface CheckpointResult {
  id: string; // Checkpoint identifier
  reached: boolean; // Whether checkpoint was reached
  reached_at_turn?: number; // Turn number when reached (if reached)
  required: boolean; // Whether checkpoint is required
  passed: boolean; // Whether assertion passed
  message?: string; // Assertion result message
  agent_validation?: {
    // Agent assertion details (for type: "agent")
    passed: boolean; // Validator's determination
    reason: string; // Explanation from validator
    criteria: string; // Validation criteria checked
    input: any; // Content sent to validator
    response: {
      // Raw validator response
      passed: boolean;
      reason: string;
    };
  };
}
```

**Note**: For agent-based assertions (`type: "agent"`), the `agent_validation` field provides full transparency into the validation process. The `input` field contains the combined output (agent text response + tool result messages) that was validated.

## Output Formats

Determined by `-o` file extension:

| Extension | Format   | Description            |
| --------- | -------- | ---------------------- |
| `.jsonl`  | JSONL    | Streaming (default)    |
| `.json`   | JSON     | Complete structured    |
| `.md`     | Markdown | Human-readable         |
| `.html`   | HTML     | Interactive web report |

## Stability Analysis

Run each test multiple times to measure consistency:

```bash
yao agent eval -i tests/inputs.jsonl --runs 5 -o stability.json
```

| Pass Rate | Classification  |
| --------- | --------------- |
| 100%      | Stable          |
| 80-99%    | Mostly Stable   |
| 50-79%    | Unstable        |
| < 50%     | Highly Unstable |

## CI Integration

```bash
# Exit code: 0 = all passed, 1 = failures
yao agent eval -i tests/inputs.jsonl --fail-fast

# Run with parallel execution
yao agent eval -i tests/inputs.jsonl --parallel 4
```

### GitHub Actions Example

```yaml
- name: Run Agent Tests
  run: |
    yao agent eval -i assistants/expense/tests/inputs.jsonl \
      -u ci-user -t ci-team \
      --runs 3 \
      -o report.json

- name: Run Dynamic Tests
  run: |
    yao agent eval -i assistants/expense/tests/dynamic.jsonl \
      --simulator tests.simulator-agent \
      -v

- name: Run Script Tests
  run: |
    yao agent eval -i scripts.expense.setup -v
```

## Format Rules Reference

| Context                | Format                   | Example                                   |
| ---------------------- | ------------------------ | ----------------------------------------- |
| `-i agents:xxx` (CLI)  | Colon prefix             | `agents:tests.generator`                  |
| `-i scripts:xxx` (CLI) | Colon prefix             | `scripts:tests.gen.Generate`              |
| `-i scripts.xxx` (CLI) | Dot prefix (test mode)   | `scripts.expense.setup`                   |
| JSONL assertion `use`  | Prefix required          | `"use": "agents:tests.validator"`         |
| JSONL `simulator.use`  | No prefix (agent only)   | `"use": "tests.simulator-agent"`          |
| `--simulator` flag     | No prefix (agent only)   | `--simulator tests.simulator-agent`       |
| `t.assert.Agent()`     | No prefix (method-bound) | `t.assert.Agent(resp, "tests.validator")` |
| JSONL `before/after`   | No prefix (in src/)      | `"before": "env_test.Before"`             |
| `--before/--after`     | No prefix (in src/)      | `--before env_test.BeforeAll`             |

**Script input modes**:

- `scripts.xxx` (dot) - Run script tests (`*_test.ts` functions)
- `scripts:xxx` (colon) - Generate test cases from a script

## Built-in Test Agents

The framework provides three specialized agents for testing:

### Generator Agent (`tests.generator-agent`)

Generates test cases based on target agent description.

**package.yao**:

```json
{
  "name": "Test Case Generator",
  "connector": "gpt-4o",
  "description": "Generates test cases for agent testing",
  "options": { "temperature": 0.7 },
  "automated": true
}
```

**prompts.yml**:

```yaml
- role: system
  content: |
    You are a test case generator. Generate test cases based on the target agent.

    ## Input Format
    - `target_agent`: Agent info (id, description, tools)
    - `count`: Number of test cases (default: 5)
    - `focus`: Focus area (e.g., "edge-cases", "happy-path")

    ## Output Format
    JSON array of test cases:
    [
      {
        "id": "test-id",
        "input": "User message",
        "assert": [{"type": "contains", "value": "expected"}]
      }
    ]
```

**Usage**:

```bash
yao agent eval -i "agents:tests.generator-agent?count=10" -n assistants.expense
```

### Validator Agent (`tests.validator-agent`)

Validates agent responses for agent-driven assertions.

**package.yao**:

```json
{
  "name": "Response Validator",
  "connector": "gpt-4o",
  "description": "Validates responses against criteria",
  "options": { "temperature": 0 },
  "automated": true
}
```

**prompts.yml**:

```yaml
- role: system
  content: |
    You are a response validator. Evaluate whether the response meets the criteria.

    ## Input Format
    - `output`: The response to validate
    - `criteria`: The validation rules
    - `input`: Original input (optional)

    ## Output Format
    JSON object (no markdown):
    {"passed": true/false, "reason": "explanation"}

    ## Examples
    Input: {"output": "Paris is the capital", "criteria": "factually accurate"}
    Output: {"passed": true, "reason": "Statement is correct"}
```

**Usage in JSONL**:

```jsonl
{
  "id": "T001",
  "input": "Hello",
  "assert": {
    "type": "agent",
    "use": "agents:tests.validator-agent",
    "value": "Response should be friendly"
  }
}
```

**Usage in script tests**:

```typescript
t.assert.Agent(response, "tests.validator-agent", {
  criteria: "Response should be helpful",
});
```

### Simulator Agent (`tests.simulator-agent`)

Simulates user behavior for dynamic mode testing.

**package.yao**:

```json
{
  "name": "User Simulator",
  "connector": "gpt-4o",
  "description": "Simulates user behavior for dynamic testing",
  "options": { "temperature": 0.7 },
  "automated": true
}
```

**prompts.yml**:

```yaml
- role: system
  content: |
    You are a user simulator. Generate realistic user messages based on persona and goal.

    ## Input Format
    - `persona`: User description (e.g., "New employee")
    - `goal`: What user wants to achieve
    - `conversation`: Previous messages
    - `turn_number`: Current turn
    - `max_turns`: Maximum turns

    ## Output Format
    JSON object:
    {
      "message": "User response",
      "goal_achieved": false,
      "reasoning": "Strategy explanation"
    }

    ## Guidelines
    1. Stay in character
    2. Work toward the goal
    3. Be realistic (include natural variations)
    4. Set goal_achieved: true when done
```

**Usage in JSONL**:

```jsonl
{
  "id": "dynamic-test",
  "input": "I need help",
  "simulator": {
    "use": "tests.simulator-agent",
    "options": {
      "metadata": {
        "persona": "New employee",
        "goal": "Submit expense report"
      }
    }
  },
  "checkpoints": [
    {
      "id": "greeting",
      "assert": {
        "type": "regex",
        "value": "(?i)hello"
      }
    }
  ],
  "max_turns": 10
}
```

**Usage via CLI**:

```bash
yao agent eval -i tests/dynamic.jsonl --simulator tests.simulator-agent
```

## Extract Command

Extract test results from output JSONL file to individual Markdown or JSON files for human review:

```bash
# Extract to Markdown files (default)
yao agent extract output-20260127104118.jsonl

# Specify output directory
yao agent extract output.jsonl -o ./reports/

# Extract to JSON format
yao agent extract output.jsonl --format json
```

### Extract Command Options

| Flag       | Description                              | Default    |
| ---------- | ---------------------------------------- | ---------- |
| `-o`       | Output directory                         | same as input |
| `--format` | Output format: `markdown`, `json`        | `markdown` |

### Output Format (Markdown)

Each test result is extracted to a separate file:

```markdown
# T001-销售分析师-月末周五

**Status**: ✅ PASSED

**Duration**: 16743ms

## Input
(Full input content in markdown code block)

## Output
(Agent's response content)
```

This is useful for:
- Human review of agent outputs
- Comparing results across test runs
- Documentation and reporting

## Exit Codes

| Code | Description                                         |
| ---- | --------------------------------------------------- |
| 0    | All tests passed                                    |
| 1    | Tests failed, configuration error, or runtime error |

---

## File: agent/docs/configuration.md

# Assistant Configuration

## Directory Structure

```
assistants/
└── <assistant-id>/
    ├── package.yao          # Required: Configuration
    ├── prompts.yml          # Optional: Default prompts
    ├── prompts/             # Optional: Prompt presets
    │   ├── chat.yml
    │   └── task.yml
    ├── locales/             # Optional: Translations
    │   ├── en-us.yml
    │   └── zh-cn.yml
    ├── src/                 # Optional: Hook scripts
    │   └── index.ts
    └── mcps/                # Optional: MCP servers
        └── tools.mcp.yao
```

## package.yao

### Basic Fields

```json
{
  "name": "{{ name }}",
  "type": "assistant",
  "avatar": "/assets/avatar.png",
  "description": "{{ description }}",
  "connector": "gpt-4o",
  "tags": ["Category1", "Category2"],
  "sort": 1
}
```

| Field         | Type     | Description                          |
| ------------- | -------- | ------------------------------------ |
| `name`        | string   | Display name (supports i18n `{{ }}`) |
| `type`        | string   | Type: `assistant` (default)          |
| `avatar`      | string   | Avatar image path                    |
| `description` | string   | Description (supports i18n)          |
| `connector`   | string   | LLM connector ID                     |
| `tags`        | string[] | Categorization tags                  |
| `sort`        | number   | Display order                        |

### Connector Options

```json
{
  "connector": "gpt-4o",
  "connector_options": {
    "optional": true,
    "connectors": ["gpt-4o", "gpt-4o-mini", "claude-3"],
    "filters": ["tool_calls", "vision"]
  }
}
```

| Field        | Type     | Description                                  |
| ------------ | -------- | -------------------------------------------- |
| `optional`   | boolean  | Allow user to select connector               |
| `connectors` | string[] | Available connectors (empty = all)           |
| `filters`    | string[] | Required capabilities: `vision`, `audio`, `tool_calls`, `reasoning` |

### Generation Options

```json
{
  "options": {
    "temperature": 0.7,
    "max_tokens": 4096
  }
}
```

### Placeholder (UI Hints)

```json
{
  "placeholder": {
    "title": "{{ chat.title }}",
    "description": "{{ chat.description }}",
    "prompts": [
      "{{ chat.prompts.0 }}",
      "{{ chat.prompts.1 }}"
    ]
  }
}
```

### Visibility & Access

```json
{
  "public": true,
  "share": "team",
  "readonly": true,
  "built_in": true,
  "mentionable": true,
  "automated": false
}
```

| Field         | Type    | Description                       |
| ------------- | ------- | --------------------------------- |
| `public`      | boolean | Visible to all users              |
| `share`       | string  | Sharing scope: `private`, `team`  |
| `readonly`    | boolean | Prevent user modifications        |
| `built_in`    | boolean | System-managed assistant          |
| `mentionable` | boolean | Can be @mentioned in chat         |
| `automated`   | boolean | Can be triggered automatically    |

### Modes

```json
{
  "modes": ["chat", "task"],
  "default_mode": "task"
}
```

### MCP Servers

```json
{
  "mcp": {
    "servers": [
      "server-id",
      { "server_id": "tools", "tools": ["tool1", "tool2"] },
      { "server_id": "resources", "resources": ["uri://pattern"] }
    ]
  }
}
```

### Knowledge Base

```json
{
  "kb": {
    "collections": ["collection-id-1", "collection-id-2"]
  }
}
```

### Database Models

```json
{
  "db": {
    "models": ["model.name", "another.model"]
  }
}
```

### Uses (Wrapper Tools)

```json
{
  "uses": {
    "vision": "vision-agent",
    "audio": "audio-agent",
    "search": "disabled",
    "fetch": "mcp:fetcher"
  }
}
```

| Field    | Description                                        |
| -------- | -------------------------------------------------- |
| `vision` | Vision processing: `<agent-id>` or `mcp:<server>`  |
| `audio`  | Audio processing: `<agent-id>` or `mcp:<server>`   |
| `search` | Search: `disabled`, `<agent-id>`, or `mcp:<server>`|
| `fetch`  | HTTP fetching: `<agent-id>` or `mcp:<server>`      |

### Search Configuration

```json
{
  "search": {
    "web": {
      "provider": "tavily",
      "max_results": 10
    },
    "kb": {
      "threshold": 0.7,
      "graph": true
    },
    "db": {
      "max_results": 20
    },
    "citation": {
      "format": "[{index}]",
      "auto_inject_prompt": true
    }
  }
}
```

## Environment Variables

Use `$ENV.VAR_NAME` for sensitive values:

```json
{
  "connector": "$ENV.LLM_CONNECTOR"
}
```

## Complete Example

```json
{
  "name": "{{ name }}",
  "type": "assistant",
  "avatar": "/assets/assistant.png",
  "connector": "gpt-4o",
  "connector_options": {
    "optional": true,
    "connectors": ["gpt-4o", "gpt-4o-mini"],
    "filters": ["tool_calls"]
  },
  "mcp": {
    "servers": [{ "server_id": "tools", "tools": ["search", "calculate"] }]
  },
  "description": "{{ description }}",
  "options": { "temperature": 0.7 },
  "public": true,
  "placeholder": {
    "title": "{{ chat.title }}",
    "description": "{{ chat.description }}",
    "prompts": ["{{ chat.prompts.0 }}", "{{ chat.prompts.1 }}"]
  },
  "tags": ["Productivity"],
  "modes": ["chat", "task"],
  "default_mode": "chat",
  "sort": 1,
  "readonly": true,
  "mentionable": true
}
```

---

## File: agent/docs/context-api.md

# Context API

The `ctx` object provides access to messaging, memory, tracing, and MCP operations.

## Properties

```typescript
interface Context {
  chat_id: string;           // Chat session ID
  assistant_id: string;      // Assistant ID
  locale: string;            // User locale (e.g., "en-us")
  theme: string;             // UI theme
  route: string;             // Request route
  referer: string;           // Request source
  metadata: Record<string, any>;    // Custom metadata
  authorized: Record<string, any>;  // Auth info

  memory: Memory;            // Memory namespaces
  trace: Trace;              // Tracing API
  mcp: MCP;                  // MCP operations
  search: Search;            // Search API
  agent: Agent;              // Agent-to-Agent calls (A2A)
  llm: LLM;                  // Direct LLM calls
  sandbox?: Sandbox;         // Sandbox operations (optional)
}
```

## Messaging

### Send Complete Message

```typescript
ctx.Send({ type: "text", props: { content: "Hello!" } });
ctx.Send("Hello!");  // Shorthand for text
```

### Streaming Messages

```typescript
const msgId = ctx.SendStream("Starting...");
ctx.Append(msgId, " processing...");
ctx.Append(msgId, " done!");
ctx.End(msgId);
```

### Update Streaming Message

```typescript
const msgId = ctx.SendStream({ type: "loading", props: { message: "Loading..." } });
// ... do work ...
ctx.Replace(msgId, { type: "text", props: { content: "Complete!" } });
ctx.End(msgId);
```

### Merge Data

```typescript
const msgId = ctx.SendStream({ type: "status", props: { progress: 0 } });
ctx.Merge(msgId, { progress: 50 }, "props");
ctx.Merge(msgId, { progress: 100, status: "done" }, "props");
ctx.End(msgId);
```

### Set Field

```typescript
const msgId = ctx.SendStream({ type: "result", props: {} });
ctx.Set(msgId, "success", "props.status");
ctx.Set(msgId, { count: 10 }, "props.data");
ctx.End(msgId);
```

### Block Grouping

```typescript
const blockId = ctx.BlockID();
ctx.Send("Step 1", blockId);
ctx.Send("Step 2", blockId);
ctx.Send("Step 3", blockId);
ctx.EndBlock(blockId);
```

### ID Generators

```typescript
const msgId = ctx.MessageID();    // "M1", "M2", ...
const blockId = ctx.BlockID();    // "B1", "B2", ...
const threadId = ctx.ThreadID();  // "T1", "T2", ...
```

## Memory

Four-level hierarchical memory system:

| Namespace            | Scope        | Persistence |
| -------------------- | ------------ | ----------- |
| `ctx.memory.user`    | Per user     | Persistent  |
| `ctx.memory.team`    | Per team     | Persistent  |
| `ctx.memory.chat`    | Per chat     | Persistent  |
| `ctx.memory.context` | Per request  | Temporary   |

### Basic Operations

```typescript
// Get/Set
ctx.memory.user.Set("theme", "dark");
const theme = ctx.memory.user.Get("theme");

// With TTL (seconds)
ctx.memory.context.Set("temp", data, 300);

// Check/Delete
if (ctx.memory.chat.Has("topic")) {
  ctx.memory.chat.Del("topic");
}

// Get and delete atomically
const token = ctx.memory.context.GetDel("one_time_token");

// Collection operations
const keys = ctx.memory.user.Keys();
const count = ctx.memory.chat.Len();
ctx.memory.context.Clear();
```

### Counters

```typescript
const views = ctx.memory.user.Incr("page_views");
const credits = ctx.memory.user.Decr("credits", 5);
```

### Lists

```typescript
ctx.memory.chat.Push("history", [msg1, msg2]);
const last = ctx.memory.chat.Pop("queue");
const items = ctx.memory.chat.Pull("queue", 5);
const all = ctx.memory.chat.PullAll("queue");
```

### Sets

```typescript
ctx.memory.user.AddToSet("visited", ["/home", "/about"]);
```

### Array Access

```typescript
const len = ctx.memory.chat.ArrayLen("messages");
const first = ctx.memory.chat.ArrayGet("messages", 0);
const last = ctx.memory.chat.ArrayGet("messages", -1);
ctx.memory.chat.ArraySet("messages", 0, newMsg);
const slice = ctx.memory.chat.ArraySlice("messages", -10, -1);
const page = ctx.memory.chat.ArrayPage("messages", 1, 20);
const all = ctx.memory.chat.ArrayAll("messages");
```

## Trace

### Create Nodes

```typescript
const node = ctx.trace.Add(
  { query: "input data" },
  {
    label: "Processing",
    type: "process",
    icon: "play",
  }
);
```

### Logging

```typescript
ctx.trace.Info("Starting process");
ctx.trace.Debug("Variable: " + value);
ctx.trace.Warn("Deprecated feature");
ctx.trace.Error("Operation failed");

// Or on node
node.Info("Step completed");
```

### Node Lifecycle

```typescript
node.SetOutput({ result: data });
node.SetMetadata("duration", 1500);
node.Complete({ status: "done" });
// or
node.Fail("Error message");
```

### Parallel Nodes

```typescript
const nodes = ctx.trace.Parallel([
  { input: { url: "api1" }, option: { label: "API 1" } },
  { input: { url: "api2" }, option: { label: "API 2" } }
]);
```

### Child Nodes

```typescript
const parent = ctx.trace.Add({}, { label: "Parent" });
const child = parent.Add({}, { label: "Child" });
```

## MCP

### Tools

```typescript
// List tools
const tools = ctx.mcp.ListTools("server-id");

// Call single tool - returns parsed result directly
const result = ctx.mcp.CallTool("server-id", "tool-name", { arg: "value" });
console.log(result.field);  // Direct access to parsed data

// Call multiple sequentially - returns array of parsed results
const results = ctx.mcp.CallTools("server-id", [
  { name: "tool1", arguments: { a: 1 } },
  { name: "tool2", arguments: { b: 2 } }
]);
results.forEach(r => console.log(r));

// Call multiple in parallel - returns array of parsed results
const results = ctx.mcp.CallToolsParallel("server-id", [
  { name: "tool1", arguments: {} },
  { name: "tool2", arguments: {} }
]);
results.forEach(r => console.log(r));
```

### Cross-Server Tool Calls

```typescript
// Call tools across multiple MCP servers (like Promise.all)
const results = ctx.mcp.All([
  { mcp: "server1", tool: "search", arguments: { q: "query" } },
  { mcp: "server2", tool: "fetch", arguments: { id: 123 } }
]);

// First success wins (like Promise.any)
const results = ctx.mcp.Any([
  { mcp: "primary", tool: "search", arguments: { q: "query" } },
  { mcp: "backup", tool: "search", arguments: { q: "query" } }
]);

// First complete wins (like Promise.race)
const results = ctx.mcp.Race([
  { mcp: "region-us", tool: "ping", arguments: {} },
  { mcp: "region-eu", tool: "ping", arguments: {} }
]);

// Result structure
interface MCPToolResult {
  mcp: string;      // Server ID
  tool: string;     // Tool name
  result?: any;     // Parsed result content
  error?: string;   // Error if failed
}
```

### Resources

```typescript
const resources = ctx.mcp.ListResources("server-id");
const data = ctx.mcp.ReadResource("server-id", "resource://uri");
```

### Prompts

```typescript
const prompts = ctx.mcp.ListPrompts("server-id");
const prompt = ctx.mcp.GetPrompt("server-id", "prompt-name", { arg: "value" });
```

## Search

### Single Search

```typescript
// Web search
const webResult = ctx.search.Web("query", {
  limit: 10,
  sites: ["example.com"],
  time_range: "week"
});

// Knowledge base
const kbResult = ctx.search.KB("query", {
  collections: ["docs"],
  threshold: 0.7,
  graph: true
});

// Database
const dbResult = ctx.search.DB("query", {
  models: ["model.name"],
  wheres: [{ column: "status", value: "active" }],
  limit: 20
});
```

### Parallel Search

```typescript
// Wait for all
const results = ctx.search.All([
  { type: "web", query: "topic" },
  { type: "kb", query: "topic", collections: ["docs"] }
]);

// First success
const results = ctx.search.Any([
  { type: "web", query: "topic" },
  { type: "kb", query: "topic" }
]);

// First complete
const results = ctx.search.Race([
  { type: "web", query: "topic" },
  { type: "kb", query: "topic" }
]);
```

### Result Structure

```typescript
interface SearchResult {
  type: "web" | "kb" | "db";
  query: string;
  source: "hook" | "auto" | "user";
  items: {
    citation_id: string;
    title: string;
    url: string;
    content: string;
    score: number;
  }[];
  error?: string;
}
```

## Agent API

The `ctx.agent` object provides methods to call other agents from within hooks, enabling agent-to-agent communication (A2A).

### Single Agent Call

```typescript
// Basic call
const result = ctx.agent.Call("assistant-id", messages);

// With options and callback
const result = ctx.agent.Call("assistant-id", messages, {
  connector: "gpt-4o",
  mode: "chat",
  metadata: { source: "hook" },
  skip: { history: false, trace: false, output: false },
  onChunk: (msg) => {
    console.log("Received:", msg.type, msg.props);
    return 0; // 0 = continue, non-zero = stop
  }
});
```

### Agent Options

```typescript
interface AgentCallOptions {
  connector?: string;            // Override LLM connector
  mode?: string;                 // Agent mode ("chat", "task")
  metadata?: Record<string, any>; // Custom metadata passed to hooks
  skip?: {
    history?: boolean;           // Skip loading chat history
    trace?: boolean;             // Skip trace recording
    output?: boolean;            // Skip output to client
    keyword?: boolean;           // Skip keyword extraction
    search?: boolean;            // Skip search
    content_parsing?: boolean;   // Skip content parsing
  };
  onChunk?: (msg: Message) => number; // Callback (0=continue, non-zero=stop)
}
```

### Parallel Agent Calls

```typescript
// Wait for all agents to complete (like Promise.all)
const results = ctx.agent.All([
  { agent: "agent-1", messages: [...] },
  { agent: "agent-2", messages: [...] }
]);

// Return first successful result (like Promise.any)
const results = ctx.agent.Any([
  { agent: "agent-1", messages: [...] },
  { agent: "agent-2", messages: [...] }
]);

// Return first completed result (like Promise.race)
const results = ctx.agent.Race([
  { agent: "agent-1", messages: [...] },
  { agent: "agent-2", messages: [...] }
]);

// With global callback for all responses
const results = ctx.agent.All([
  { agent: "agent-1", messages: [...] },
  { agent: "agent-2", messages: [...] }
], {
  onChunk: (agentId, index, msg) => {
    console.log(`Agent ${agentId} [${index}]:`, msg.type);
    return 0;
  }
});
```

### Result Structure

```typescript
interface AgentResult {
  agent_id: string;
  response?: Response;
  content?: string;
  error?: string;
}
```

### Message Object (onChunk callback)

```typescript
interface Message {
  type: string;                // "text", "thinking", "tool_call", "error"
  props?: Record<string, any>; // e.g., { content: "Hello" }
  chunk_id?: string;           // C1, C2, ...
  message_id?: string;         // M1, M2, ...
  delta?: boolean;             // Incremental update flag
}
```

## Sandbox API

The `ctx.sandbox` object provides access to sandbox operations when the assistant is configured with a sandbox executor (e.g., Claude CLI). Only available when `sandbox` is configured in `package.yao`.

### Properties

```typescript
ctx.sandbox.workdir  // Workspace directory path (e.g., "/workspace")
```

### File Operations

```typescript
// Read file
const content = ctx.sandbox.ReadFile("config.json");

// Write file
ctx.sandbox.WriteFile("output.txt", "Hello World");

// List directory
const files = ctx.sandbox.ListDir("src");
files.forEach(f => console.log(f.name, f.is_dir, f.size));
```

### Command Execution

```typescript
// Execute command (returns stdout)
const output = ctx.sandbox.Exec(["npm", "test"]);

// Handle errors
try {
  ctx.sandbox.Exec(["git", "commit", "-m", "fix"]);
} catch (e) {
  console.error("Command failed:", e.message);
}
```

### FileInfo Structure

```typescript
interface FileInfo {
  name: string;      // File/directory name
  size: number;      // Size in bytes
  is_dir: boolean;   // True if directory
}
```

### Use Cases

```typescript
// Prepare workspace before execution
function Create(ctx, messages) {
  if (ctx.sandbox) {
    ctx.sandbox.WriteFile("config.json", JSON.stringify({ debug: true }));
  }
  return { messages };
}

// Post-process results
function Next(ctx, payload) {
  if (ctx.sandbox && !payload.error) {
    const files = ctx.sandbox.ListDir("output");
    return { data: { generated: files.map(f => f.name) } };
  }
  return null;
}
```

## LLM API

The `ctx.llm` object provides direct access to LLM connectors for streaming completions.

### Single LLM Call

```typescript
// Basic streaming call
const result = ctx.llm.Stream("gpt-4o", [
  { role: "user", content: "Hello" }
]);

// With options and callback
const result = ctx.llm.Stream("gpt-4o", messages, {
  temperature: 0.7,
  max_tokens: 2000,
  onChunk: (msg) => {
    console.log("Chunk:", msg.props?.content);
    return 0;
  }
});
```

### Parallel LLM Calls

```typescript
// Wait for all LLM calls (like Promise.all)
const results = ctx.llm.All([
  { connector: "gpt-4o", messages: [...] },
  { connector: "claude-3", messages: [...] }
]);

// Return first successful result (like Promise.any)
const results = ctx.llm.Any([
  { connector: "gpt-4o", messages: [...] },
  { connector: "claude-3", messages: [...] }
]);

// Return first completed result (like Promise.race)
const results = ctx.llm.Race([
  { connector: "gpt-4o", messages: [...] },
  { connector: "claude-3", messages: [...] }
]);

// With global callback
const results = ctx.llm.All([
  { connector: "gpt-4o", messages: [...] },
  { connector: "claude-3", messages: [...] }
], {
  onChunk: (connectorId, index, msg) => {
    console.log(`LLM ${connectorId} [${index}]:`, msg.type);
    return 0;
  }
});
```

### LLM Options

```typescript
interface LlmOptions {
  temperature?: number;
  max_tokens?: number;
  max_completion_tokens?: number;
  top_p?: number;
  presence_penalty?: number;
  frequency_penalty?: number;
  stop?: string | string[];
  user?: string;
  seed?: number;
  tools?: object[];
  tool_choice?: string | object;
  response_format?: { type: string; json_schema?: object };
  reasoning_effort?: string;
  onChunk?: (msg: Message) => number;
}
```

### Result Structure

```typescript
interface LlmResult {
  connector: string;
  response?: CompletionResponse;
  content?: string;
  error?: string;
}

---

## File: agent/docs/hooks.md

# Hooks

Hooks allow you to customize agent behavior at key points in the execution lifecycle.

## Lifecycle

```
User Input → Create Hook → LLM Call → Tool Execution → Next Hook → Response
```

## Create Hook

Called before LLM call. Use to preprocess messages, configure request, or delegate.

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  // Return null for default behavior
  return null;

  // Or return configuration
  return {
    messages,                    // Modified messages
    temperature: 0.7,            // Override temperature
    max_tokens: 2000,            // Override max tokens
    connector: "gpt-4o-mini",    // Override connector
    prompt_preset: "task",       // Select prompt preset
    disable_global_prompts: true,// Skip global prompts
    mcp_servers: [               // Add MCP servers
      { server_id: "tools", tools: ["search"] }
    ],
    uses: {                      // Override wrapper tools
      vision: "vision-agent",
      search: "disabled"
    },
    force_uses: true,            // Force use wrapper tools
    locale: "zh-cn",             // Override locale
    metadata: { key: "value" },  // Pass data to context
  };
}
```

### Delegation (Skip LLM)

Route to another agent immediately:

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  if (shouldDelegate(messages)) {
    return {
      delegate: {
        agent_id: "specialist.agent",
        messages: messages,
        options: { metadata: { source: "main" } }
      }
    };
  }
  return { messages };
}
```

## Next Hook

Called after LLM response and tool execution. Use to post-process or delegate.

```typescript
function Next(ctx: agent.Context, payload: agent.Payload): agent.Next {
  const { messages, completion, tools, error } = payload;

  // Handle errors
  if (error) {
    return { data: { status: "error", message: error } };
  }

  // Process tool results
  if (tools?.length > 0) {
    const results = tools.map(t => t.result);
    return { data: { status: "success", results } };
  }

  // Delegate based on response
  if (completion?.content?.includes("transfer")) {
    return {
      delegate: {
        agent_id: "transfer.agent",
        messages: payload.messages
      }
    };
  }

  // Return null for standard response
  return null;
}
```

### Payload Structure

```typescript
interface Payload {
  messages: Message[];           // Messages sent to LLM
  completion?: {
    content: string;             // LLM text response
    tool_calls?: ToolCall[];     // Tool calls from LLM
    usage?: UsageInfo;           // Token usage
  };
  tools?: ToolCallResponse[];    // Tool execution results
  error?: string;                // Error message
}

interface ToolCallResponse {
  toolcall_id: string;
  server: string;                // MCP server ID
  tool: string;                  // Tool name
  arguments?: any;               // Tool arguments
  result?: any;                  // Tool result
  error?: string;                // Tool error
}
```

### Return Values

```typescript
interface NextResponse {
  delegate?: {                   // Route to another agent
    agent_id: string;
    messages: Message[];
    options?: Record<string, any>;
  };
  data?: any;                    // Custom response data
  metadata?: Record<string, any>;// Debug metadata
}
```

## Sending Messages

Use `ctx` to send messages to the client:

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  // Send complete message
  ctx.Send({ type: "text", props: { content: "Processing..." } });

  // Streaming message
  const msgId = ctx.SendStream("Starting...");
  ctx.Append(msgId, " step 1...");
  ctx.Append(msgId, " step 2...");
  ctx.End(msgId);

  return { messages };
}
```

## Memory

Share data between hooks:

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  // Store in request-scoped memory
  ctx.memory.context.Set("start_time", Date.now());
  ctx.memory.context.Set("query", messages[0]?.content);

  return { messages };
}

function Next(ctx: agent.Context, payload: agent.Payload): agent.Next {
  // Retrieve data
  const startTime = ctx.memory.context.Get("start_time");
  const duration = Date.now() - startTime;

  return { data: { duration_ms: duration } };
}
```

## Tracing

Add trace nodes for debugging and UI:

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  const node = ctx.trace.Add(
    { query: messages[0]?.content },
    { label: "Preprocessing", type: "process", icon: "play" }
  );

  node.Info("Starting analysis");
  // ... processing ...
  node.Complete({ status: "done" });

  return { messages };
}
```

## Error Handling

```typescript
function Next(ctx: agent.Context, payload: agent.Payload): agent.Next {
  try {
    if (payload.error) {
      ctx.trace.Error(payload.error);
      return {
        data: { status: "error", message: "Something went wrong" }
      };
    }
    // ... normal processing
  } catch (e) {
    ctx.trace.Error(e.message);
    return { data: { status: "error", message: e.message } };
  }
}
```

## Multi-Agent Orchestration

```typescript
// Main agent delegates based on intent
function Next(ctx: agent.Context, payload: agent.Payload): agent.Next {
  const { tools } = payload;

  // Route based on tool result
  const intent = tools?.[0]?.result?.intent;
  const agentMap = {
    "search": "search.agent",
    "calculate": "calc.agent",
    "translate": "translate.agent"
  };

  if (intent && agentMap[intent]) {
    return {
      delegate: {
        agent_id: agentMap[intent],
        messages: payload.messages
      }
    };
  }

  return null;
}
```

## Complete Example

```typescript
import { agent } from "@yao/runtime";

function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  const query = messages[messages.length - 1]?.content || "";
  
  // Store for Next hook
  ctx.memory.context.Set("query", query);
  ctx.memory.context.Set("start", Date.now());

  // Add trace
  ctx.trace.Add({ query }, { label: "Create", type: "hook" });

  // Check if needs special handling
  if (query.toLowerCase().includes("urgent")) {
    return {
      messages,
      temperature: 0,
      prompt_preset: "task"
    };
  }

  return { messages };
}

function Next(ctx: agent.Context, payload: agent.Payload): agent.Next {
  const { completion, tools, error } = payload;
  const start = ctx.memory.context.Get("start");
  const duration = Date.now() - start;

  ctx.trace.Add(
    { duration },
    { label: "Next", type: "hook" }
  ).Complete();

  if (error) {
    return { data: { status: "error", error } };
  }

  if (tools?.length > 0) {
    return {
      data: {
        status: "success",
        response: completion?.content,
        tools: tools.map(t => ({ name: t.tool, result: t.result })),
        duration_ms: duration
      }
    };
  }

  return null;
}
```

---

## File: agent/docs/i18n.md

# Internationalization (i18n)

## Locale Files

Create `locales/` directory in the assistant:

```
assistants/my-assistant/
└── locales/
    ├── en-us.yml
    ├── zh-cn.yml
    └── ja.yml
```

## Locale File Format

```yaml
# locales/en-us.yml
name: My Assistant
chat:
  title: New Chat
  prompts:
    - What can you do?
    - Help me with a task
    - Tell me about yourself

messages:
  welcome: Welcome back!
  error: Something went wrong
  processing: Processing your request...
```

```yaml
# locales/zh-cn.yml
name: 我的助手
chat:
  title: 新对话
  prompts:
    - 你能做什么？
    - 帮我完成一个任务
    - 介绍一下你自己

messages:
  welcome: 欢迎回来！
  error: 出了点问题
  processing: 正在处理您的请求...
```

## Using Translations

### In package.yao

Use `{{ key }}` syntax:

```json
{
  "name": "{{ name }}",
  "description": "{{ description }}",
  "placeholder": {
    "title": "{{ chat.title }}",
    "description": "{{ chat.description }}",
    "prompts": [
      "{{ chat.prompts.0 }}",
      "{{ chat.prompts.1 }}",
      "{{ chat.prompts.2 }}"
    ]
  }
}
```

### In Prompts

```yaml
- role: system
  content: |
    You are {{ name }}.
    {{ description }}

    Respond in the user's language.
```

## Locale Detection

The system detects locale from:

1. Request header `Accept-Language`
2. User preference (stored in memory)
3. Default: `en-us`

### Override in Hook

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  // Get user preference
  const userLocale = ctx.memory.user.Get("preferred_locale");

  return {
    messages,
    locale: userLocale || "en-us"
  };
}
```

## Global Translations

Define global translations in `agent/locales/`:

```
agent/
└── locales/
    ├── en-us.yml
    └── zh-cn.yml
```

These are available to all assistants via the `__global__` namespace.

## Accessing Translations in Hooks

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  const locale = ctx.locale;  // e.g., "en-us"

  // Use locale for custom logic
  if (locale.startsWith("zh")) {
    return {
      messages,
      prompt_preset: "chinese"
    };
  }

  return { messages };
}
```

## Nested Keys

Access nested values with dot notation:

```yaml
# locales/en-us.yml
errors:
  validation:
    required: This field is required
    invalid: Invalid value
  network:
    timeout: Connection timed out
```

```json
{
  "placeholder": {
    "title": "{{ errors.validation.required }}"
  }
}
```

## Fallback Behavior

If a translation key is not found:

1. Try the requested locale
2. Fall back to `en-us`
3. Return the key itself if not found

## Dynamic Locale Content

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  // Add locale-specific system message
  const localeGreeting = {
    "en-us": "Hello! How can I help you?",
    "zh-cn": "你好！有什么可以帮您的？",
    "ja": "こんにちは！何かお手伝いできますか？"
  };

  const greeting = localeGreeting[ctx.locale] || localeGreeting["en-us"];

  return {
    messages: [
      { role: "system", content: `Greeting: ${greeting}` },
      ...messages
    ]
  };
}
```

## Best Practices

1. **Keep keys consistent** - Use the same keys across all locale files
2. **Use nested structure** - Organize related translations together
3. **Provide fallbacks** - Always have `en-us` as the base locale
4. **Test all locales** - Verify translations render correctly
5. **Use context variables** - Combine with `$CTX.locale` in prompts

---

## File: agent/docs/mcp.md

# MCP Integration

Model Context Protocol (MCP) enables tool integration with external services.

## Directory Structure

Assistants can define their own namespaced MCP servers in the `mcps/` directory:

```
assistants/
└── my-assistant/
    ├── package.yao
    └── mcps/
        ├── tools.mcp.yao           # → agents.my-assistant.tools
        ├── calculator.mcp.yao      # → agents.my-assistant.calculator
        └── mapping/
            └── tools/
                └── schemes/
                    ├── search.in.yao
                    └── search.out.yao
```

MCP servers are automatically loaded with `agents.<assistant-id>.` prefix.

## Defining MCP Servers

Create `mcps/tools.mcp.yao` in the assistant directory:

```json
{
  "label": "Tools",
  "description": "Custom tools for the assistant",
  "transport": "process",
  "tools": {
    "search": "scripts.tools.Search",
    "create": "models.data.Create"
  }
}
```

### Transport Types

**Process (Yao Internal)**

Map Yao Processes directly to MCP tools:

```json
{
  "transport": "process",
  "tools": {
    "search": "models.data.Paginate",
    "create": "models.data.Create"
  },
  "resources": {
    "detail": "models.data.Find"
  }
}
```

**STDIO (Local Server)**

```json
{
  "transport": "stdio",
  "command": "python",
  "arguments": ["mcp_server.py"],
  "env": { "API_KEY": "$ENV.API_KEY" }
}
```

**HTTP (REST API)**

```json
{
  "transport": "http",
  "url": "https://mcp.example.com/api",
  "authorization_token": "$ENV.TOKEN"
}
```

**SSE (Server-Sent Events)**

```json
{
  "transport": "sse",
  "url": "https://mcp.example.com/events",
  "authorization_token": "$ENV.TOKEN"
}
```

## Configuring in package.yao

### All Tools

```json
{
  "mcp": {
    "servers": ["tools"]
  }
}
```

### Specific Tools

```json
{
  "mcp": {
    "servers": [{ "server_id": "tools", "tools": ["search", "calculate"] }]
  }
}
```

### With Resources

```json
{
  "mcp": {
    "servers": [
      {
        "server_id": "data",
        "tools": ["query"],
        "resources": ["data://users/*"]
      }
    ]
  }
}
```

## Dynamic Configuration in Hooks

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  return {
    messages,
    mcp_servers: [
      { server_id: "tools", tools: ["search"] },
      { server_id: "data", resources: ["data://reports"] },
    ],
  };
}
```

## Using MCP in Hooks

### List Available Tools

```typescript
const tools = ctx.mcp.ListTools("server-id");
// { tools: [{ name: "search", description: "...", inputSchema: {...} }] }
```

### Call Tool

```typescript
// Returns parsed result directly - no wrapper object
const result = ctx.mcp.CallTool("server-id", "search", {
  query: "example",
  limit: 10,
});
console.log(result.items);  // Direct access to parsed data
```

### Batch Tool Calls

```typescript
// Sequential - returns array of parsed results
const results = ctx.mcp.CallTools("server-id", [
  { name: "step1", arguments: { input: "a" } },
  { name: "step2", arguments: { input: "b" } },
]);
results.forEach(r => console.log(r));

// Parallel - returns array of parsed results
const results = ctx.mcp.CallToolsParallel("server-id", [
  { name: "api1", arguments: {} },
  { name: "api2", arguments: {} },
]);
results.forEach(r => console.log(r));
```

### Read Resources

```typescript
const resources = ctx.mcp.ListResources("server-id");
const data = ctx.mcp.ReadResource("server-id", "data://users/123");
```

### Get Prompts

```typescript
const prompts = ctx.mcp.ListPrompts("server-id");
const prompt = ctx.mcp.GetPrompt("server-id", "system", { role: "helper" });
```

### Cross-Server Tool Calls

Call tools across multiple MCP servers concurrently:

```typescript
// Wait for all (like Promise.all)
const results = ctx.mcp.All([
  { mcp: "server1", tool: "search", arguments: { q: "query" } },
  { mcp: "server2", tool: "analyze", arguments: { data: "input" } }
]);

// First success (like Promise.any) - good for fallback
const results = ctx.mcp.Any([
  { mcp: "primary", tool: "fetch", arguments: { id: 1 } },
  { mcp: "backup", tool: "fetch", arguments: { id: 1 } }
]);

// First complete (like Promise.race) - good for latency
const results = ctx.mcp.Race([
  { mcp: "region-us", tool: "ping", arguments: {} },
  { mcp: "region-eu", tool: "ping", arguments: {} }
]);

// Access results
results.forEach(r => {
  if (r.error) {
    console.log(`${r.mcp}/${r.tool} failed: ${r.error}`);
  } else {
    console.log(`${r.mcp}/${r.tool} result:`, r.result);
  }
});
```

## Tool Schema Mapping

Define input schemas for process transport tools:

```
mcps/
└── mapping/
    └── <server-id>/
        └── schemes/
            ├── search.in.yao      # Input schema
            └── search.out.yao     # Output schema (optional)
```

**mapping/tools/schemes/search.in.yao**

```json
{
  "type": "object",
  "description": "Search data",
  "properties": {
    "keyword": { "type": "string" },
    "page": { "type": "integer" }
  },
  "x-process-args": [":arguments"]
}
```

The `x-process-args` maps MCP arguments to Yao Process parameters:

- `":arguments"` - Pass entire arguments object
- `"$args.field"` - Extract specific field

### Schema with Nested Objects

```json
{
  "type": "object",
  "description": "Extract structured data from input",
  "properties": {
    "intent": {
      "type": "string",
      "enum": ["query", "create", "update"],
      "description": "Operation intent"
    },
    "items": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": { "type": "string" },
          "value": { "type": "number" }
        },
        "required": ["name", "value"]
      }
    }
  },
  "required": ["intent"],
  "x-process-args": [":arguments"]
}
```

## Using Assistant Models in MCP

MCP tools can reference assistant's own models:

**mcps/data.mcp.yao**

```json
{
  "label": "Data Tools",
  "transport": "process",
  "tools": {
    "list_orders": "models.agents.my-assistant.order.Paginate",
    "get_order": "models.agents.my-assistant.order.Find",
    "create_order": "models.agents.my-assistant.order.Create",
    "custom_query": "agents.my-assistant.orders.Query"
  }
}
```

See [Models](models.md) for defining assistant models.

## Error Handling

```typescript
function Next(ctx: agent.Context, payload: agent.Payload): agent.Next {
  const { tools } = payload;

  if (tools) {
    for (const tool of tools) {
      if (tool.error) {
        ctx.trace.Error(`Tool ${tool.tool} failed: ${tool.error}`);
        // Handle error
      } else {
        // Process result
        console.log(tool.result);
      }
    }
  }

  return null;
}
```

## Complete Example

**mcps/calculator.mcp.yao**

```json
{
  "label": "Calculator",
  "description": "Math operations",
  "transport": "process",
  "tools": {
    "add": "scripts.math.Add",
    "multiply": "scripts.math.Multiply"
  }
}
```

**package.yao**

```json
{
  "name": "Math Assistant",
  "connector": "gpt-4o",
  "mcp": {
    "servers": [{ "server_id": "calculator", "tools": ["add", "multiply"] }]
  }
}
```

**src/index.ts**

```typescript
function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  // Check if calculation is needed
  const query = messages[messages.length - 1]?.content || "";
  if (/\d+\s*[\+\-\*\/]\s*\d+/.test(query)) {
    // Enable calculator
    return {
      messages,
      mcp_servers: [{ server_id: "calculator" }],
    };
  }
  return { messages };
}

function Next(ctx: agent.Context, payload: agent.Payload): agent.Next {
  const { tools } = payload;

  if (tools?.length > 0) {
    const calcResult = tools.find((t) => t.server === "calculator");
    if (calcResult?.result) {
      return {
        data: {
          answer: calcResult.result,
          expression: calcResult.arguments,
        },
      };
    }
  }

  return null;
}
```

---

## File: agent/docs/models.md

# Assistant Models

Assistants can define their own namespaced data models in the `models/` directory. These models are automatically loaded with the `agents.<assistant-id>.` prefix and use isolated database tables.

## Directory Structure

```
assistants/
└── my-assistant/
    ├── package.yao
    └── models/
        ├── order.mod.yao       # → agents.my-assistant.order
        ├── item.mod.yao        # → agents.my-assistant.item
        └── nested/
            └── log.mod.yao     # → agents.my-assistant.nested.log
```

## Model Definition

Standard Yao model definition with automatic table prefixing.

**models/order.mod.yao**

```json
{
  "name": "Order",
  "label": "Order Record",
  "description": "Customer orders",
  "table": {
    "name": "order",
    "comment": "Order records"
  },
  "columns": [
    {
      "name": "id",
      "type": "ID",
      "label": "ID",
      "primary": true
    },
    {
      "name": "order_no",
      "type": "string",
      "label": "Order Number",
      "length": 100,
      "nullable": false,
      "unique": true,
      "index": true
    },
    {
      "name": "customer_id",
      "type": "string",
      "label": "Customer ID",
      "length": 255,
      "nullable": false,
      "index": true
    },
    {
      "name": "total_amount",
      "type": "decimal",
      "label": "Total Amount",
      "precision": 15,
      "scale": 2,
      "nullable": false
    },
    {
      "name": "status",
      "type": "enum",
      "label": "Status",
      "option": ["pending", "confirmed", "shipped", "completed", "cancelled"],
      "default": "pending",
      "nullable": false,
      "index": true
    },
    {
      "name": "metadata",
      "type": "json",
      "label": "Metadata",
      "nullable": true
    }
  ],
  "relations": {
    "items": {
      "type": "hasMany",
      "model": "item",
      "key": "order_id",
      "foreign": "id"
    }
  },
  "indexes": [
    {
      "name": "idx_customer_status",
      "columns": ["customer_id", "status"],
      "type": "index"
    }
  ],
  "option": {
    "timestamps": true,
    "soft_deletes": true
  }
}
```

## Table Naming

Tables are automatically prefixed with `agents_<assistant-id>_`:

| Assistant ID | Model File             | Model ID                 | Table Name               |
| ------------ | ---------------------- | ------------------------ | ------------------------ |
| `expense`    | `models/order.mod.yao` | `agents.expense.order`   | `agents_expense_order`   |
| `tests.demo` | `models/user.mod.yao`  | `agents.tests.demo.user` | `agents_tests_demo_user` |

## Using Models

### In Hooks

```typescript
import { Process } from "@yao/runtime";

function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create {
  // Query assistant's own model
  const orders = Process("models.agents.my-assistant.order.Paginate", {
    wheres: [{ column: "status", value: "pending" }],
    limit: 10,
  });

  return { messages };
}
```

### In MCP Tools

```json
{
  "transport": "process",
  "tools": {
    "list_orders": "models.agents.my-assistant.order.Paginate",
    "get_order": "models.agents.my-assistant.order.Find",
    "create_order": "models.agents.my-assistant.order.Create",
    "update_order": "models.agents.my-assistant.order.Update"
  }
}
```

### In Scripts

**src/orders.ts**

```typescript
import { Process } from "@yao/runtime";

export function ListPending(): any[] {
  return Process("models.agents.my-assistant.order.Get", {
    wheres: [{ column: "status", value: "pending" }],
    orders: [{ column: "created_at", option: "desc" }],
  });
}

export function CreateOrder(data: any): any {
  return Process("models.agents.my-assistant.order.Create", data);
}

export function UpdateStatus(id: number, status: string): any {
  return Process("models.agents.my-assistant.order.Update", id, { status });
}
```

## Column Types

| Type         | Description                | Options                 |
| ------------ | -------------------------- | ----------------------- |
| `ID`         | Auto-increment primary key | `primary: true`         |
| `string`     | VARCHAR                    | `length` (default: 255) |
| `text`       | TEXT                       | -                       |
| `integer`    | INT                        | -                       |
| `bigInteger` | BIGINT                     | -                       |
| `float`      | FLOAT                      | `precision`, `scale`    |
| `decimal`    | DECIMAL                    | `precision`, `scale`    |
| `boolean`    | BOOLEAN                    | -                       |
| `date`       | DATE                       | -                       |
| `datetime`   | DATETIME                   | -                       |
| `timestamp`  | TIMESTAMP                  | -                       |
| `json`       | JSON/JSONB                 | -                       |
| `enum`       | ENUM                       | `option: [...]`         |

## Column Options

| Option      | Type      | Description       |
| ----------- | --------- | ----------------- |
| `nullable`  | `boolean` | Allow NULL values |
| `default`   | `any`     | Default value     |
| `unique`    | `boolean` | Unique constraint |
| `index`     | `boolean` | Create index      |
| `primary`   | `boolean` | Primary key       |
| `length`    | `integer` | String length     |
| `precision` | `integer` | Decimal precision |
| `scale`     | `integer` | Decimal scale     |
| `comment`   | `string`  | Column comment    |

## Relations

```json
{
  "relations": {
    "items": {
      "type": "hasMany",
      "model": "item",
      "key": "order_id",
      "foreign": "id"
    },
    "customer": {
      "type": "hasOne",
      "model": "customer",
      "key": "id",
      "foreign": "customer_id"
    }
  }
}
```

| Type             | Description                   |
| ---------------- | ----------------------------- |
| `hasOne`         | One-to-one relationship       |
| `hasMany`        | One-to-many relationship      |
| `hasOneThrough`  | Has one through intermediate  |
| `hasManyThrough` | Has many through intermediate |

## Model Options

```json
{
  "option": {
    "timestamps": true,
    "soft_deletes": true,
    "permission": true
  }
}
```

| Option         | Description                            |
| -------------- | -------------------------------------- |
| `timestamps`   | Add `created_at`, `updated_at` columns |
| `soft_deletes` | Add `deleted_at` for soft delete       |
| `permission`   | Enable permission checks               |

## Process Reference

Common model processes:

| Process       | Description      | Arguments                   |
| ------------- | ---------------- | --------------------------- |
| `Find`        | Get by ID        | `id`, `query?`              |
| `Get`         | Get records      | `query`                     |
| `Paginate`    | Paginated list   | `query`, `page`, `pagesize` |
| `Create`      | Create record    | `data`                      |
| `Update`      | Update record    | `id`, `data`                |
| `Save`        | Create or update | `data`                      |
| `Delete`      | Delete record    | `id`                        |
| `Destroy`     | Hard delete      | `id`                        |
| `Insert`      | Batch insert     | `columns`, `rows`           |
| `UpdateWhere` | Batch update     | `query`, `data`             |
| `DeleteWhere` | Batch delete     | `query`                     |

## Migration

Models are automatically migrated when Yao starts. The migration:

1. Creates tables if not exist
2. Adds new columns
3. Creates indexes
4. Does NOT drop columns (safe migration)

To force schema sync:

```bash
yao migrate --reset  # Warning: drops and recreates tables
```

## Example: Complete Assistant with Models

**assistants/inventory/package.yao**

```json
{
  "name": "Inventory Assistant",
  "connector": "gpt-4o",
  "mcp": {
    "servers": [{ "server_id": "inventory" }]
  }
}
```

**assistants/inventory/models/product.mod.yao**

```json
{
  "name": "Product",
  "table": { "name": "product" },
  "columns": [
    { "name": "id", "type": "ID", "primary": true },
    { "name": "sku", "type": "string", "length": 50, "unique": true },
    { "name": "name", "type": "string", "length": 200 },
    { "name": "quantity", "type": "integer", "default": 0 },
    { "name": "price", "type": "decimal", "precision": 10, "scale": 2 }
  ],
  "option": { "timestamps": true }
}
```

**assistants/inventory/mcps/inventory.mcp.yao**

```json
{
  "label": "Inventory",
  "transport": "process",
  "tools": {
    "list_products": "models.agents.inventory.product.Paginate",
    "get_product": "models.agents.inventory.product.Find",
    "update_stock": "agents.inventory.stock.Update"
  }
}
```

**assistants/inventory/src/stock.ts**

```typescript
import { Process } from "@yao/runtime";

export function Update(args: { sku: string; quantity: number }): any {
  const product = Process("models.agents.inventory.product.Get", {
    wheres: [{ column: "sku", value: args.sku }],
    limit: 1,
  });

  if (!product || product.length === 0) {
    throw new Error(`Product not found: ${args.sku}`);
  }

  return Process("models.agents.inventory.product.Update", product[0].id, {
    quantity: args.quantity,
  });
}
```