# RunLedger API Reference (CLI + Config)

CLI commands
- runledger init: scaffold a demo suite.
- runledger run: execute a suite in record, replay, or live mode.
- runledger diff: compare a run to a baseline.
- runledger baseline promote: create a baseline from a passing run.

```bash
# Initialize a demo suite
runledger init
# Record live tool calls to cassettes
runledger run ./evals/demo --mode record
# Replay deterministically with a baseline
runledger run ./evals/demo --mode replay --baseline baselines/demo.json
# Diff a run against a baseline
runledger diff --baseline baselines/demo.json --run runledger_out/demo/<run_id>
# Promote a run to a baseline
runledger baseline promote --from runledger_out/demo/<run_id> --to baselines/demo.json
```

suite.yaml (key fields)
```yaml
suite_name: support-triage            # stable suite id for CI
agent_command: ["python", "agent.py"]    # agent launch command
mode: replay                           # record | replay | live
cases_path: cases                      # directory of cases
tool_registry:
  - search_docs                         # allowed tool names
  - create_issue
assertions:
  - type: json_schema                   # schema enforced on final output
    schema_path: schema.json
budgets:
  max_wall_ms: 20000                    # wall time cap
  max_tool_calls: 10                    # tool call cap
  max_tool_errors: 0                    # fail on any tool error
baseline_path: baselines/support.json  # regression gate baseline
```

cases/*.yaml (key fields)
```yaml
id: t1                                 # case id
description: "triage a login ticket"
input:
  ticket: "User cannot login"          # input forwarded to agent
  context:
    plan: "pro"
cassette: cassettes/t1.jsonl           # cassette used in replay
assertions:
  - type: required_fields              # per-case assertion override
    fields: ["category", "reply"]
budgets:
  max_wall_ms: 5000                    # per-case budget override
```

Protocol (JSONL over stdio)
- Runner -> Agent: task_start, tool_result
- Agent -> Runner: tool_call, final_output, optional log and task_error

```jsonc
{ "type": "task_start", "task_id": "t1", "input": { "ticket": "..." } }   // runner starts a task
{ "type": "tool_call", "name": "search_docs", "call_id": "c1", "args": { "q": "..." } } // agent requests a tool
{ "type": "tool_result", "call_id": "c1", "ok": true, "result": { "hits": [] } }        // runner returns tool output
{ "type": "final_output", "output": { "category": "billing", "reply": "..." } }         // agent final response
```

Reference links
- CLI and config: https://runledger.io/reference.html
- Golden Path: https://runledger.io/answers/golden-path