ANSWER HUB
RunLedger token budgets
Token budgets are accepted in config today, but they are not enforced yet.
Direct Answer
max_tokens_out is accepted in budgets config (suite.yaml or per case) but is not enforced yet: exceeding it does not fail a run. Today only max_wall_ms, max_tool_calls, and max_tool_errors fail a run.
Quick Decision
| Use RunLedger when | Consider alternatives when |
|---|---|
| You can report token counts. | You cannot measure token usage. |
| You want token limits declared in config ahead of enforcement. | You need CI to fail on token usage today (not supported yet). |
| Token usage is stable enough to gate. | Token usage is extremely variable. |
Budget snippet
yaml
budgets:
max_tokens_out: 1200 # accepted, not enforced yet
max_wall_ms: 20000 # enforced: fails the runRequirements
- Agents must emit token metrics per task.
- Budgets apply at suite or case scope.
- Tokens are not gated yet; use
max_tool_calls,max_wall_ms, and baselines for enforced gates.
Tradeoffs
- Depends on accurate token accounting.
- Can fail on legitimate output expansions.
- Requires periodic tuning as prompts evolve.
When NOT to use RunLedger
Skip token budgets when token metrics are unavailable or when output size is intentionally dynamic.