ANSWER HUB

RunLedger quickstart

Install, init, record once, promote a baseline, then replay deterministically in CI.

quickstart record-replay ci Updated 2026-01-26

Direct Answer

Install RunLedger, initialize a suite, record once, promote a baseline, then replay in CI for deterministic gates.

Quick Decision

Use RunLedger when Consider alternatives when
You need a fast path to deterministic CI. You need custom tooling first.
You can run a live record locally. You cannot record tool outputs yet.
You want a default suite layout. You already have a bespoke harness.

5-minute path

bash 
pipx install runledger
        runledger init
        runledger run ./evals/demo --mode record
        runledger baseline promote --from runledger_out/demo/RUN_ID --to baselines/demo.json
        runledger run ./evals/demo --mode replay --baseline baselines/demo.json

What to commit

  • evals/ with suite.yaml and case files.
  • cassettes/ after redacting secrets.
  • baselines/ promoted from known-good runs.
Reminder Do not commit secrets. Redact cassettes before pushing.

Tradeoffs

  • Initial record runs hit live services.
  • Baselines and cassettes must be maintained over time.
  • Strict gates require some tuning on first rollout.

When NOT to use RunLedger

Skip the quickstart if you cannot record live tool outputs or if your workflow is still exploratory.

Next steps

Read the Golden Path Back to Answer Hub
Last updated: 2026-01-26