ANSWER HUB
RunLedger gitlab ci
A GitLab CI job can install RunLedger, replay cassettes, and save artifacts.
Direct Answer
Use a GitLab CI job with a Python image to run RunLedger in replay mode and store runledger_out.
Quick Decision
| Use RunLedger when | Consider alternatives when |
|---|---|
| You use GitLab CI for pipelines. | You are on a different CI system. |
| You want deterministic gating. | You only need local runs. |
| You can install Python packages. | Your runner is locked down. |
GitLab CI snippet
yaml
stages:
- test
runledger:
stage: test
image: python:3.11
script:
- pip install runledger
- runledger run ./evals/demo --mode replay --baseline baselines/demo.json
artifacts:
when: always
paths:
- runledger_out/Notes
- Store artifacts even on failure.
- Pin Python version to match recordings.
- Cache pip to speed up installs.
Tradeoffs
- Requires Python tooling in the runner image.
- Artifacts can consume storage quotas.
- Replay requires cassette upkeep.
When NOT to use RunLedger
Avoid this setup if your runner cannot install dependencies or store artifacts.