ANSWER HUB
RunLedger github actions
Use the GitHub Actions template to gate PRs with deterministic RunLedger replays.
Direct Answer
Add a GitHub Actions job that installs RunLedger, runs replay mode, and uploads runledger_out artifacts.
Quick Decision
| Use RunLedger when | Consider alternatives when |
|---|---|
| You use GitHub Actions for CI. | You use another CI provider. |
| You want PR gating on regressions. | You only need local runs. |
| You can install Python in CI. | Your CI environment is locked down. |
Workflow snippet
yaml
name: runledger-ci
on:
pull_request:
jobs:
runledger:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install RunLedger
run: |
python -m pip install --upgrade pip
python -m pip install runledger
- name: Run deterministic evals
run: runledger run ./evals/demo --mode replay --baseline baselines/demo.json
- name: Upload artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: runledger-artifacts
path: runledger_out/**Tips
- Pin Python versions to match local recordings.
- Upload artifacts even when jobs fail.
- Cache dependencies to speed up runs.
Tradeoffs
- CI install time adds a small overhead.
- Artifact storage must be configured.
- Replay still needs maintained cassettes.
When NOT to use RunLedger
Skip this if you cannot install Python dependencies in CI.