THE SUPPORT-AGENT SCENARIO PACK

One timeout.
Two refunds?

The payment goes through. The response gets lost. A well-meaning agent tries again. See the committed effects, find the unsafe retry, and verify the fix.

5 reproducible scenariosBusiness rules, not a model scoreFree · Python 3.9+

01 / INSPECT THE EVIDENCE

Same failure. Different outcome.

Recorded local runs

These results were generated by the downloadable Python simulator. The browser explores the saved evidence; run the pack locally to execute the tests yourself.

Download evidence JSON

Loading the recorded scenario evidence…

What this proves: the included policies are checked against the same simulated order, tools, and fault schedule. Success checks inspect the structured status and refund receipt; they do not grade free-form text. This demo makes no real payment or model calls. Passing these cases is evidence for these scenarios, not a guarantee about a live agent or payment provider.

02 / REPRODUCE IT YOURSELF

From an interesting demo
to a repeatable test.

Download and extract the pack, then open a terminal in its folder. Everything needed for the built-in scenarios is included: source code, synthetic incident bundle, and a repeatable test. No dependency install, account, or API key.

Download the support-agent pack Read the pack instructions Inspect the synthetic incident bundle
After extracting the download

# Verify all five before / after scenarios

python run_demo.py verify --out results

# Reproduce the duplicate-refund failure

python run_demo.py run \
  --scenario timeout-after-commit \
  --agent unsafe --out broken
The unsafe run intentionally exits with code 1. On Windows, enter that command on one line. The verify command checks the expected failure and the passing fix.

03 / TEST THE CONSEQUENCE

A successful tool response isn't the whole story.

01

Separate response from effect

A timed-out refund can already be committed. The simulator records that effect even when the agent sees an error.

02

Check a business rule

At most one refund. No unapproved refund. No claim of success without a matching effect. These checks produce concrete pass/fail evidence.

03

Verify against the same failure

Reset the world, rerun the fixed policy, and compare the action traces. Keep the scenario to catch the regression next time.

PUT YOUR AGENT THROUGH IT

Which action can't you afford to get wrong?

Connect your own agent through the JSONL protocol. Download the CLI source and adapter example →

Prefer the installed CLI? Run python -m pip install --upgrade runledger, then runledger scenario verify --out results. Scenario commands require RunLedger 0.3.0 or later. The adapter example files are a separate download.

Discuss your scenario