Replay A Counterfactual

Create a branch from a checkpoint, apply one intervention, run replay, and inspect comparison output.

This tutorial assumes you already have a captured run with at least one checkpoint. Use First captured run if you need one.

1. Discover The Trace

npx isplay discover run <runId>
npx isplay runs checkpoints <runId>
npx isplay runs context <runId>

Choose the earliest checkpoint that still contains the state needed to replay the suspected decision.

2. Create A Branch

npx isplay branch create \
  --from <runId> \
  --checkpoint <checkpointId> \
  --project "$ISPLAY_PROJECT_ID" \
  --name "change claim status"

Save the returned branchId.

3. Add One Intervention

Create a patch file:

{
  "status": "approved"
}

Attach it as a state patch:

npx isplay branch intervene <branchId> \
  --project "$ISPLAY_PROJECT_ID" \
  --kind state_patch \
  --target-context-path state.claim.status \
  --patch patch.json \
  --description "Change claim status before decision"

Use one branch for one hypothesis. If you also want to remove a prompt clause, make a second branch.

4. Run Replay

npx isplay replay <runId> \
  --project "$ISPLAY_PROJECT_ID" \
  --branch <branchId> \
  --model-policy recorded-only \
  --tool-policy pause-for-fixture

The replay may finish or pause. A pause is not a failure; it means the branch reached a divergent tool output and needs an explicit fixture.

5. Inspect Output

npx isplay replay get <replayId>
npx isplay replay events <replayId>
npx isplay replay diff <replayId>
npx isplay replay metrics <replayId>
npx isplay effects replay <replayId>

Read the outputs in this order:

Replay status: ok, paused, or error.

First divergence: which event changed first.

Trace diff: whether event order, types, or summaries changed.

Tool diff: whether tool args, names, or outputs changed.

Metrics: divergence step, tool sequence distance, fixture dependency count, and tool argument change marker.

Effects: ranked explanation leads and validity labels.

6. If Replay Paused

npx isplay replay requirements <replayId>

Then follow Resolve fixtures. Do not add a fixture without recording why it is valid.