Lab 01 — Replay-Safe Durable Workflow Engine
Phase 08 · Lab 01 · Phase README · Warmup
The problem
An agent that runs for minutes or hours will be interrupted — a deploy, a crash, a preemption. If your agent is "an LLM in a for-loop in a process," an interruption loses the plan, the partial results, and the money already spent on tool calls. Durable execution (the core of Temporal, a whole JD in this track) fixes this with one idea: event sourcing + deterministic replay. Every side effect (an activity) is recorded with its result; to recover, the engine re-runs the workflow from the top, and each recorded activity returns its saved result instead of executing again. The workflow fast-forwards to where it crashed and continues.
What you build
| Piece | What it does |
|---|---|
Context.call_activity | run an activity with retries, OR replay its recorded result (no re-execution) |
Context.now | a replay-safe clock — recorded on first read, replayed thereafter |
Context.wait_signal | return an external signal, or suspend the workflow (WorkflowBlocked) |
DurableEngine.run | execute-or-replay-then-continue; catch block/fail/non-determinism |
DurableEngine.signal | deliver an external event (human approval) and resume |
File map
| File | Role |
|---|---|
lab.py | your implementation (fill the # TODOs) |
solution.py | reference + a durable refund workflow (main()): block → approve → resume → replay |
test_lab.py | 14 tests incl. the headline "replay does not re-execute side effects" and non-determinism detection |
requirements.txt | pytest only |
Run it
pytest test_lab.py -v
LAB_MODULE=solution pytest test_lab.py -v
python solution.py
Success criteria
- You can explain why a replay must NOT re-execute recorded activities (no double charges).
-
You can explain why the workflow must be deterministic (no wall-clock/random/I/O) and
what
Context.now()does instead. - Your engine suspends on a missing signal and resumes when it's delivered.
- Retries succeed a flaky activity; exhaustion fails cleanly; the workflow can catch and compensate.
- The non-determinism guard fires when the workflow issues a different command on replay.
-
All 14 tests pass under both
labandsolution.
How this maps to the real stack
- This is a miniature of Temporal (and Azure Durable Functions, AWS Step Functions, Vercel Workflow / WDK, DBOS): a workflow is deterministic orchestration; activities are the side-effecting, retried steps; the history is the event-sourced log; signals resume paused workflows.
- The "no wall-clock/random/I/O in workflow code" rule is the durable-execution constraint — Temporal's determinism checker exists precisely to catch violations. Your non-determinism guard is the miniature of it.
wait_signalis how human-in-the-loop approval works durably (Phase 10): the workflow parks for minutes/days holding no resources, and a delivered signal wakes it.- Activity memoization is why a durable agent never double-charges a card or double-sends an email after a crash — the reliability lever from Phase 00 §5 made crash-safe.
Limits. Real engines add timers/heartbeats, activity task queues and workers, versioning of running workflows, child workflows, and a persistent store; the mechanism (history + replay + memoized activities + signals) is exactly this.
Extensions (your own machine)
- Add timers (
ctx.sleep(ticks)) recorded likenow(), so a workflow can wait durably. - Add a saga / compensation: on a late failure, run recorded activities' compensating actions in reverse.
- Port a workflow to a real Temporal Python worker and observe the same history/replay model in the Temporal Web UI.
Interview / resume signal
"Built a replay-safe durable execution engine — event-sourced history, deterministic replay that memoizes activities (no double side effects after a crash), retries with backoff, and human-approval signals — the Temporal model from first principles, including the determinism constraint and its enforcement."