« System-Design Cases · Interview Prep · Behavioral & Staff-Signal »
Coding & Take-Home Drills
Agentic-AI interviews increasingly include a hands-on coding round or a take-home: implement a small piece of agent infrastructure and defend it. The good news — you already built all of these in the labs, so this is recall, not discovery. Each drill below maps to a lab; do it on a blank file, timed, then compare to your lab solution. For every drill, "what good looks like" is the part interviewers actually grade: the edge cases, the invariants, and the one sentence of reasoning.
Universal rubric. Handle the empty/degenerate input; make failures structured (return an error, don't crash); guard any loop with a budget; keep it deterministic (inject the model/clock); and state the one invariant that must hold. Saying the invariant out loud ("the loop never exceeds
max_steps", "a replay produces byte-identical output") is a senior signal.
Drill 1 — The ReAct loop (Phase 01)
Prompt. Implement run_react(policy, tools, task, max_steps) where policy(scratchpad) -> str
is an injected model. Parse the model's step into a tool call or a final answer, dispatch the tool,
append the observation, repeat under a step budget.
What good looks like. A malformed step becomes a recoverable observation (no crash); a tool
error is returned, not raised; max_steps is a hard guard; llm_calls == steps (you can explain
the quadratic cost). Trap: forgetting the guard → an infinite loop.
Follow-up they'll ask. "Now make it ReWOO." → plan once, execute tools without the LLM via #E1
variable substitution, solve once → two LLM calls. Explain the adaptivity/cost tradeoff.
Drill 2 — A JSON-Schema validator + repair loop (Phase 02)
Prompt. Write validate(instance, schema) -> list[str] (errors) for a subset: type,
required, properties, enum, ranges, additionalProperties. Then a bounded run_with_repair
that feeds errors back to a fixer.
What good looks like. Collect all errors, not the first (the repair loop needs them); reject a
bool where an integer is required (isinstance(True, int) is True); handle a double-encoded
arguments string; bound the repairs. Say: "constrained decoding gives syntactic validity, not
semantic — hence the repair loop."
Drill 3 — An MCP tools/call handler (Phase 03)
Prompt. Given a JSON-RPC 2.0 request {"jsonrpc","id","method":"tools/call","params":{name, arguments}}, dispatch it: unknown method → -32601, unknown/denied tool → error, missing arg →
-32602, tool exception → error content (not a crash). Reject calls before initialize.
What good looks like. You know the standard error codes; a notification (no id) gets no
response; a tool exception is reported as error content, not a raised exception; permission gating
is enforced. Say: "the server is untrusted code; I permission-gate and audit it."
Drill 4 — Reciprocal Rank Fusion (Phase 05)
Prompt. Implement rrf(rankings, k=60) fusing several ranked ID lists by Σ 1/(k + rank).
What good looks like. Order-independent across the input lists; deterministic tie-breaking; you can explain why RRF is robust (it's score-scale-free — it uses ranks, not raw scores, so a lexical and a dense retriever fuse without normalization). Follow-up: "why hybrid?" → lexical catches rare tokens/IDs, dense catches paraphrase.
Drill 5 — BM25 from scratch (Phase 05)
Prompt. Implement BM25 scoring (k1=1.5, b=0.75): IDF, term-frequency saturation, length
normalization.
What good looks like. The formula is right, a term-matching doc ranks first, a common term is
down-weighted by IDF, an empty query is handled. Say the formula: score = Σ IDF(q) · tf·(k1+1) / (tf + k1·(1 - b + b·|d|/avgdl)).
Drill 6 — A RAPTOR node / collapsed-tree retrieval (Phase 06)
Prompt. Given chunks, cluster + summarize recursively into a tree; implement
collapsed_tree_search(query_vec, k) that searches nodes across all levels.
What good looks like. The tree's node count shrinks per level to a single root; a synthesis query surfaces a summary node above any single leaf (you can say why: no single chunk has the whole answer). Contrast: GraphRAG answers global questions via community summaries.
Drill 7 — A replay-safe activity memoizer (Phase 08)
Prompt. Implement call_activity(name, *args) on a durable context: if the i-th command is
already in the recorded history, return its result without running the activity; else run it and
record it.
What good looks like. Replay does not re-execute recorded activities (no double side
effects); the workflow is deterministic (time comes from ctx.now(), recorded once); a changed
replay is detected as non-determinism, not silently wrong. Say: "activities are memoized, so a
crash + replay is exactly-once."
Drill 8 — A capability check with path-traversal defense (Phase 09)
Prompt. Implement check_fs_read(path, allowed_prefixes) that denies paths outside the
allow-list, including .. traversal, and never performs the read on denial.
What good looks like. Canonicalize then check (normpath), reject .. escapes, boundary-match
so /work doesn't authorize /workshop, return a structured denial (no exception, no side effect).
Say: "allow-list beats deny-list; the check is deterministic — it doesn't depend on the model."
Drill 9 — An injection guardrail chain (Phase 10)
Prompt. Given context items labeled trusted/untrusted, implement extract_directives with a
trust boundary (instructions only from trusted items) + an input guard (quarantine untrusted items
that trip an injection detector), and an output_guardrail that catches a secret / exfil URL.
What good looks like. The trust boundary stops indirect injection; the layers are independent (each alone blocks the attack); the markdown-image beacon is caught. Say: "prompt injection is unsolved; I contain it architecturally, in layers."
Drill 10 — LLM-as-judge + Cohen's κ (Phase 11)
Prompt. Implement cohens_kappa(judge_labels, human_labels) and a judge_is_trustworthy(kappa, threshold=0.6) gate.
What good looks like. κ = 1 for perfect agreement, ~0 for chance; you compute observed vs expected agreement correctly; you gate on it before trusting a judge in CI. Say: "an LLM judge is a model that can be biased (position/verbosity/self-preference); I validate it against human labels before I trust it."
Drill 11 — A token-bucket quota (Phase 13/14)
Prompt. Implement a per-tenant RateLimiter(capacity, refill, now) with allow(cost) that
refills over time and never goes negative.
What good looks like. Refill is elapsed · rate capped at capacity; allow deducts and returns
True only if enough tokens; it never goes negative; time is an injected clock (deterministic).
Say: "per-tenant buckets are noisy-neighbor isolation."
Drill 12 — A semantic cache (Phase 14)
Prompt. Implement semantic_cache_lookup(query_vec, threshold) returning a cached answer when
cosine similarity to a stored query exceeds the threshold, with a TTL.
What good looks like. Cosine is right; a near-duplicate above threshold hits, below misses; a TTL expires stale entries (injected clock); you flag the false-hit risk ("cancel my order" vs "cancel my subscription") and say high-stakes queries should bypass it.
Drill 13 — apply-patch (Phase 15)
Prompt. Implement apply_patch(repo, {path, old, new}): find old exactly in the file and
replace it, failing (without corrupting) if old is missing or ambiguous (appears more than once).
What good looks like. Exact match; a not-found or ambiguous hunk returns an error and leaves the file unchanged; a snapshot/rollback is available. Say: "exact search/replace is reviewable and fails loudly — better than a fuzzy diff that can corrupt."
How to practice
- Pick a drill, set a 25-minute timer, blank file, no reference.
- Write it, including the edge cases from "what good looks like."
- Diff against your lab solution; note what you missed.
- Say the invariant and the one-sentence tradeoff out loud — that's the part that's graded.
- Rotate; hit every drill twice before an onsite. The recurring skills — inject the model, structured errors, bounded loops, deterministic time — transfer across all of them.