Interview Prep 06 — Agent Engineer

Specialist in tool-using agents: the loop, reliability, structured output, safety. Phases: 10, 14.01. Cheatsheet: 15.

1. Core concepts

  • The agent loop (reason → act → observe); ReAct / plan-execute / reflection / orchestrator-worker.
  • Tool/function calling protocol; trust boundary (model proposes, app executes).
  • Structured output: constrained ≠ correct; JSON Schema + validation/repair.
  • Per-step reliability compounds (0.95^10 ≈ 0.60); least-agentic that works.
  • Agent memory (working/session/long-term) (Phase 10.04).
  • Safety: sandbox, approval gates, indirect prompt injection (Phase 14.01).
  • Agent eval: outcome + trajectory + safety; observability/traces.

2. System design questions

  1. Design an agent that takes real actions (email/calendar/DB) safely. (Trust boundary, whitelist, approval gates, audit.)
  2. Design a research agent that browses + summarizes. (Indirect injection containment, tool results = untrusted, output exfil scan.)
  3. Make a flaky 12-step agent reliable. (Reduce steps, per-step validation, checkpoints, eval.)

agent loop diagram.

3. Debugging questions

  • Agent emails an attacker after browsing a page → indirect injection; trust boundary + least privilege (exercise RA7).
  • Agent loops forever / burns cost → step cap, loop detection, budget.
  • JSON valid but wrong → semantic validation + repair.
  • Works in demo, fails in prod → compounding reliability; fewer steps + checks.

4. Coding exercises

  • Build a tool-calling agent with the trust boundary + approval gate (labs/lab-04).
  • Implement structured output validation + repair loop (labs/lab-10).
  • Instrument per-step reliability + correlation-ID tracing.

5. Architecture exercises

  • Draw the agent tool-calling loop with the trust boundary + approval gate marked.
  • Design planner-executor vs single-loop for a multi-step task.

6. Model selection questions

  • Reasoning vs normal model for the planner? (hard planning → reasoning; executors → cheap).
  • Route per step by difficulty to control cost.

7. Production incident questions

  • Prompt-injection exfil via tool result → output scan + isolation; red-team (template 06).
  • Destructive action taken without approval → approval gate gap; add HITL.
  • Cost spike from agent loops → step/budget caps; per-resolved-task pricing.

8. Portfolio project

A safe tool-using agent for a real task: trust boundary (model proposes/app executes), whitelisted tools + approval gate for high-impact, structured output validation, per-step reliability eval, correlation-ID tracing, and a prompt-injection red-team report. README with the reliability + safety numbers.

9. 30-minute drill

  • 5 min: the agent loop + trust boundary + compounding reliability.
  • 10 min: design a safe action-taking agent (whiteboard).
  • 10 min: debug indirect-injection exfil + a runaway loop.
  • 5 min: agent eval (outcome/trajectory/safety).

10. 2-hour take-home

Build an agent with 2–3 tools that completes a multi-step task: enforce the trust boundary + an approval gate for the risky tool; validate structured outputs; add a step cap + tracing; eval task success + per-step reliability; include a short injection red-team.

11. Senior-level expectations

  • Junior: single tool call works.
  • Mid: multi-step agent with tools + basic validation.
  • Senior: designs the trust boundary, makes agents reliable (fewer steps + per-step checks), contains injection architecturally, and evals outcome+trajectory+safety.
  • Principal: sets agent-safety + reliability standards; decides where agency is worth it ("least-agentic"); designs the sandbox/approval/audit platform.

Next: 07 — AI Coding Tools Engineer · Index: interview-prep/