Roles and Career Paths

Phase 0 · Document 01 · Orientation Prev: 00 — What Is an LLM Engineer? · Next: 02 — How to Study Fast-Moving AI

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

00 named the roles; this document is the detailed field guide to each one — responsibilities, required skills, tools, systems owned, production metrics, example tasks, interview bar, portfolio project, how to train, and startup relevance. This is the document you return to when deciding what to learn next, preparing for an interview, or scoping a hire. Without this depth, "I want to be an LLM engineer" stays a vibe; with it, you have a concrete skills checklist and a measurable target.

2. Core Concept

Plain-English primer (the tools & techniques named in these role cards)

The cards list tools as shorthand. Here's what each is, grouped by what it's for, so the "required skills/tools" lines are readable with no prior background. (Each is covered fully in a later phase; this is orientation.)

• Serving / inference engines — software that runs models fast for many users: vLLM, SGLang, TGI (open-model GPU servers), TensorRT-LLM (NVIDIA-optimized), llama.cpp / Ollama (run models locally). Purpose: turn a model file into a fast API. → Phase 6/7
• Training / fine-tuning — PyTorch (the core ML framework), LoRA / QLoRA (cheap ways to fine-tune by training a small add-on instead of the whole model), DPO / SFT / RLHF (methods to teach a model to follow instructions and preferences), DeepSpeed / FSDP (tools to train across many GPUs). Purpose: adapt or create models. → Phase 13
• RAG / retrieval — vector databases (Chroma, Qdrant, Weaviate, pgvector) store embeddings (text-as-numbers) for similarity search; rerankers re-order search hits by relevance. Purpose: let a model answer from your documents. → Phase 9
• Gateways / platform — LiteLLM (an open proxy in front of many providers), Postgres/Redis (database/cache), OpenTelemetry/Prometheus/Grafana (logging, metrics, dashboards). Purpose: run LLM infrastructure for a whole team. → Phase 8
• Code tooling (for AI coding tools) — AST (a program's grammar tree), Tree-sitter (a fast parser that builds ASTs), LSP (the protocol IDEs use to understand code), diff/patch (applying edits). Purpose: feed a codebase to a model and apply its edits safely. → Phase 11
• Infra — Kubernetes (run containers at scale), CUDA (NVIDIA GPU programming), Docker (package software), Terraform (declare cloud infrastructure), Ray/Slurm (schedule GPU jobs). Purpose: keep GPUs and services running reliably/cheaply. → Phase 7
• Evaluation — OpenAI Evals / Inspect AI / Ragas (frameworks to score model outputs), golden datasets (known-good test cases), LLM-as-judge (using a model to grade outputs). Purpose: prove and protect quality. → Phase 12

You won't know all of these yet — that's expected. Use this as a legend while reading the role cards; each phase teaches the relevant tools hands-on.

Each role below uses the same card so you can compare them directly.

AI Application Engineer

• Responsibilities: build product features on LLM APIs — chat, RAG, tool calling, structured output, streaming UIs.
• Required skills: strong general software engineering; API integration; prompt design; RAG basics; evals; cost/latency awareness.
• Tools: OpenAI/Anthropic SDKs, Vercel AI SDK, LangChain/LlamaIndex, a vector DB, a web framework.
• Systems owned: application code, prompt logic, retrieval pipeline, integration with the gateway.
• Production metrics: response quality, latency (TTFT), cost per request, error/refusal rate.
• Example tasks: add citations to a RAG answer; make a flaky tool call reliable; cut cost per request 30%.
• Interview bar: RAG architecture, tool calling, structured output, streaming, basic evals.
• Portfolio: a working RAG system with citations + an eval harness.
• How to train: Phases 1, 3, 4, 5, 9, 10, 12, 13.
• Startup relevance: highest — this is most of an early AI product.

LLM Platform Engineer

• Responsibilities: build the internal platform other engineers use — gateway, model registry, routing, metering, shared evals, cost visibility.
• Required skills: backend services, databases, queues, observability, multi-tenancy, API design.
• Tools: LiteLLM, FastAPI/Fastify, Postgres, Redis, OpenTelemetry, Prometheus, Grafana.
• Systems owned: LLM gateway, model registry, usage metering, eval infrastructure.
• Production metrics: gateway availability, routing correctness, cost-accounting accuracy, per-team budget adherence.
• Example tasks: add a fallback provider; build per-key budgets; expose a usage dashboard.
• Interview bar: gateway design, routing, metering, observability, multi-tenancy.
• Portfolio: an LLM gateway with routing, cost tracking, and a dashboard.
• How to train: Phases 1, 5, 7, 8, 12, 15.
• Startup relevance: high — becomes your cost-control and reliability backbone.

ML Engineer

• Responsibilities: train, fine-tune, and adapt models; own datasets and eval-before/after.
• Required skills: PyTorch, distributed training, data engineering, evaluation, some CUDA.
• Tools: PyTorch, HF Transformers/TRL, Axolotl, Unsloth, DeepSpeed/FSDP.
• Systems owned: training pipelines, datasets, fine-tunes, model eval.
• Production metrics: eval score before/after, training cost/time, drift.
• Example tasks: LoRA fine-tune a small model; build an instruction dataset; measure regression.
• Interview bar: SFT/LoRA/DPO, dataset quality, overfitting/forgetting, eval.
• Portfolio: a fine-tuned model with before/after eval vs prompting/RAG.
• How to train: Phases 1, 2, 13(FT), 12.
• Startup relevance: medium — usually after prompting+RAG are exhausted.

AI Infrastructure Engineer

• Responsibilities: own compute/orchestration for training and serving.
• Required skills: Kubernetes, cloud, networking, storage, GPU operations, IaC.
• Tools: Kubernetes, CUDA, Docker, Terraform, Ray, Slurm.
• Systems owned: GPU clusters, scheduling, storage, networking.
• Production metrics: GPU utilization, cluster availability, job completion, cost per GPU-hour.
• Interview bar: cluster design, scheduling, GPU memory, cost efficiency, reliability.
• Portfolio: a reproducible GPU serving cluster with autoscaling + cost dashboard.
• How to train: Phases 6, 7, 15.
• Startup relevance: low early (use managed), rising with self-hosting scale.

LLM Inference Engineer

• Responsibilities: make serving fast, cheap, scalable.
• Required skills: serving frameworks, KV cache/PagedAttention, batching, quantization, profiling, CUDA.
• Tools: vLLM, SGLang, TensorRT-LLM, llama.cpp, Triton, CUDA profilers.
• Systems owned: inference servers, batching/scheduling, quantization pipeline.
• Production metrics: TTFT, TPOT, tokens/sec, cost per 1M tokens, GPU memory efficiency.
• Interview bar: KV cache design, PagedAttention, continuous batching, quantization, speculative decoding.
• Portfolio: a vLLM deployment with a rigorous benchmark report.
• How to train: Phases 2, 5, 6, 7.
• Startup relevance: medium-high once self-hosting for unit economics.

RAG Engineer

• Responsibilities: own retrieval-augmented systems end-to-end.
• Required skills: embeddings, chunking, vector/hybrid search, reranking, grounding, RAG evals.
• Tools: Chroma/Qdrant/Weaviate/pgvector, Cohere/BGE rerankers, Ragas.
• Systems owned: ingestion, index, retrieval, reranking, citation, RAG eval.
• Production metrics: retrieval precision/recall, faithfulness, citation accuracy, latency.
• Interview bar: chunking, embedding choice, hybrid search, reranking, faithfulness eval.
• Portfolio: a doc assistant with hybrid search, reranking, citations, and eval.
• How to train: Phases 1, 9, 12, 13.
• Startup relevance: high for any knowledge/search product.

Agent Engineer

• Responsibilities: build reliable, safe tool-using agents.
• Required skills: tool-call design, JSON schema, loop control, approval gates, memory, tracing.
• Tools: OpenAI/Anthropic tool APIs, agent frameworks, tracing/observability.
• Systems owned: agent loop, tool schemas, validation, approval, audit.
• Production metrics: task success rate, valid-tool-call rate, steps, cost, rollback rate.
• Interview bar: tool schema design, loop safety, approval gates, memory, observability.
• Portfolio: a repo-editing agent with tools, traces, and approval gates.
• How to train: Phases 1, 10, 12, 14.
• Startup relevance: high and rising for automation products.

AI Coding Tools Engineer

• Responsibilities: build AI dev tools (Cursor/Copilot-style).
• Required skills: codebase indexing, AST/Tree-sitter, context collection, diff/patch, editor extension APIs, model routing.
• Tools: LSP, Tree-sitter, embedding indexes, diff libraries, VS Code extension API.
• Systems owned: indexer, context engine, apply-patch, autocomplete/agent loop, model router.
• Production metrics: suggestion accept rate, patch-apply success, tests-pass rate, latency.
• Interview bar: indexing, AST, context windows, patch apply, multi-model routing.
• Portfolio: a VS Code extension that collects context and applies model edits.
• How to train: Phases 1, 9, 10, 11.
• Startup relevance: high for developer-tool startups.

Model Evaluation Engineer

• Responsibilities: build/run evals that measure quality, reliability, safety.
• Required skills: golden datasets, LLM-as-judge calibration, regression harnesses, statistics.
• Tools: OpenAI Evals, Inspect AI, Ragas, custom harnesses.
• Systems owned: eval datasets, judges, CI eval gates, dashboards.
• Production metrics: eval coverage, regression-detection rate, judge-human agreement.
• Interview bar: benchmark vs eval, golden sets, judge bias, regression gating.
• Portfolio: a CI eval harness that blocks regressions on a real task.
• How to train: Phases 12, 13, 14.
• Startup relevance: high — evals are a moat and a trust signal.

AI Safety / Governance Engineer

• Responsibilities: keep systems safe and compliant.
• Required skills: prompt-injection defense, output classification, PII handling, audit logging, policy.
• Tools: guardrail libraries, classifiers, PII detectors, audit infra.
• Systems owned: input/output filters, audit logs, tenant isolation, retention controls.
• Production metrics: violation rate, false-refusal rate, audit completeness, incident MTTR.
• Interview bar: injection defense, data retention, isolation, compliance readiness.
• Portfolio: a guardrail + audit layer with an injection test suite.
• How to train: Phases 12, 14.
• Startup relevance: rising fast for enterprise sales.

Applied AI Architect → Startup CTO

• Responsibilities: design cross-team architecture; (CTO) make every technical decision under constraints.
• Required skills: breadth across all roles + product judgment + cost modeling + team-building.
• Systems owned: model/serving/gateway/eval strategy; (CTO) the whole stack.
• Production metrics: quality/cost/latency at the product level; gross margin; reliability.
• Portfolio: an end-to-end platform (the capstone) doubling as an investor demo.
• How to train: all phases; prioritize 5, 8, 9, 12, 15, 16.
• Startup relevance: is the startup.

3. Mental Model

CAREER LADDER (typical):
  Software Engineer
        │
        ▼
  AI Application Engineer ──► RAG Engineer / Agent Engineer / AI Coding Tools Engineer
        │                                   │
        ▼                                   ▼
  LLM Platform Engineer ──► LLM Inference / AI Infra Engineer
        │
        ▼
  Applied AI Architect ──► Startup CTO

  Cross-cutting at every level: Model Evaluation · AI Safety/Governance
  Each step = a shipped portfolio artifact, not just study.

4. Hitchhiker's Guide

What to do first: pick the role card that best matches your current strengths + interest, and treat its "Portfolio" line as your North Star.

What to ignore at first: roles two steps away on the ladder. You don't need inference depth to land an application role.

What misleads beginners: collecting tools (LangChain, vector DBs, vLLM) without building anything end-to-end; optimizing for the most "advanced" role instead of the most hireable next one.

How experts reason: they match the role's production metrics to skills they can demonstrate, then build the one portfolio artifact that proves those metrics — and they can speak to cost/latency for it.

What matters for the career: depth in one role + literacy across adjacent ones. The architect/CTO track is breadth on top of at least one deep role.

How to verify readiness: can you, for your target role, name the systems you'd own and show an artifact where you moved its key metric? If not, that's your next project.

Questions to ask in interviews: What metric will I own? Who owns cost? Is there an eval culture? Do I touch infra/GPUs? What does the on-call look like?

What silently stalls careers: never shipping; ignoring evals; treating cost/latency as someone else's job; staying a generalist who's read everything and built nothing.

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• Each role is defined by its systems owned and metrics moved — not its title.
• A portfolio artifact that moves a metric is the strongest hiring signal in this field.
• Application and Platform roles are the most common entry points for software engineers.
• Inference/Infra roles require GPU + systems depth and usually come later.
• Evaluation and Safety are cross-cutting and increasingly their own roles, especially for enterprise.
• Senior/architect roles are breadth on top of at least one deep role.
• Startup CTO = breadth across all roles plus product and cost judgment.
• You can change specialization — the application role is a hub that branches to RAG, agents, tools, or platform.

9. Observations from Real Systems

• OpenRouter teams exemplify Platform + Inference: provider routing, metering, multi-provider reliability are their daily metrics.
• Cursor exemplifies AI Coding Tools + Application + RAG: their hard problems are context assembly, patch apply, and model routing.
• Enterprise gateways show Platform + Safety/Governance: budgets, audit, isolation, retention dominate.
• RAG SaaS companies hire RAG + Eval engineers because faithfulness and regression-prevention are the product.
• Agent startups weight Agent + Inference: loop reliability and latency define UX.
• Seed-stage AI startups post for "AI Engineer (generalist)" — really the Applied AI Architect doing everything, the exact reason this curriculum spans all phases.

10. Common Misconceptions

11. Engineering Decision Framework

Plan your next 90 days:
  1. Pick TARGET role card (Section 2).
  2. List its production metrics; circle which you can already demonstrate.
  3. Choose its Portfolio project.
  4. Map "How to train" phases → schedule them.
  5. Build the artifact; write it up with the metric you moved.
  6. Interview-prep against its "Interview bar" (Phase: interview-prep).

Choosing the NEXT role after your first:
  Enjoyed retrieval/grounding?  → RAG Engineer.
  Enjoyed tool loops/automation? → Agent Engineer.
  Enjoyed infra/abstractions?    → Platform Engineer.
  Enjoyed perf/GPUs?             → Inference Engineer.
  Enjoyed correctness/trust?     → Evaluation / Safety Engineer.
  Want to lead/build a company?  → Architect → CTO.

12. Hands-On Lab

Goal

Turn your chosen role into a measurable 90-day plan with a portfolio artifact and an interview-readiness checklist.

Prerequisites

• The role-plan.md from 00's lab.

Steps

1. Copy your target role's card into role-plan.md.
2. For each production metric, write one sentence on how your portfolio project will demonstrate moving it.
3. Build a phase schedule from "How to train" (dates).
4. Turn the "Interview bar" into a checklist of topics to drill.
5. Define "done": the artifact exists, is deployed/demoable, and you can speak to its quality/cost/latency.

Expected output

A 90-day plan: artifact spec, metric-demonstration notes, phase schedule, interview checklist.

Debugging tips

• Can't connect the project to a metric? The project is too vague — narrow it.
• Plan too big? Cut to one artifact that proves 2–3 metrics well.

Extension task

Draft the project's README now (problem, architecture, metrics) as a contract you build toward.

Production extension

Add a "metrics achieved" section you fill in as you build, so the writeup is interview-ready on completion.

What to measure

Metrics you can demonstrate now vs at plan completion; interview-checklist coverage.

Deliverables

• A 90-day role plan with artifact spec, schedule, and interview checklist.

13. Verification Questions

Basic

1. For your target role, what are the systems owned and the top production metric?
2. Which roles are typical first targets for a backend engineer?
3. What single signal most strongly predicts getting hired here?

Applied 4. You want to move from Application to RAG Engineer. What artifact and phases do you need? 5. Translate the LLM Platform Engineer "interview bar" into a study checklist.

Debugging 6. Strong résumé, no offers. What's the likely gap and the fix? 7. You're labeled "too junior" despite years of SWE. What deliverable signals seniority here?

System design 8. Staffing a 5-person enterprise-gateway startup: which roles, in what order, and why?

Startup / product 9. As a solo founder, which single portfolio artifact best doubles as your product demo and proves the most metrics?

14. Takeaways

1. Every role = systems owned + metrics moved; read those, not titles.
2. Build the one portfolio artifact your target role demands — it's the top hiring signal.
3. Application/Platform are the common entry points; specialize into RAG/agents/inference next.
4. Evaluation and Safety are real, growing roles, especially for enterprise.
5. Architect/CTO = breadth on top of one deep role + product/cost judgment.
6. Turn the role card into a dated 90-day plan with a clear "done."

15. Artifact Checklist

• 90-day role plan (artifact spec + schedule + interview checklist).
• Target role card copied with metric-demonstration notes.
• Portfolio project README drafted as a build contract.
• Phase schedule with dates.
• Interview-bar checklist for your role.
• Notes: the career ladder and your next two rungs.

Next: 02 — How to Study Fast-Moving AI