02 — System Design Walkthroughs

Practice prompts tailored to the AI Specialist / Digital Twin role. Each maps to a detailed solution in ../system-design/.

How to Use These Prompts

  1. Read the prompt only — do not read the solution first.
  2. Set a 45-minute timer.
  3. Structure your answer: Clarifying Questions → Capacity Estimation → Architecture → 3 Deep Dives → Trade-offs.
  4. Compare to the solution document.
  5. Note 3 things you missed in a personal gaps.md for spaced repetition.

For phone/video screens: practice the 10-minute version — just clarifying questions + architecture + one deep dive. Interviewers at this level often only have time for one thread.

Prompt 1 — "Design the AI Assistant for an Air-Gapped Digital Twin"

You are joining a programme to deliver an AI-powered assistant for a national water infrastructure operator in the Middle East. The deployment environment has no internet access. The assistant must answer natural language questions about the network (10,000 km of pipelines, 200 pump stations) and query live SCADA data. Design the system.

Clarifying questions to ask:

  • Expected concurrent users? (Assume 50 engineers across 3 sites)
  • Languages? (English + Arabic)
  • Types of queries? (Document Q&A, real-time data, workflow guidance, out-of-scope refusal)
  • Existing infrastructure? (Windows Server 2019, NVIDIA A40 GPU available, internal Intranet)
  • Acceptable response latency? (< 30s end-to-end)
  • Any existing data format? (SCADA historian exports as CSV, asset DB is Oracle)

Estimation you should do:

  • Model size at Q4_K_M: 30B = ~17 GB (fits A40 with 48 GB VRAM)
  • Corpus size: 10,000 docs × 1,000 tokens avg = 10M tokens; chunked at 1000 chars → ~100K chunks; ChromaDB at 384 dims × 4 bytes × 100K = ~150 MB index
  • QPS: 50 engineers, 1 query per 2 min avg = 0.42 QPS — a single GPU handles this trivially

Key architecture decisions:

  • Query router (LLM-based classifier or rule-based heuristic)
  • RAG pipeline for document questions
  • Tool-calling agent for SCADA data queries
  • Multilingual embedding model (intfloat/multilingual-e5-large or equivalent)
  • Guardrail layer (faithfulness check, PII filter, format validation)

➜ See ../system-design/01-air-gapped-digital-twin-assistant.md

Prompt 2 — "Design a RAG Pipeline for 100 GB of Infrastructure Documents"

The programme has accumulated 100 GB of infrastructure documents: engineering drawings (PDF), maintenance manuals (PDF), GIS layer metadata (JSON/XML), SCADA alarm history reports (Excel), and regulatory standards (Word/PDF). Design a RAG pipeline that keeps this corpus fresh, supports both English and Arabic queries, and achieves > 90% retrieval precision at p99 < 2s.

Clarifying questions to ask:

  • How often does the corpus update? (New documents weekly; manuals revised quarterly)
  • Who curates the corpus? (Document management team uploads via SharePoint)
  • Maximum acceptable staleness? (48 hours from document upload to searchable)
  • Are all documents full-text-searchable? (Some drawings are scanned PDFs → need OCR)
  • Metadata available? (document type, date, classification level, asset tag)

Estimation:

  • 100 GB PDFs × avg 10K chars/page × 0.5 pages/doc (rough) → ~50M chars text after extraction
  • Chunk at 1000 chars, 150 overlap → ~60M chars / 850 stride ≈ 70K chunks
  • Embedding time: all-MiniLM-L6-v2 at ~10ms/chunk CPU = 700 seconds; parallelise to 4 workers = 175s = 3 min (batch ingest, not blocking)
  • Index size: 70K chunks × 384 dims × 4 bytes = ~107 MB (fits in RAM)

Key architecture decisions:

  • OCR stage for scanned PDFs (Tesseract or PaddleOCR, offline)
  • Metadata extraction for filtering (where={"doc_type": "maintenance_manual"})
  • Incremental ingest: hash each doc; skip if hash unchanged; delete+re-embed if changed
  • Hybrid search: dense (sentence-transformers) + sparse (BM25 via rank_bm25) with RRF fusion
  • Reranker: cross-encoder (cross-encoder/ms-marco-MiniLM-L-6-v2) on top-20 → top-4

➜ See ../system-design/02-rag-infrastructure-pipeline.md

Prompt 3 — "Design a QLoRA Fine-Tuning Pipeline for Domain Adaptation"

Your base model (Qwen2.5-7B-Instruct) produces syntactically correct responses but uses general vocabulary rather than the infrastructure domain's terminology. It also inconsistently follows the required output format (JSON with answer, confidence, sources fields). Design a fine-tuning pipeline using QLoRA that runs within the air-gapped environment and produces a model that can be deployed via Ollama.

Clarifying questions to ask:

  • What GPU is available? (NVIDIA A40, 48 GB VRAM → 7B QLoRA comfortable)
  • How many training examples can be produced? (100–500 initially, scale to 2K)
  • How should data be sourced? (Export Q&A from existing SharePoint wikis; synthetic augmentation from base model)
  • How do we validate the fine-tuned model? (Gold eval set of 50 questions with expert-verified answers)
  • What's the deployment target? (Ollama via GGUF)

Estimation:

  • 7B QLoRA (NF4 base + FP16 adapters): ~8 GB VRAM → leaves 40 GB for activations/batch
  • Training time: 500 examples × 3 epochs at batch size 8 on A40 ≈ 12–18 minutes
  • Adapter size: ~8–16 MB (for r=16 on 4 projections × 32 layers)
  • GGUF conversion: llama.cpp convert-hf-to-gguf.py + quantize to Q4_K_M

Key architecture decisions:

  • Data curation pipeline (SharePoint export → format normalisation → dedup → instruction formatting)
  • Validation gate before deployment (perplexity + task accuracy + format compliance)
  • Adapter management: version adapters separately from base model; swap at runtime with Ollama Modelfile
  • Rollback strategy: keep previous adapter version; A/B test on 10% of traffic

➜ See ../system-design/03-qlora-domain-adaptation-pipeline.md

Prompt 4 — "How would you evaluate and monitor the Digital Twin AI Assistant post-deployment?"

The assistant has been running in production for 3 months with 50 users and 200 queries/day. The programme manager reports that some engineers have stopped using it after receiving two wrong answers. How do you set up a monitoring, evaluation, and continuous improvement loop?

Clarifying questions to ask:

  • Is there a feedback mechanism? (Currently just a thumbs-up/down button; engineers don't always click it)
  • Are queries and responses being logged? (Yes, but only to a flat log file, no structured analysis)
  • What changed between deployment and the wrong answers? (New manuals were added but the corpus wasn't re-ingested)
  • Are there domain experts available to label data? (One infrastructure engineer can review 20–30 responses/week)

The monitoring stack you should design:

  1. Structured logging: every request → {timestamp, user_id, query_hash, retrieved_chunk_ids, response_hash, tool_calls, latency_ms, faithfulness_score, user_feedback}
  2. Online evaluation: async faithfulness judge on every response; alert when rolling 1-hour faithfulness < 0.85
  3. Corpus freshness check: daily cron that detects new/modified documents in SharePoint → trigger incremental ingest
  4. User feedback loop: thumbs-down triggers a review queue; reviewed examples added to eval set and optionally to fine-tuning dataset
  5. Periodic eval runs: weekly automated run of the 50-question gold set; alert on > 5% regression from baseline
  6. Drift detection: monitor embedding distribution of queries (using a reference set from week 1); alert when new queries drift far from training distribution (engineers are asking questions the system wasn't designed for)

Key talking point: "The two wrong answers were caused by stale corpus, not model failure. The fix is process, not retraining: automatic corpus refresh + staleness alerting."

➜ See ../system-design/01-air-gapped-digital-twin-assistant.md (section 6: Monitoring and Evaluation)

Bonus Variants (For Deeper Prep)

Security-focused variant:

"The digital twin infrastructure is classified as critical national infrastructure. How do you ensure the AI assistant meets data security requirements?"

Key points: air-gap (no exfiltration path), audit logging, PII redaction, no model weights leaving the facility, prompt injection prevention, classification-level propagation from source docs to responses, separate deployment environments for different classification levels.

Scale variant:

"The programme expands to 500 concurrent users across 10 sites. What breaks in your single-server design?"

Key points: single Ollama instance becomes a bottleneck → load balancer + multiple inference replicas, session affinity for conversation history, distributed ChromaDB (ChromaDB Cloud or Weaviate on-prem), shared corpus but per-site query logs.

Multilingual variant:

"The client insists that engineers can query in Arabic and receive answers in Arabic, referencing English-language technical manuals."

Key points: multilingual embedding model for cross-lingual retrieval, query language detection (langdetect), response language instruction in system prompt ("Answer in the same language as the question"), Arabic tokenisation differences (right-to-left, morphologically rich → chunk on sentence boundaries, not fixed characters).