03 — Behavioral Questions

STAR stories tailored to GovTech / national-infrastructure / secure-environment contexts. Use the STAR format: Situation → Task → Action → Result. Keep each story to ~2 minutes. End with a measurable or specific outcome.

Core Stories to Prepare (Have These Four Ready)

Prepare these four stories, then flex them to any behavioural question:

Question Bank with Guidance

Technical Questions

Q: Walk me through how you would deploy an LLM in an environment with no internet access.

This is asked at the technical screen. Don't just recite a checklist — tell a story.

Preparation: Use Lab 1 as your foundation. Structure the answer as: the challenge (air-gap constraints), the decision (model selection / quantization trade-off), the execution (download, package, verify), and the result (the system passing offline tests).

Model answer (1 minute): "On this project I had to get an inference stack running on a facility server with no outbound internet. I started by selecting Qwen-30B at Q4_K_M — that's the 4.5-bit quantized format — because it fits in 18 GB, leaving headroom for the KV cache. Before the handover day I pulled the model weights via Ollama, downloaded all Python wheel files with pip download --dest ./wheels, and pre-cached the sentence-transformer embedding model. On arrival I verified the full stack offline: Ollama serving, FastAPI wrapper responding, ChromaDB returning results — all with the network cable unplugged. Then I wrote a health check endpoint that returns 200 only when the model is warm, which solved the cold-start problem by triggering a dummy request at boot."

Q: Describe a time you had to choose between RAG and fine-tuning for a real problem.

Guidance: The answer should show you understand both and can reason about the trade-off, not just recite definitions.

Story framework:

• Situation: You had a model producing technically correct but domain-incorrect responses (used general vocabulary, wrong output format).
• Task: Improve response quality without losing the grounding properties of RAG.
• Action: Analysed the failure modes: 60% of errors were vocabulary/format issues (fine-tuning would fix), 40% were factual gaps (RAG would fix). Concluded: QLoRA for style/format + RAG on top for factual grounding.
• Result: Fine-tuned model reduced format violations from 35% to 3%; RAG maintained faithfulness above 0.87. The combination was necessary; neither alone was sufficient.

Q: Tell me about a time you caught a hallucination before it reached a user.

Guidance: Show you have a systematic approach, not just luck.

Story framework:

• Situation: RAG pipeline returning answers that sounded plausible but had numbers not present in retrieved chunks.
• Task: Diagnose and prevent this class of error.
• Action: Implemented an LLM-as-judge faithfulness check on every response. The judge compared each factual claim in the answer against the retrieved context. Discovered that 11% of responses contained at least one unsupported numeric claim.
• Root cause: The embedding model was matching conceptually related chunks but not the specific document containing the actual value. Fixed by adding hybrid BM25+dense retrieval — BM25 catches exact-match numbers that semantic search misses.
• Result: Faithfulness rate improved from 0.87 to 0.94 after hybrid retrieval. Set up automated alert when rolling faithfulness drops below 0.90.

Stakeholder & Communication Questions

Q: Describe a time you had to explain AI limitations to a non-technical client or stakeholder.

Guidance: This is a core requirement of the Parsons JD ("explain AI concepts, limitations, and results to non-technical stakeholders"). Prepare a concrete story, not a generic answer.

Story framework:

• Situation: An infrastructure operations manager was using the AI assistant to check pump pressure thresholds before a maintenance decision. He trusted the output without checking the source citation.
• Task: Correct the behaviour without making the manager distrust the tool entirely.
• Action: Sat with the manager for 30 minutes. Showed him: (1) what a "hallucination" looks like (answer sounds right but source citation points to a different asset); (2) how to verify by clicking the citation; (3) that temperature setting 0.2 means the same question gives the same answer every time. Reframed the AI as "a very fast search that can explain itself" rather than "a system that knows things".
• Result: Manager started checking citations on safety-critical decisions. No incorrect operational decision was taken based on an AI answer. He became the most vocal champion of the tool in the programme.

Q: How do you handle it when a client asks the AI to do something it shouldn't?

Guidance: This probes your security and ethics mindset. Parsons JD mentions "guardrails" and "robust, secure" AI capabilities.

Answer framework: "I've encountered three categories of requests that AI assistants shouldn't fulfil:

1. Out-of-scope queries (asking the infrastructure assistant about HR policy): graceful refusal with a clear explanation — 'I'm configured to answer questions about the water network; for HR questions please contact [team].'

2. Security-risk queries (asking for raw database schemas, internal system architecture): the system prompt explicitly excludes these, and I add instruction-injection detection to catch attempts to override the prompt via user input.

3. Potentially unsafe operational queries (asking the AI to recommend a pump setpoint change): the assistant is read-only by design. I explain this as a feature, not a limitation — 'The AI can tell you what the current setpoint is and what the manual recommends, but changes must go through the HMI with supervisor approval. That's intentional.'

The key principle: the AI should never be the last gate before an irreversible operational decision."

Q: Tell me about a time you worked on a project where you had to rapidly learn a new domain.

Guidance: Infrastructure clients need to know you can absorb domain context quickly. Show a learning methodology, not just enthusiasm.

Story framework:

• Situation: Joined a Digital Twin programme with no prior SCADA or water infrastructure background.
• Task: Become productive enough in 2 weeks to design the AI integration.
• Action: Used a structured learning approach: (a) read the JD line by line and mapped each term to a Wikipedia/textbook reference; (b) asked the domain expert for the 5 most common questions engineers ask each day; (c) sat in on one SCADA training session to understand the operator's mental model; (d) built the mock database (Lab 4) using real asset types from the domain (pump stations, pressure sensors, flow meters) rather than generic "users and products".
• Result: Within 2 weeks could have an informed conversation about SCADA architecture, RTU/PLC distinction, and OPC-UA. Domain expert rated the AI assistant's tool definitions as "accurate enough to ship."

Collaboration and Team Questions

Q: Describe a challenge you faced working in a multinational team.

Guidance: The Parsons JD explicitly mentions "multinational, international experts across multiple time zones."

Story framework:

• Situation: Team split across the UK, UAE, and India. Decisions were getting blocked because async written communication created misunderstandings about technical trade-offs.
• Task: Unblock a key architecture decision (on-prem Ollama vs vLLM) that three stakeholders couldn't agree on.
• Action: Prepared a one-page written comparison (not a 20-slide deck): two columns, five rows (performance, memory, operational complexity, air-gap feasibility, cost). Shared 48 hours before the meeting so UAE and India teams could read it in their timezone. During the call, started with 5 minutes of questions before presenting any recommendation. Discovered the UAE team's concern was actually about GPU driver maintenance, not model quality — not captured in the original debate.
• Result: Decision made in one 45-minute call instead of 3 weeks of email. The revised decision (Ollama for dev, vLLM path documented for scale) addressed all concerns.

Q: Tell me about a time you pushed back on a technical decision.

Guidance: Probes whether you can advocate technically while respecting authority.

Story framework:

• Situation: Programme lead proposed using a cloud-based LLM API for the first proof-of-concept, reasoning that it would be "faster to demo."
• Task: Make the case that using a cloud API at demo stage would create a technical debt that was impossible to pay off in the air-gapped deployment.
• Action: Instead of objecting in the meeting, prepared a 15-minute working prototype (Lab 1 FastAPI server + Ollama) and demonstrated it the next day. Key message: "The cloud API costs 1 day to set up; the local stack costs 1 week. But the cloud stack requires complete rework before the actual deployment. The local stack is the production stack — we just run it on a laptop for the demo."
• Result: Programme lead agreed. The PoC used the local stack. When the facility deployment came 6 months later, it was the same codebase with changed configuration.

Ownership and Quality Questions

Q: Describe a mistake you made on an AI project and what you learned.

Guidance: This is mandatory preparation. Panels will probe for self-awareness. A good story shows a real mistake, a clear learning, and evidence the learning was applied.

Model story framework:

• Situation: Built and deployed a RAG pipeline that tested well on a 20-question eval set. First week of production, engineers complained that answers about alarm procedures were wrong.
• Mistake: The 20-question eval set was drawn from the same documents used for chunking. It tested retrieval, not answer quality on documents similar to but different from the training corpus (distribution shift).
• Action: Added 30 new questions from different operational documents (ones not in the original corpus) to the eval set. Discovered answer accuracy dropped from 84% to 61% on out-of-distribution questions. Root cause: chunks from procedural documents (step-by-step instructions) were getting retrieved for factual questions because the procedural language was superficially similar.
• Fix: Added metadata filtering (doc_type) so procedural documents are only retrieved for "how do I…" queries; factual queries use doc_type != "procedure".
• Result: Accuracy on the expanded eval set reached 81%. Now always include out-of-distribution documents in eval corpus from day one.
• Learning: "An eval set that only tests your training distribution is measuring overfitting, not quality."

Q: How do you decide when a model is "good enough" to ship?

Answer framework: "I apply three gates before shipping any model update:

Gate 1 — Regression: run the gold eval set (22+ questions). New model must match or improve baseline accuracy. Any > 5% regression on any question category (factual, procedural, data-query) is a blocker.

Gate 2 — Faithfulness SLA: automatic faithfulness judge on 50 test queries. Mean faithfulness must be ≥ 0.90. Below that I don't ship regardless of accuracy — wrong answers that sound right are worse than no answer.

Gate 3 — Sanity check on adversarial queries: manually test 5 prompt-injection attempts, 3 out-of-scope queries, 2 edge cases (empty corpus, SCADA offline). These must all be handled gracefully.

If all three pass, the model is ready for a 10% traffic shadow deployment. Full rollout after 48 hours with no regression in production faithfulness scores.

The definition of 'good enough' is never absolute — it's 'better than the current baseline and above the SLAs'. Premature perfection is the enemy of shipping."

Questions About the Programme Specifically

Q: Why do you want to work on a Digital Twin programme?

Don't give a generic answer. Show specific technical interest.

"The Digital Twin use case is technically interesting specifically because it combines three hard problems: air-gap deployment (which forces you to solve all the engineering problems that cloud abstracts away), multimodal retrieval (documents + real-time sensor data + geospatial data — each requiring different retrieval strategies), and Arabic language support in a domain where the vocabulary doesn't exist in generic multilingual training data. Most AI projects let you use OpenAI and call it done. This one requires real engineering."

Q: What does 'no internet connectivity' mean for your deployment architecture?

Probe: do they understand the implications or just the words?

"It means every dependency must be pre-staged: model weights (GGUF on disk, not downloaded at runtime), Python packages (wheel files, not pip install), embedding models (HuggingFace cache, not downloaded on first call), ChromaDB (local persistence, not cloud API). It also means no telemetry, no model update calls, no license checks that phone home. You validate offline behaviour explicitly — not just 'assume it works' — by actually disconnecting the network and running the test suite. The deployment artefact is a tarball: model weights + docker images + wheel files + ChromaDB index. If everything in that tarball is present, the system boots and runs. If anything is missing, it fails fast with a clear error. That's the contract."

Q: How do you approach knowledge transfer and documentation for a client team?

The JD explicitly requires "support demonstrations and user-training sessions" and "documentation."

"I document at three levels: (1) a one-page 'what can the AI do and not do' for end users — no technical terms; (2) a 5-page operations guide for the team managing the system — how to ingest new documents, update the model, check logs; (3) technical architecture docs for the future maintainers who will extend the system after handover. For training sessions I use demo-then-try: show one use case live, then let the engineer ask their own question on their screen while I watch. The questions they ask (and the ones they don't) tell me what's missing from the docs. I add those to an FAQ before the next session."