01 — Air-Gapped Digital Twin AI Assistant

Role: AI Specialist / Senior AI Engineer
Asked at: Parsons, AECOM, Jacobs, Leidos, Mott MacDonald, GovTech / national infrastructure programmes
Constraint: Fully air-gapped deployment; English + Arabic; 50 concurrent users; < 30s end-to-end latency

1. Clarifying Questions

Before drawing anything, ask these:

Functional

What query types must be supported? (Document Q&A / live SCADA data / workflow guidance / out-of-scope refusal)
Languages? (English + Arabic confirmed)
Conversation memory required? (Multi-turn or single-turn?) — assume multi-turn for operations context
Does the AI ever write to SCADA or asset systems? (No — read-only. State this as a design invariant.)

Non-functional

Concurrent users? (50 engineers, 3 sites → ~0.5 QPS peak)
Latency SLO? (< 30s end-to-end for document Q, < 30s for 2-hop tool call)
Availability? (Business hours only; 99% is fine → no HA multi-GPU required)
Data classification? (Infrastructure data — log everything; no response caching to disk if classified)

Infrastructure available at deployment site

1× NVIDIA A40 (48 GB VRAM), Windows Server 2019
Internal Intranet (HTTPS within facility, no external routing)
Document management: SharePoint on-prem
Asset DB: Oracle 19c (read-only service account available)
SCADA historian: OSIsoft PI (REST API available on intranet)

2. Capacity Estimation

Model selection

30B Q4_K_M: 16.8 GB — fits in A40 with 31 GB headroom for KV-cache
At 0.5 QPS, 1 request active at a time → single-user throughput is sufficient

Document corpus

Estimate: 1,000 infrastructure documents × avg 8,000 chars = 8M chars
Chunked at 1,000 chars, 150 overlap → stride = 850 chars → ~9,400 chunks
Embedding: 384 dims × 4 bytes × 9,400 = ~14 MB index (trivial in RAM)

KV-cache budget (30B Q4_K_M on A40)

Per-token KV: 2 (K+V) × 64 layers × 40 heads × 128 head_dim × 2 bytes = 1.3 MB/token
At 8,192-token context: 1.3 MB × 8,192 ≈ 10 GB → leaves 21 GB headroom after model weights
Safe for single concurrent request with 8K context window

Storage

Model weights: 16.8 GB
ChromaDB index: ~200 MB (10K chunks with metadata)
Document store: 1 GB (original PDFs for citation links)
Wheel cache + application: < 2 GB
Total: ~20 GB on-server SSD

3. High-Level Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│  Intranet (facility, no external routing)                               │
│                                                                         │
│  ┌─────────────┐    HTTPS/443     ┌───────────────────────────────────┐ │
│  │  Engineer   │ ──────────────►  │         Nginx (TLS termination)   │ │
│  │  Browser    │ ◄──────────────  │         + static files             │ │
│  └─────────────┘                  └──────────────┬────────────────────┘ │
│                                                  │ HTTP/8000             │
│                                   ┌──────────────▼────────────────────┐ │
│                                   │        FastAPI Gateway            │ │
│                                   │                                   │ │
│                                   │  ┌─────────────────────────────┐  │ │
│                                   │  │  Query Router (LLM or rules)│  │ │
│                                   │  └───────┬─────────────┬───────┘  │ │
│                                   │          │             │           │ │
│                              doc_question  data_query  out_of_scope    │ │
│                                   │          │             │           │ │
│                             ┌─────▼──┐  ┌───▼────┐   ┌───▼───┐       │ │
│                             │  RAG   │  │ Agent  │   │Refusal│       │ │
│                             │Pipeline│  │ Loop   │   │Handler│       │ │
│                             └─────┬──┘  └───┬────┘   └───────┘       │ │
│                                   │         │                         │ │
│                    ┌──────────────┘         └──────────────┐          │ │
│                    ▼                                        ▼          │ │
│  ┌─────────────────────────────┐            ┌──────────────────────┐  │ │
│  │  ChromaDB (local)           │            │  Ollama + llama.cpp  │  │ │
│  │  9,400 chunks               │            │  qwen2.5-multilingual│  │ │
│  │  all-multilingual-e5-large  │            │  :30b Q4_K_M         │  │ │
│  │  384-dim HNSW index         │            │  48 GB A40           │  │ │
│  └─────────────────────────────┘            └──────────────────────┘  │ │
│                                                          ▲             │ │
│  ┌─────────────────────────────┐            ┌───────────┴──────────┐  │ │
│  │  Document Store (local)     │            │  Tool Executor       │  │ │
│  │  PDFs / SharePoint export   │◄───────────│  get_asset_status()  │  │ │
│  └─────────────────────────────┘            │  get_alarms()        │  │ │
│                                             │  search_by_location()│  │ │
│  ┌──────────────────────────┐               │  PI historian query  │  │ │
│  │  Asset DB (Oracle 19c)   │◄──────────────│  oracle_query()      │  │ │
│  │  READ-ONLY service acct  │               └──────────────────────┘  │ │
│  └──────────────────────────┘                                          │ │
│  ┌──────────────────────────┐                                          │ │
│  │  PI Historian (OSIsoft)  │◄──────────────── PI REST API             │ │
│  └──────────────────────────┘                                          │ │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

4. Deep Dives

4.1 Query Router Design

The router classifies each incoming query into one of four categories. Two implementation options:

Option A — Rule-based heuristics (fast, deterministic):

def classify(query: str) -> str:
    q = query.lower()
    if any(kw in q for kw in ["according to", "manual says", "procedure", "standard", "what is", "explain"]):
        return "doc_question"
    if any(kw in q for kw in ["current", "right now", "alarm", "status", "reading", "sensor", "pump", "station"]):
        return "data_query"
    if any(kw in q for kw in ["password", "salary", "personal", "political", "external"]):
        return "out_of_scope"
    return "doc_question"  # safe default

Option B — LLM classifier (higher accuracy, +300ms latency):

CLASSIFIER_PROMPT = """Classify the following query into exactly one category:
- "doc_question": asks about documented knowledge (manuals, procedures, standards)  
- "data_query": asks for current operational data (sensor readings, alarms, asset status)
- "out_of_scope": unrelated to infrastructure operations

Query: {query}
Respond with only the category name."""

Recommendation: start with heuristics for determinism and speed; upgrade to LLM classifier if misclassification rate > 5% on a 100-query sample.

Arabic router consideration: Arabic queries use different keywords. Pre-translate the heuristic keyword list to Arabic, or use the LLM classifier (language-agnostic).

4.2 RAG Pipeline (Document Q&A Path)

query (Arabic or English)
      │
      ▼ embed with multilingual-e5-large (~10ms)
query_vector [1024-dim]
      │
      ▼ ChromaDB cosine search, top-8
      + BM25 keyword search, top-8
      │
      ▼ Reciprocal Rank Fusion (RRF) → unified top-8
      │
      ▼ cross-encoder reranker → top-4 chunks
      │
      ▼ build prompt with numbered citations
      │
      ▼ LLM generation (~15s, streaming)
      │
      ▼ faithfulness judge (async, non-blocking)
      │
      ▼ response + citations → client

Prompt template (multilingual):

System: You are the AI Assistant for the National Infrastructure Digital Twin.
Answer ONLY from the provided context. If the answer is not in the context, say so.
Always cite sources as [1], [2], etc. Answer in the same language as the question.

Context:
[1] Source: pipeline-operations-manual.pdf (p.12)
Normal operating pressure range: 40–80 PSI for distribution mains...

[2] Source: pump-maintenance-guide.pdf (p.7)  
Vibration alarm threshold: 7.5 mm/s (warning), 10.0 mm/s (critical)...

Question: {user_query}

Why reranking matters: semantic search retrieves conceptually related chunks. A cross-encoder scores each (query, chunk) pair together — it sees both simultaneously and can judge relevance precisely. Typically improves precision by 15–25% at the cost of one model inference call per candidate chunk.

4.3 Tool-Calling Agent (Data Query Path)

Tool definitions exposed to the LLM:

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_asset_status",
            "description": "Get current operational status, active alarms, and latest sensor readings for a specific asset. Use this when the user asks about a specific pump, valve, or sensor by ID or name.",
            "parameters": {
                "type": "object",
                "properties": {
                    "asset_id": {"type": "string", "description": "Asset identifier, e.g. 'PUMP-007' or 'VALVE-Station-A-12'"},
                },
                "required": ["asset_id"]
            }
        }
    },
    {
        "type": "function", 
        "function": {
            "name": "get_active_alarms",
            "description": "Get all currently active alarms filtered by severity, site, or asset type.",
            "parameters": {
                "type": "object",
                "properties": {
                    "severity": {"type": "string", "enum": ["low", "medium", "high", "critical"], "description": "Filter by alarm severity"},
                    "site": {"type": "string", "description": "Filter by site name, e.g. 'Station-A'"},
                    "asset_type": {"type": "string", "enum": ["pump", "valve", "sensor", "pipeline"], "description": "Filter by asset type"}
                },
                "required": []
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_pi_historian",
            "description": "Query the OSIsoft PI historian for time-series sensor data. Use for trend analysis or 'over the last N hours/days' questions.",
            "parameters": {
                "type": "object",
                "properties": {
                    "tag_name": {"type": "string", "description": "PI tag name, e.g. 'Station-A.PUMP-001.Vibration'"},
                    "start_time": {"type": "string", "description": "ISO 8601 datetime or relative like '*-24h'"},
                    "end_time": {"type": "string", "description": "ISO 8601 datetime or '*' for now"},
                    "interval": {"type": "string", "description": "Sampling interval, e.g. '1h', '15m'"}
                },
                "required": ["tag_name", "start_time", "end_time"]
            }
        }
    }
]

Agent loop safety:

MAX_ITERATIONS = 5  # prevent infinite loops
for i in range(MAX_ITERATIONS):
    response = llm.chat(messages=messages, tools=TOOLS)
    if response.finish_reason == "stop":
        return response.content   # final answer
    # Execute tool calls, append results, continue
    for tool_call in response.tool_calls:
        result = executor.execute(tool_call)
        messages.append({"role": "tool", "tool_call_id": tool_call.id, "content": json.dumps(result)})

return "I was unable to retrieve the required data. Please try again."

4.4 Guardrail Stack

Applied in order, from cheapest to most expensive:

Layer	What it checks	Cost	Blocking?
Input: PII scan	Names, IDs, phone numbers, coordinates	~1ms	Yes — redact before LLM
Input: Injection detection	"Ignore previous instructions" patterns	~0ms	Yes — refuse and log
Output: Format validation	Expected JSON keys present, no truncation (`finish_reason != "length"`)	~0ms	Yes — retry with stricter prompt
Output: Classification check	If retrieved docs are RESTRICTED, response carries RESTRICTED marking	~0ms	No — but adds metadata
Output: Faithfulness judge	All factual claims grounded in context	~2s (async)	No — flags in log; pages if < 0.90

Presidio integration (PII detection, offline):

pip download presidio-analyzer presidio-anonymizer -d ./wheels/
# Presidio uses its own NLP models — download the spaCy model too:
python -m spacy download en_core_web_lg --target ./wheels/

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

def redact_pii(text: str) -> str:
    results = analyzer.analyze(text=text, entities=["PERSON", "PHONE_NUMBER", "EMAIL_ADDRESS"], language="en")
    return anonymizer.anonymize(text=text, analyzer_results=results).text

4.5 Conversation Memory Management

Multi-turn sessions accumulate context. Two failure modes:

Context exceeds num_ctx (8,192 tokens) → model silently truncates → loses early conversation context
Context becomes too long → model "loses focus" on the current question

Solution — Sliding summarisation:

def trim_messages(messages: list, token_budget: int = 6000) -> list:
    """Keep system message + last 4 turns + summary of rest."""
    if count_tokens(messages) <= token_budget:
        return messages
    
    system = messages[0]
    recent = messages[-4:]  # keep last 2 user/assistant exchanges
    to_summarise = messages[1:-4]
    
    if not to_summarise:
        return messages
    
    summary = llm_call(
        messages=[system, {"role": "user", "content": 
            f"Summarise this conversation history in 3-5 sentences for context: {json.dumps(to_summarise)}"}]
    )
    
    return [
        system,
        {"role": "system", "content": f"[Prior conversation summary]: {summary}"},
        *recent
    ]

5. Trade-offs and Alternatives

Decision	Chosen	Alternative	Why chosen
Inference runtime	Ollama	vLLM	Ollama runs on Windows Server without Docker GPU setup; vLLM requires Linux + CUDA 11.8+. At 0.5 QPS, Ollama's lower throughput is irrelevant.
Vector DB	ChromaDB	Weaviate, Qdrant (on-prem)	ChromaDB is a single Python process — simpler air-gap packaging. Weaviate/Qdrant need Docker + separate service management.
Embedding model	multilingual-e5-large	all-MiniLM-L6-v2	Arabic support required. `e5-large` has strong Arabic performance; MiniLM is English-optimised.
Query router	Rule-based heuristics	LLM classifier	Deterministic and 300ms faster. Upgrade only if misclassification becomes a problem.
Reranker	cross-encoder/ms-marco	None	Single biggest quality improvement with modest cost. At 0.5 QPS, the ~500ms rerank latency is acceptable.
Conversation storage	In-memory (per-session)	SQLite persistence	At 50 users × 30 min sessions, full in-memory is feasible. Persist to SQLite only if sessions must survive server restarts.

6. Monitoring and Evaluation

What to log per request:

{
  "request_id": "uuid",
  "timestamp": "ISO-8601",
  "user_id": "hashed",
  "query_language": "ar|en",
  "route": "doc_question|data_query|out_of_scope",
  "tool_calls": [{"name": "get_asset_status", "args": {"asset_id": "PUMP-007"}}],
  "retrieved_chunks": ["chunk_id_1", "chunk_id_2"],
  "response_length_tokens": 187,
  "latency_ms": 14320,
  "finish_reason": "stop",
  "faithfulness_score": 0.91,
  "user_feedback": null
}

Alerting thresholds:

Rolling 1-hour mean faithfulness < 0.85 → alert on-call
p95 latency > 45s → alert on-call (model warm-up failure or GPU pressure)
Injection attempt detected → alert security team immediately
Corpus freshness: last ingest > 72 hours ago → warning (stale corpus risk)

Weekly eval run:

python eval.py --gold-set eval/gold-questions-50.json --model digital-twin-assistant
# Reports: accuracy, faithfulness, format-compliance, latency P50/P95/P99
# Compared to baseline in eval/baseline.json
# Blocks deployment if any metric regresses > 5%

7. Interview Talking Points

The three things that differentiate a good answer from a great one:

"The AI is read-only by design — not by accident." Many interviewers probe whether you've thought about safety. Frame read-only access as an explicit architectural invariant, not a current limitation. "We will never grant write access to SCADA without a separate human approval workflow — that's a policy boundary enforced in code."
"Corpus freshness is an operational concern, not a one-time setup." Most junior answers treat the corpus as static. Point out that new engineering standards, updated maintenance procedures, and post-incident reports need to flow into the corpus within 48 hours of approval. This requires a scheduled ingest job, staleness alerting, and change management.
"The query router is where the quality story starts." A RAG pipeline can't answer "what are the current alarms?" and a tool-calling agent doesn't know what the manual says. Without the router, one will always fail. Show you know this distinction.