Interview Prep — File 06: Behavioral STAR Stories

====================================================

Prepared STAR stories for each core competency of a Senior Staff ML Engineer.

Customize with your own specific numbers and outcomes.

Format per story:

S — Situation (context, role, scale)

T — Task (what specifically was needed)

A — Action (what YOU did, technical depth)

R — Result (quantified outcome, business impact)

Competency 1: Technical Leadership & Driving Cross-functional Impact

Story: "Tell me about a time you defined the technical direction for a major ML initiative"

S: I was the tech lead on a 6-person team tasked with deploying a next-gen vision model on Snapdragon 8 Gen 2 chips for real-time on-device inference. The product deadline was 6 months away; existing INT8 quantization gave 28% accuracy drop on our key task — unacceptable for production.

T: I needed to define a quantization strategy that met both the accuracy target (< 1% drop from FP32) and the latency target (< 12ms on the HTP) within the team's expertise and timeline.

Ran a systematic layer-by-layer sensitivity analysis using a cosine similarity probe (layers with CosSim < 0.99 got flagged). Found that 4 attention layers were responsible for 80% of the accuracy loss.
Designed a mixed-precision strategy: INT4 for feed-forward layers (70% of parameters), FP16 for attention (sensitive), INT8 for embeddings.
Introduced AWQ for the INT4 layers to handle activation outliers — reduced the W4A8 accuracy gap from 3.2% to 0.7%.
Built an automated sensitivity + mixed-precision pipeline that ran overnight and produced a deployment artifact.
Presented the strategy to VP Engineering with accuracy/latency/model-size Pareto frontier, got alignment on the FP16 attention exception.

Shipped on time with 0.6% accuracy delta (vs 28% with naive INT8).
Latency: 9.4ms on HTP (21% under the 12ms target).
The sensitivity pipeline became the company-wide standard for quantization decisions — adopted by 3 other teams.
Model file size: 72% reduction (350MB → 98MB).

How to sell it: Emphasize the systematic, data-driven approach (not guessing), the cross-functional alignment (VP buy-in), and the artifact that outlasted the project.

Competency 2: Debugging Complex, Ambiguous Problems

Story: "Describe a time you diagnosed a subtle accuracy regression"

S: Three days before a major product release, model accuracy on a customer-facing benchmark unexpectedly dropped from 76.2% to 71.4% — a 4.8% drop that appeared overnight with no code changes.

T: Find the root cause within 48 hours, propose a fix, and either revert or patch without delaying the release.

First, verified the regression was real: re-ran evaluation twice with different seeds to rule out measurement noise. Confirmed it was consistent (±0.1%).
Checked git blame on the evaluation script — found a silent tokenizer version bump from transformers 4.38 to 4.40 that changed the special token handling (BOS token was now doubled in 4.40 for this model family).
Wrote a minimal repro: 10-example test that showed the score difference was entirely explained by the tokenizer change (forward pass outputs were identical).
Options: (a) pin tokenizer version, (b) fix evaluation harness, (c) re-quantize model for new tokenizer. Chose (b) — fix the harness to strip the duplicate BOS — since the model itself was fine.
Added a tokenizer version assertion to CI to prevent silent future changes.

Found and fixed in 18 hours; release was not delayed.
Added regression test that has caught 2 similar tokenizer-related issues in subsequent months.
Wrote an internal post-mortem that became required reading for all eval harness contributors.

How to sell it: Show methodical hypothesis elimination. Interviewers love when you describe the dead ends too.

Competency 3: Delivering Results Under Constraints

Story: "Tell me about a project where you had to make significant technical trade-offs under time pressure"

S: Joining a 2-month sprint to optimize a transformer model for on-device deployment. I was given a quantized model that was 4× over the memory budget (1.2 GB actual vs 300 MB target) with no timeline extension possible.

T: Reduce the model to fit in memory while keeping accuracy within 2% of FP32 baseline.

Week 1 — profiled where parameters actually lived: 60% in feed-forward, 30% in attention, 10% embeddings. The 4B parameter model was too large even at INT4.
Decided to do structured pruning to 70% sparsity on FFN layers (2:4 NVIDIA sparse format + NPU structured pruning for Snapdragon). This required a 3-day fine-tuning run.
Week 2 — combined pruning with INT4 quantization (AWQ). Result: 285 MB, within target.
Accuracy: 3.1% drop — over budget. Tracked it to pruning ratio being too aggressive.
Week 3 — ran per-layer sensitivity analysis; found 8 FFN layers were contributing disproportionately to the drop. Reverted those to 50% sparsity.
Final: 310 MB (+3% over target), 1.8% accuracy drop.
Negotiated with PM: 310 MB was acceptable given the HW roadmap would add 64MB in next chip revision.

Shipped within the sprint: 310 MB, 1.8% accuracy delta.
The pruning + INT4 recipe was documented and reused for 5 subsequent model deployments.
Learned: always negotiate the target before optimizing, not after. (Said this directly in my feedback.)

Competency 4: Influence Without Authority

Story: "How have you influenced a technical decision you didn't own?"

S: A partner team was planning to adopt a simple round-to-nearest INT8 quantization approach for a 7B LLM deployment. Their accuracy estimate was "good enough" — but they hadn't measured on our specific distribution.

T: I believed their approach would fail on our domain-specific queries, but I had no direct authority over their roadmap.

Spent 3 hours running a targeted experiment: took 200 examples from our internal test set and measured accuracy with their INT8 scheme vs. GPTQ INT4. GPTQ INT4 was 2.1% more accurate AND 2× smaller.
Requested 30 minutes at their team's tech review. Presented a one-page comparison: accuracy table, latency numbers, model size. No opinion — just data.
Offered to contribute: "If you're open to it, I can add GPTQ support to your quantization pipeline in a week."
They agreed to a 1-week experiment. I implemented GPTQ in their pipeline, validated on their test set.
The GPTQ INT4 model became their deployment candidate.

2.1% accuracy improvement in production.
My contribution was acknowledged in their team's Q3 review.
Established a pattern: cross-team quantization benchmarking before committing to a strategy.

How to sell it: Emphasize that you led with data, not opinion. Offered concrete help rather than just criticism.

Competency 5: Growing Others

Story: "How have you grown junior engineers technically?"

S: A new hire (2 years experience) was assigned to run model accuracy evaluations. They were producing results with high variance (±3%) because they were running without seeds and using different tokenizer configs.

T: Upskill them on rigorous eval methodology without micromanaging or demoralizing.

Code reviewed their eval script and left detailed comments — not "this is wrong" but "this particular seed issue caused a 12% variance in my experiment — here's the math."
Shared the relevant sections from the lm_eval documentation and our internal best practices doc.
Pair-programmed for 2 hours: walked through McNemar's test for statistical significance, why paired tests matter, how to set seeds correctly.
Gave them ownership: "You're now the team's eval accuracy owner. When anyone runs an eval, you review the methodology."
Monthly 1:1s to review their eval results together and discuss statistical interpretation.

Variance in their evaluations dropped from ±3% to ±0.15% within 2 months.
They wrote the team's "Evaluation Best Practices" document that's now onboarding material.
In their 6-month review, they were promoted to senior engineer 6 months ahead of schedule.

Competency 6: Handling Failure / Resilience

Story: "Tell me about a time a project failed and what you learned"

S: I led an effort to train a specialized code generation model using QLoRA to run on-device. After 4 weeks of training, the model performed well on held-out code, but completely failed on the production metric (user satisfaction, measured by human evaluation).

T: Understand the failure, decide whether to continue or pivot.

Root cause analysis: the training dataset was scraped from public GitHub (clean, structured code). Production queries were messy, partial code snippets with typos and context-less fragments.
Distribution mismatch was the root cause — not the model architecture or the QLoRA setup.
Organized a postmortem with the team: framed it as a data strategy failure, not a personal failure. Identified 3 things we'd do differently.
Proposed a pivot: collect 10k examples of actual user queries (with consent), re-annotate with correct completions, retrain with 20% domain-specific data mixed in.
Presented the pivot to leadership: here's what we learned, here's the revised plan, here's the revised timeline (+3 weeks).

The pivot worked: production human eval score went from 52% (original model) to 71%.
I was recognized for the transparency and the clean postmortem process.
Added "distribution match analysis" as a required step before any fine-tuning project.

What to emphasize: Taking responsibility without catastrophizing. Showing the systematic approach to root-causing and the proactive communication with leadership.

Quick STAR Bullets (for follow-up prompts)

"Tell me about a time you disagreed with leadership." → Story: Disagreed with a 2-week deadline for INT8 quantization that would've required skipping calibration data collection. Presented a data risk analysis (expected 4% accuracy drop vs 0.5% with proper calibration). Leadership extended the timeline by 1 week. Shipped with 0.6% accuracy delta.

"Describe a time you had to learn something quickly." → Story: Given 2 weeks to become proficient in Qualcomm AI Hub before a major partner demo. Systematically worked through all Hub tutorial notebooks, ran each example, built a custom pipeline integrating our internal model. Demo succeeded; became the team's AI Hub expert.

"Give an example of owning a process improvement." → Story: Noticed eval results were taking 8+ hours and blocking releases. Profiled bottleneck: redundant tokenization (re-tokenizing the same dataset for every model). Implemented a tokenized dataset cache. Evaluation time: 8 hours → 45 minutes. Adopted by the full ML platform team.

Preparation Checklist

Practice each story out loud — aim for 2-3 minutes per story
Have quantified numbers memorized: accuracy deltas, latency improvements, team size
For each story, prepare a 30-second "elevator version" for follow-up
Know the "failure" story cold — it's often the most memorable
Ask yourself: "Would a senior engineer find this technically impressive?" — if yes, keep it
Avoid: vague language ("we improved performance"), team credit without your role, stories that are pure process (no technical content)

Model Accuracy & AI Performance