The AI-Era Engineering Playbook

System Engineer: Interview Scorecard


Stage 1 — Systems Thinking and Architecture

Case study discussion. Scenario provided in writing. 10 min think time, then 35 min discussion.

QuestionWeightScoreNotes
S1-Q1 — Constraint definition: what must be defined before implementation begins?
S1-Q2 — Tradeoff reasoning: where would you sacrifice and why?
S1-Q3 — Junior failure paths: what breaks first, architecturally?1.5×
S1-Q4 — Silent failure: what's invisible until production?1.5×
S1-Q5 — Why documentation: intent, not just implementation?0.5×
Stage 1: / 27.5

Stage 2 — Failure Mode Reasoning

Incident scenario. System description + incident report provided in writing. 10 min review, then 35 min discussion.

QuestionWeightScoreNotes
S2-Q1 — Hypothesis generation: top three failure candidates?1.5×
S2-Q2 — Evidence-based diagnosis: confirm or eliminate without system access?
S2-Q3 — Test suite gap: why this failure wouldn't appear in tests?
S2-Q4 — Class-level fix: architectural change, not a patch?1.5×
Stage 2: / 25

Stage 3 — AI Output Audit

60–100 lines of AI-generated code with 3 planted issues. 15 min review, then 30 min discussion.

QuestionWeightScoreNotes
S3-Q1 — Code review: find issues, explain failure behaviour (highest weight — auto-reject if 1)
S3-Q2 — Test/production gap: what passes tests but fails in prod?
S3-Q3 — Questions before approving: spec recovery, not judgement?
S3-Q4 — Test case design: specific cases for each found issue?
Stage 3: / 25

Stage 4 — Structured Behavioural

Consistent questions, same order per candidate. Score on observed evidence, not impression.

QuestionWeightScoreNotes
S4-Q1 — Most complex system designed: constraints and tradeoffs?
S4-Q2 — Wrong architectural decision: what was missed and why?1.5×
S4-Q3 — Prevented without taking over: teaching under constraint?
S4-Q4 — AI delegation line: principled boundary, not habit?
S4-Q5 — Current experiments: active early adoption with specific examples (auto-reject if 1)1.5×
Stage 4: / 30

/ 100
Score questions above to see recommendation

Debrief notes

Complete before comparing with co-interviewer. Write your answer first — this reduces anchoring to the first opinion expressed.

Most decisive positive signal Most decisive negative signal or absence If Hire with conditions — what specifically needs to develop One thing this candidate does that most candidates don't

Fill in the scores and debrief notes above, then generate a structured prompt to paste into Claude, ChatGPT, or any LLM for a calibrated debrief summary.

✓ Copied to clipboard

© Gabor Mayer. Licensed under Creative Commons Attribution 4.0 (CC BY 4.0). Free to share and adapt with attribution.