Hiring Audit Worksheet
How signal-efficient is your current interview process?
What This Is
A self-assessment tool for engineering leaders to evaluate their existing technical interview process against the AI-era framework. The output is a single signal-efficiency score and a prioritised list of changes.
Time required: 30–45 minutes to complete honestly.
Who should complete it: The person who owns your technical hiring process. If that's a committee, complete it independently then compare — disagreements are useful data.
Part A: Process Inventory
List every step in your current technical interview process. Include phone screens, technical screens, live coding sessions, take-homes, system design rounds, behavioural interviews, reference checks — everything.
For each step, complete columns 2–6 using the reference tables in Part B.
| # | Step name | What skill does it test? | Skill relevance today (1–5) | Method validity (1–5) | Effective signal score (col 4 × col 5 ÷ 5) |
|---|---|---|---|---|---|
| 1 | |||||
| 2 | |||||
| 3 | |||||
| 4 | |||||
| 5 | |||||
| 6 | |||||
| 7 | |||||
| 8 |
Total interview hours (approximate): —
Sum of effective signal scores: — / — (max = number of steps × 5)
Process signal efficiency: sum ÷ max × 100 = — %
Part B: Reference Tables
Use these to fill Part A columns 4 and 5. If your step doesn't appear here, use the definitions at the bottom to assign scores yourself.
Skill Relevance Today (Column 4)
How relevant is the skill being tested to actual job performance in 2026?
| Skill | Score | Rationale |
|---|---|---|
| Specification quality (writing testable behavioural requirements) | 5 | Primary bottleneck in AI-era development |
| AI output evaluation (reviewing AI-generated code for correctness) | 5 | Daily job for all roles |
| Failure mode reasoning (what breaks, when, and why) | 5 | Core System Engineer skill; critical for Product Engineers |
| Early adoption behaviour (self-directed tooling experimentation) | 5 | Highest-value hiring signal; compounds over time |
| Hypothesis-driven debugging | 4 | AI generates more bugs; debugging is now a differentiator |
| Systems design and constraint identification | 4 | Higher leverage as implementation moves to AI |
| Domain knowledge | 4 | Required for specification correctness; not replaceable by AI |
| Data modelling | 4 | AI generates bad schemas; humans need to catch them |
| Communication and specification clarity | 4 | Specification is communication; both are now primary skills |
| Code quality and structural thinking | 3 | Still matters; style automates, structure doesn't |
| API design | 3 | Relevant but not primary |
| General problem-solving process | 3 | Relevant but hard to isolate from other signals |
| Algorithm implementation (DSA / LeetCode) | 1 | AI automates; passes are indistinguishable from AI-assisted passes |
| Framework-specific knowledge ("5 years of React") | 1 | Half-life 1–3 years; AI generates idiomatic framework code |
| Language syntax and API recall | 1 | A prompt away; measures memory, not judgement |
| CV walk-through / "tell me about yourself" | 1 | Measures rehearsal quality, not engineering capability |
| Brain teasers / Fermi estimation | 1 | Google's own research: zero relationship to job performance |
Method Validity (Column 5)
How well does this method actually measure what it claims to measure?
| Method | Score | Validity basis |
|---|---|---|
| Live AI-assisted build (observed) | 5 | Closest to actual job task; r=.44 (Schmidt & Hunter 1998) |
| Work sample — realistic task (code review, spec writing) | 5 | r=.33–.54 (Roth et al. 2005); high ecological validity |
| Structured behavioural interview (consistent questions, rubric) | 4 | r=.42 (Sackett et al. 2022) |
| Failure mode / incident review | 4 | High ecological validity; directly mirrors real diagnosis work |
| Technical presentation of past project (deep follow-up) | 4 | High resistance to fabrication; behavioural evidence from past |
| Specification exercise (write requirements from vague input) | 4 | Observable output; directly testable against criteria |
| Portfolio / GitHub review (with structured evaluation) | 4 | r=.33+ for well-structured review; evaluator quality is the variable |
| System design (restructured: constraints + failure modes, not patterns) | 4 | High validity when format is ecological; lower when whiteboard-only |
| Unstructured behavioural interview | 2 | r=.38 (Schmidt & Hunter 1998); amplifies impression management and bias |
| Take-home assignment (code output evaluated) | 2 | AI completes these undetectably; output may not reflect candidate at all |
| Timed coding exercise (speed metric) | 2 | Measures implementation speed, which is no longer the bottleneck |
| System design (whiteboard, pattern recognition format) | 2 | Extroversion and drawing ability add noise; ecological validity is low |
| Online coding test (HackerRank / Codility) | 1 | AI assistance is undetectable; filters for AI use, not engineering reasoning |
| Algorithm puzzle / LeetCode | 1 | Measures interview prep; no published correlation with job performance |
| Brain teaser | 1 | "Zero relationship" to job performance (Bock, Google, 2013) |
| CV walk-through | 1 | r≈−0.5 for deception (Weiss & Feldman 2006: 81% of candidates lie) |
| "What is your biggest weakness?" | 1 | Near-universally answered with a rehearsed non-answer |
Part C: Signal Analysis
C1. Redundancy Check
Are you testing the same skill more than once?
| Skill | Steps that test it (from Part A) | Is the duplication adding signal? |
|---|---|---|
| Y / N | ||
| Y / N | ||
| Y / N | ||
| Y / N |
Total redundant steps: —
C2. Coverage Gap Check
Which high-relevance skills are you NOT currently assessing at all?
| High-relevance skill (score 4–5) | Currently assessed? | Steps that should cover it |
|---|---|---|
| Specification quality | Y / N | |
| AI output evaluation | Y / N | |
| Failure mode reasoning | Y / N | |
| Early adoption behaviour | Y / N | |
| Hypothesis-driven debugging | Y / N | |
| Domain knowledge | Y / N |
Total gaps (skills rated 4–5 not currently assessed): —
C3. False Signal Check
Which steps are consuming significant interview time with low effective signal?
From Part A, list all steps where effective signal score ≤ 1.5:
| Step | Time spent (hours) | Effective signal score | Candidate impact if removed |
|---|---|---|---|
Total hours spent on low-signal steps: —
Part D: Output Summary
Your Current Score
| Metric | Your value | Benchmark |
|---|---|---|
| Process signal efficiency | — % | Target: ≥65% |
| Hours on low-signal steps | — hrs | Target: ≤20% of total |
| Coverage gaps (high-relevance skills untested) | — | Target: 0 |
| Redundant steps | — | Target: ≤1 |
Recommended Changes (prioritised)
Based on your answers above, list changes in priority order. Start with the highest-impact, lowest-cost changes.
Priority 1 — Remove (low signal, replaceable):
| Step to remove | Hours recovered | Replace with |
|---|---|---|
Priority 2 — Restructure (right skill, wrong method):
| Step to restructure | Current method | Better method |
|---|---|---|
Priority 3 — Add (high-relevance skill not currently tested):
| Skill to add | Recommended method | Source |
|---|---|---|
| Specification quality | Specification exercise (Stage 1, Product Engineer guide) | 05-product-engineer-question-bank.md |
| AI output evaluation | Code review with planted issues (Stage 3, both guides) | 04-system-engineer-question-bank.md |
| Failure mode reasoning | Incident scenario review (Stage 2, SE guide) | 04-system-engineer-question-bank.md |
| Early adoption behaviour | Structured behavioural Q5 (both guides) | 03-early-adopter-framework.md |
Part E: Honest Assessment Questions
Answer these before finalising your change list. They surface issues the scoring doesn't catch.
1. What does your current process select for — not what you intend, but what candidates who pass it have in common?
Answer:
2. Think of the last engineer you hired who underperformed. What would a better interview have caught?
Answer:
3. Think of the best engineer you know who would struggle with your current interview. What does that tell you?
Answer:
4. What does your current process not test at all that has turned out to matter?
Answer:
5. If a candidate spent six weeks specifically preparing for your interview, what skills would they improve that have nothing to do with the job?
Answer:
Part F: Baseline Score
Record your pre-change score here. Revisit after implementing changes.
| Before | After (date: —) | |
|---|---|---|
| Signal efficiency % | ||
| Hours on low-signal steps | ||
| Coverage gaps | ||
| Redundant steps |
Reference
Full method validity data: 06-skill-map-research-backing.md Assessment theory: 04-assessment-theory.md Skill relevance data: 01-traditional-skills-weighted.md | 02-new-skills-weighted.md Interview guides: System Engineer | Product Engineer
© Gabor Mayer. Licensed under Creative Commons Attribution 4.0 (CC BY 4.0). Free to share and adapt with attribution.