The Interview Skill Map — Pre-AI Baseline

Same methods, same axes — scored against pre-2022 skill relevance
X = how relevant the skill was then · Y = how well the method measured that skill · hover for detail

↖ Good test, wrong skillAccurately measures something that didn't matter even then. Rare in the pre-AI world — most bad tests were also inaccurate.

↗ Right skill, well measuredMost of the traditional methods sat here. This quadrant largely emptied after 2022 as skills shifted left.

↙ Double failureBad methods for irrelevant skills. These were already wrong before AI — brain teasers, "biggest weakness", culture fit.

↘ Right skill, wrong testSkills that mattered but were measured poorly. The opportunity that existed even then — and still does.

Sound method for the era

Widely used, had issues

Wrong even then

Valid but rarely used / didn't exist yet

The same interview methods mapped against pre-2022 skill relevance. Before large language models, syntax knowledge, implementation speed, and algorithm skill sat in the top-right quadrant — relevant and well-tested. This chart shows the baseline most hiring processes were designed for, and shows exactly which methods lost their signal when AI arrived.

How to read this chart

Each dot is one (skill, method) pair. Hover over any dot to see the skill, the method, the coordinates, and the research behind the placement.

The X axis scores how relevant that skill is for software engineering today — far right means essential, far left means obsolete. The Y axis scores how accurately the method measures what it claims to test — top means reliable signal, bottom means noise.

Four quadrants: top-right (right skill, well measured — use these), top-left (good test, irrelevant skill — phase out), bottom-right (important skill, poor measurement — redesign these), bottom-left (irrelevant skill, poor measurement — drop immediately).

The article series

This chart is part of a three-article series on hiring software engineers in the AI era.
Article 1 — The interview skills that AI made obsolete Article 2 — The top-right quadrant: what still works Article 3 — How to rebuild your hiring process AI-era map (2026) →

All methods — full reference table

Every (skill, method) pair plotted on the map. Relevance and measurement quality are scored −10 to +10. Hover the chart for the full note and research citation on each point.

Method label	Skill tested	How it is tested	Relevance (X)	Measurement quality (Y)	Quadrant
CV walk-through	Self-presentation	CV walk-through / 'tell me about yourself'	-1	-5	Double failure
'What's your biggest weakness?'	Self-awareness	'What is your biggest weakness?'	-1	-7	Double failure
Brain teasers	Puzzle-solving / lateral thinking	Brain teasers ('how many golf balls fit in a plane')	-7	-4	Double failure
AI output review	AI output evaluation	Code review of AI-generated code with planted bugs	-9	-9	Double failure
Fermi estimation questions	Estimation / structured thinking	Fermi estimation ('how many X in Y')	-1	1	Good test, wrong skill
Unstructured behavioural	Communication & problem process	Unstructured behavioural interview	4	-3	Right skill, wrong test
Culture fit interview	Values / cultural alignment	Culture fit interview (unstructured)	2	-5	Right skill, wrong test
Work sample test	Real job performance	Work sample test (actual task from the job)	9	9	Right skill, well measured
Paid trial period	Actual job performance	Paid trial period (1–5 days of real work)	9	9	Right skill, well measured
Take-home assignment	Real implementation ability	Take-home coding assignment (1–7 days)	8	7	Right skill, well measured
Debug approach test	Debug reasoning (hypothesis-driven)	Give broken system — observe: hypothesis or paste?	8	7	Right skill, well measured
Framework years requirement	Framework knowledge	Job req: '5 years of React'	7	4	Right skill, well measured
Failure mode exercise	Failure mode reasoning	Describe a system — ask how it fails	7	7	Right skill, well measured
Pair programming	Collaboration + real-time reasoning	Pair programming with interviewer	7	7	Right skill, well measured
LeetCode	Algorithm reasoning	LeetCode / timed puzzle	6	3	Right skill, well measured
Timed bug fix	Implementation speed	Timed bug fix (30 min)	6	5	Right skill, well measured
Online coding test (HackerRank / Codility)	Algorithm implementation	Online coding test (HackerRank, Codility)	6	5	Right skill, well measured
System design whiteboard	Systems thinking & architecture	Whiteboard system design	6	3	Right skill, well measured
Portfolio / GitHub review	Real engineering capability (past work)	Portfolio / GitHub review	6	5	Right skill, well measured
Syntax & trivia quiz	Language internals / syntax	Trivia quiz ('explain GC', 'what does X do')	5	7	Right skill, well measured
Structured behavioural (STAR)	Communication & problem process	Structured behavioural interview (STAR + rubric)	5	6	Right skill, well measured
Technical presentation (past project)	Deep knowledge of past work	Technical presentation of a past project	5	6	Right skill, well measured
Domain scenario test	Domain-contextual judgement	Domain-specific scenario: what's wrong here that tests won't catch?	5	7	Right skill, well measured
Specification test	Specification quality	Give vague req — watch what happens before any tool is touched	3	8	Right skill, well measured
Early adoption question	Early adoption / adaptability	'What have you tried in the last 3 months that nobody told you to?'	1	7	Right skill, well measured

Research backing

The Y-axis placement of each method is grounded in I/O psychology research. Key sources: Schmidt & Hunter (1998) Psychological Bulletin (meta-analysis of 85 years of selection research); Sackett, Zhang, Berry & Lievens (2022) Journal of Applied Psychology (corrected validity coefficients); Roth, Bobko & McFarland (2005) Personnel Psychology (work sample tests); Rivera (2012) American Sociological Review (cultural fit bias); Laszlo Bock / Google (2013) on brainteasers.

The X-axis reflects the documented shift in software engineering work caused by large language models. Implementation tasks (syntax recall, algorithm puzzles, fast bug-fixing) have been automated since 2022. Specification, architectural reasoning, and AI output evaluation have been elevated. The before/after comparison between this chart and the pre-AI baseline quantifies the shift.