Eighteen minutes for a LeetCode hard. Clean approach, correct complexity reasoning, narrated every step while he wrote it. The panel was impressed. We made the offer that week.
Three months later we were quietly cleaning up behind him. The shape of it was always the same: a ticket came back marked done, the feature worked on the happy path, and something surfaced a week later. A retry loop that never actually retried. A total that was right in staging and wrong in production. An error that was caught and then silently swallowed.
He could not explain what the code did, because he had not written it. He had been pasting tickets into an AI tool, accepting the first answer, and calling it shipped.
We have seen versions of this at more than one company since. The details change. The interview does not.
The interview worked exactly as designed. The design is the problem.
There is a name for the gap that produced that hire. Interview drift is the distance between what a hiring process tests and what the role actually requires. Drift is not the same thing as a sloppy process — ours was well run, consistently scored, and staffed by good engineers. It measured carefully. It measured the wrong things.
Drift builds up slowly, and then something moves the target all at once. We call that a relevance shift: the point at which a capability becomes abundant enough that testing for it stops telling you anything. Implementation speed crossed that line somewhere around 2022, and most hiring processes have not been re-pointed since.
Four of them are worth going through in detail.
The algorithm question
The technical interview has a founding myth: that asking candidates to solve algorithmic puzzles under pressure reveals something real about how they think.
It probably did, once. Translating a problem into an efficient algorithm takes structured reasoning, and that signal mattered when writing the algorithm was the bottleneck.
But we should be honest about how well it ever worked. When we scored the standard methods on how accurately they measure the thing they claim to measure, the algorithm round came out at 3 out of 10 — and that was its score before AI. It was always a weak instrument. What changed is that it is now a weak instrument pointed at a skill that is no longer scarce.
We are not going to quote a benchmark at you, because you can check this yourself in about ten minutes. Take the last three problems your panel used and paste them into whatever model your engineers already have open. When we did this with ours, they came back correct on the first attempt, with tidier complexity analysis than most candidates produce out loud under time pressure. The problems where the model still struggles are the genuinely novel ones, and those are not the problems on your list — yours came from the same few hundred that everyone practises.
Which points at the real issue. Six weeks of preparation gets a candidate through this round whether or not they can build software. A candidate who passes an algorithm round today has demonstrated one thing clearly: they prepared for the algorithm round.
There is a second problem that most panels talk around. Nearly every company bans AI assistance during a live interview. In a remote interview that ban is unenforceable, and everyone in the call knows it. So the round now sorts candidates on two axes: how much they prepared, and how willing they are to ignore a rule nobody can check. Neither is what you were trying to measure.
The cost of keeping it is not zero. Every hour spent on algorithm questions is an hour not spent on something that predicts job performance. If you are keeping this round because it is the only thing that scales to your applicant volume, that is a fair reason, and we deal with it further down.
Stack knowledge
"Minimum 5 years of React experience."
This requirement feels safe. It is concrete, it is easy to filter on, and it rules out candidates who obviously cannot do the job.
It also rules out the people we increasingly need most: domain experts who move between tools, fast learners who have been through two frameworks in three years, engineers who picked up the right new thing before it was obvious enough to hire for.
The useful half-life of specific framework knowledge is now roughly one to three years. Not the framework's lifetime — the lifetime of the answer. The specific APIs, the patterns, the "right way to do it in this version" all rotate underneath you, faster now that AI generates most of the boilerplate.
React, which feels permanent, will end up where jQuery is. That is worth being precise about, because jQuery is still running on a large share of the web and people are still paid to maintain it. Frameworks do not disappear. They stop being where the value is. Someone with deep React knowledge remains employable — they are simply no longer scarce, and scarcity is the thing a hiring filter is supposed to find.
Here is the practical version. AI generates framework code. Give a capable engineer a new framework and a model and they are productive in days, not months. What you pay a premium for when you filter on stack depth is the memorised answer to a question the job rarely asks anymore.
The engineers who are genuinely valuable understand why frameworks work the way they do. They can pick up a new one. They bring judgement, not recall. A stack requirement used as a filter selects against exactly that profile.
Syntax and trivia
"Explain how garbage collection works." "What's the difference between a process and a thread?" "What does this keyword do?"
This is the one category where the method was never the problem. A syntax question measures recall accurately — it scored 7 out of 10 for measurement quality in our pre-AI baseline, one of the highest on the board. It hit what it aimed at.
The trouble was always what it aimed at. Recall stood in for understanding, on the theory that someone who knew the internals would be better at using the surface. Sometimes that held. It was never a tight relationship, and it is now a broken one: the information is a prompt away, and retention of it tells you nothing about judgement, specification quality, or whether someone can evaluate generated code for correctness.
A well-built instrument pointed at something that stopped mattering is the cleanest case of drift there is. This category can go. It frees time for questions that cannot be answered by asking an AI.
The timed bug fix
Give the candidate a broken piece of code. Thirty minutes. Fix it.
This is the one to be careful with, because it is the easiest to over-correct, and we have watched teams over-correct it.
What the timer measured was speed of mechanical implementation: fast hands, pattern recognition under stress, reading unfamiliar code quickly. That is genuinely less relevant now, when a model reads code faster than any human and produces a plausible fix in seconds.
What the exercise missed entirely was the part that now matters most. Does this person understand what correct looks like before they see it? Can they describe what the fix should achieve before writing it? Can they notice that the proposed fix addresses the symptom and not the cause?
So the thing to remove is the stopwatch, not the exercise. Debugging is one of the few traditional technical skills whose value has gone up — AI writes more code faster, which produces more bugs faster, and finding them now requires a mental model that a fast producer of AI output does not have. In our own scoring, a debugging exercise sits in the strongest quadrant both before and after the shift. It is the framing that has to change: stop timing how quickly they reach a fix, start watching whether they form a hypothesis in the first two minutes or start throwing prompts at the symptom.
That distinction is the whole of Part 2, so we will leave it there.
The map
Those four are not outliers. They are a pattern: methods built for a bottleneck that moved, still running on inertia.
We scored every common interview method on two axes — how relevant the underlying skill is to the job today, and how well the method actually measures that skill — once for the pre-2022 world and once for now. The distance between the two pictures is interview drift, made visible.
A word on what these charts are and are not, because it matters for how much weight you put on them. The positions are our scores. They are not survey results and we are not going to present them as such.
The measurement axis leans on the selection-validity research where it exists — work samples and structured interviews have decades of it behind them, and brain teasers have Google's own finding that they predict nothing at all. The relevance axis is a judgement call about 2026. We have written the reasoning for each point into its hover note, so you can disagree with a specific dot rather than with the chart as a whole. That is the more useful argument to have.
The pre-2022 picture genuinely does look different. Take-home assignments, pair programming, work samples and structured behavioural interviews sat high and to the right — the right skills, measured reasonably well. Several of them still do. The four methods above are the ones that moved.
If you want to score your own process rather than ours, that is what the Interview Drift Toolkit is for.
"But we screen a thousand people"
This is the first objection we get, and it is the right one.
The algorithm round did not survive on merit. It survived because it is cheap, runs in parallel, needs no senior time, and produces a number you can sort on. That is a real operational requirement, and any replacement has to clear the same bar. "Have a thoughtful conversation about failure modes" does not scale to eight hundred applicants, and pretending otherwise is how good advice gets ignored.
Two things do clear it.
A short asynchronous work sample. On our map the work sample scores highest of any method on both axes, before and after the shift. Not a four-hour take-home — a forty-minute task drawn from something your team actually shipped last quarter. It is gradeable against a fixed rubric, it does not need the candidate and the interviewer in a room at the same time, and it is much harder to prepare for than an algorithm round because there is no public bank of them.
A review task with planted defects. Give everyone the same forty to sixty lines of AI-generated code with the same three problems planted in it, and ask what they would change before shipping. The objective half grades itself — did they find the three? — which is what makes it survivable at volume. The interesting half, how they read, comes later with far fewer people in the pool.
Then keep the funnel honest. The expensive judgement exercises belong at stage two, against a much smaller group. The mistake is not that the algorithm round is at the top of the funnel. The mistake is treating a volume filter as though it were a quality signal.
"But we need to defend these decisions"
The second objection is quieter and usually arrives from HR rather than engineering. Standardised tests exist partly to make hiring decisions consistent and defensible. Replacing them with open-ended judgement exercises sounds like the opposite of that.
It is worth separating two things that get bundled together. The risk is not judgement. The risk is unstructured judgement — different tasks for different candidates, no rubric, and a decision assembled afterwards out of impressions.
Structured interviewing is one of the best-evidenced methods in the whole selection literature, and it is more defensible than an algorithm round, not less. The requirements are unglamorous and non-negotiable: every candidate gets the same task, scored on the same rubric, against criteria written down before anyone was interviewed, with each interviewer recording their score before the group discusses. That is what the scorecards are for.
Structure is what makes a decision defensible. Puzzles were never the thing doing that work.
What this is not
This is not an argument that algorithms, frameworks, syntax and debugging speed are worthless. Engineers who understand these things deeply still carry real advantages, and for some System Engineer roles the algorithm intuition is genuinely load-bearing.
The question is whether a ninety-minute interview is the right place to measure them, and whether they are the scarce skills worth the weight they currently carry.
The interview is a limited resource. Spending it on questions whose signal has degraded is a choice with a cost, and the cost is not spending that time on something that tells you what you need to know.
What to change first
If you do one thing after reading this, make it the smallest one.
Take your current process and write down, for each round, the skill it is supposed to measure. Not the format — the skill. Then ask two questions of each line: is that skill still scarce, and does this round actually measure it?
Most teams find one round that fails both tests. Remove that round before adding anything. You will free ninety minutes and lose nothing, and it is a great deal easier to get agreement on than a redesign.
Then take the stopwatch off the debugging exercise and leave everything else alone until you have read Part 2.
What goes in the space you just cleared? That is Part 2.
This article builds on an earlier argument about where the role is heading: The Future Software Engineer Will Not Be a Programmer. That piece was about what the job is becoming. This series is about the practical consequence, starting with which interview questions are still worth asking.
How many of these engineers you actually need, and how teams get structured around them, is a separate question — we took it up in Why Adding AI to Your Existing Team Structure Doesn't Work.
Tools referenced in this article: the Interview Drift Toolkit for scoring your current process, the Pre-AI Baseline Map and 2026 Skill Map for the underlying data, the Hiring Audit Worksheet for running the round-by-round audit above, and the Product Engineer and System Engineer question banks for what to ask instead.
© Gabor Mayer. Licensed under Creative Commons Attribution 4.0 (CC BY 4.0). Free to share and adapt with attribution.