The AI-Era Engineering Playbook

The AI-Era Engineer's Playbook — Part 6

The Prompt That Reveals Everything

Two developers on the same team got the same ticket last month: "Add rate limiting to the public API."

The first one opened his agent and typed, near enough, the ticket. "Add rate limiting to our public API endpoints." He got back a clean middleware, a token bucket holding its counters in process memory, tests, and a config block. It shipped that afternoon.

The second one did not open her agent for about ten minutes. When she did, the prompt was most of a page. Limits per API key rather than per IP, because the customers behind corporate NAT would otherwise share a bucket. State in Redis rather than in process, because we run four instances and an in-memory limiter means the real limit is four times what we advertise. Return 429 with Retry-After. Fail open if Redis is unreachable, log loudly, because a rate limiter that takes down the API when its own dependency dies is worse than no rate limiter. Different limits for the write endpoints. Failed auth attempts limited separately by IP, so the limiter does not leave a credential stuffing hole wide open.

Both prompts produced working code. Only one produced correct code.

What is interesting is that nobody caught it. Both PRs looked good, and the in-memory counter was findable by anyone who thought to ask what happens with more than one instance. Nobody asked. The gap was not in the code, and it was not in either developer's prompting technique. It was upstream of both, in a decision the ticket never surfaced.


The prompt is not the skill

There is an industry built on the idea that prompting is a craft you can learn. Templates, frameworks, role prefixes, "act as a senior engineer," few-shot examples.

Some of that helps at the margins. None of it is the thing that separated those two developers.

The second developer did not write a better prompt because she knew a better prompt format. She wrote a better prompt because she had thought about corporate NAT, and horizontal scaling, and what a rate limiter should do when its own datastore is down. The prompt was a transcript of thinking that had already happened.

This is why we keep saying the prompt is a diagnostic rather than a skill. It does not create understanding. It exposes it. Whatever you have or have not worked out before you start typing, the prompt shows it, in the same way a written specification has always shown it.

Which leads to an uncomfortable use for your own history.


Read your last five prompts

Go and look. Not the conversation, just the opening message of your last five real tasks.

For each one, ask: does this contain anything that was not already in the ticket?

The honest answer, for most of us most of the time, is no. We paste the requirement, add a sentence about the stack, and press enter. That is the Dumper pattern wearing a different face: prompt first, understand later, if at all. The prompt is a restatement. And a restatement cannot contain more understanding than the thing it restates, which means the agent is now making every decision the ticket left open, silently, according to whatever is most common in its training data.

That is the actual mechanism behind the failure we described in Part 4. The agent did not get it wrong. It got it unspecified, and unspecified resolves to plausible.


Requirements and specifications are different objects

We use the words as though they are synonyms. They are not, and the difference matters more now than it ever did.

A requirement describes what someone wants. "Add rate limiting to the public API." It is a statement of intent, written by someone who is allowed to not know how it works. It is complete as a request.

A specification describes what the system must do, in enough detail that two competent implementers would build the same behaviour. Not the same code; the same behaviour. It answers the questions the requirement left open, and it names the ones it is deliberately leaving open.

Almost everything that arrives in a ticket is a requirement. Almost nothing that arrives in a ticket is a specification.

For most of us and most of our careers this did not matter much, because the gap got closed during implementation. You started building, you hit the question, you asked someone or you decided. The specification did get written. It just got written incrementally, in your head, in the order the code demanded it, and it never existed as a document. That is the important part: the thinking happened, but always fused to the act of building, never as a separate step you could perform on its own. Some fields always wrote real specifications as documents, and they are about to look prescient.

Take away the implementation loop and the gap does not close. It gets filled by the agent instead, using a default you never saw and never approved.


What a specification has that a requirement does not

Five categories. Most underspecified prompts are missing several of them.

Valid inputs. What is in range and what is not. Not the type; the domain. An integer is a type. "Between 1 and 1000, and 0 means unlimited" is a specification.

Expected outputs. Including the shape of the thing on the boundary. What comes back for an empty result set is a decision, and "empty list" versus "null" versus "404" are three different systems.

Failure definition. What counts as failure at all, and what the system does about it. This is the one people skip most, and it is where agents are weakest, because they will invent an error path that looks reasonable and is wrong for your domain. Fail open or fail closed is a business decision, not a technical one.

Performance and scale assumptions. How many, how fast, how concurrent. The rate limiter above is a pure example: the entire correctness of the naive version depends on an assumption about instance count that nobody wrote down.

Dependency behaviour. What happens when the thing you depend on is slow, absent, or lying. Every system has this and almost no ticket mentions it.

You can test this on any requirement in about two minutes. Take it, go through the five, and write down what it does not say. Some tickets genuinely are well specified. Most are not, and the ones that feel most complete are usually the ones worth checking hardest.


The same ticket, three ways

Requirement, as it arrived:

Add rate limiting to the public API.

The prompt most people write:

Add rate limiting to our public API endpoints. We use Express.

The prompt that produces the right code:

Add rate limiting to the public API.

Scope: per API key, not per IP. Several customers are behind corporate NAT and would otherwise share a bucket.

Limits: 1000 requests per hour on read endpoints, 100 per hour on writes.

Failed authentication: cannot be attributed to a key, so limit those separately by IP. This conflicts with the NAT decision above, and we are choosing the tighter option deliberately: 100 failed attempts per hour per IP, which a shared office will not hit under normal use but a credential stuffing run will. Leaving auth failures unmetered is the worse trade.

State: Redis, not in-process. We run four instances behind the load balancer, so an in-memory counter would let through four times the advertised limit.

When exceeded: 429 with a Retry-After header giving seconds until the window resets.

When Redis is unavailable: fail open and log at error level. A limiter that takes the API down when its own dependency dies is worse than no limiter.

Out of scope for now: per-endpoint overrides, burst allowances.

Most of those blocks map onto the five categories. The 429 with Retry-After is the expected output. The failed auth rule and the Redis fallback are failure definition. Four instances is the scale assumption. Redis is the dependency.

Two of them do not map, and that is worth noticing rather than forcing. Scope is a prior question the five categories assume you have already answered: what is the unit being measured. And the final block is not a category at all.

Notice what the long version is not. It is not longer because it is more polite, or because it uses a better template, or because it tells the agent to think step by step. It is longer because it contains seven decisions that the one-line prompt delegated by accident.

Notice also the last line. Naming what you are deliberately not doing is part of a specification, and it is the part that stops an agent from helpfully building it anyway.

And notice what it still does not cover. The auth limiter also lives in Redis, so when Redis fails open, credential stuffing goes unmetered at exactly the moment someone is most likely to be probing. That is a real gap, and it is there on purpose: a specification is never complete, only complete enough for the decisions in front of you. The skill is not producing an exhaustive document. It is knowing which unanswered questions are the expensive ones.


Why this is the hardest of the three skills

Part 3 laid out what the new interview tests: specification quality, output evaluation, failure-mode reasoning. Of those three, specification is the one people find hardest to develop, for a reason worth naming.

Output evaluation has a target. You are looking at something, and either you find the problem or you do not. Failure-mode reasoning has a prompt: someone asks how it breaks, and you answer.

Specification has neither a target nor a prompt. You are trying to notice the absence of something, before anything exists, with no external cue that anything is missing. The requirement looks complete. That is what makes it dangerous. Nobody ever feels the gap, because a gap is precisely the thing that does not announce itself.

Part 2 measured which skills degrade under delegation. Specification is not on that list, and the reason is worth naming: it cannot atrophy the way debugging did, because it was never a standalone habit to begin with. It only ever ran attached to implementation.

There is also an uncomfortable diagnostic buried in this. If you cannot specify it, you do not understand it yet. Not the code; the problem. The developer who cannot say what should happen when Redis is down has not thought about what the rate limiter is for. That was always true, and implementation used to hide it, because you could start typing and work it out on the way.

You cannot work it out on the way any more. The way is gone.


How to practise it this week

Write the spec before the prompt, in a separate place. Not in the agent window. Somewhere you cannot start building. Five headings, the five categories above. It takes a few minutes and it will feel like overhead until the first time it catches something expensive.

Count the decisions your prompt delegated. After you write a prompt and before you send it, read it back and count how many of the five categories it leaves open. That number is how much of the design you just handed to a statistical model.

Practise on tickets you are not going to build. Take any ticket from your backlog, run it through the five categories, and move on without building anything. It is cheap and fast, and unlike the other two it does not require having real work in front of you. The interview prep coach runs this as a scored drill in your own domain if you want the feedback loop.

Read the question banks as specifications. The Product Engineer Question Bank and System Engineer Question Bank were written as interviewer tools, but every question in them has anchored strong and weak answers, which is what a specification of a good answer looks like. Reading them backwards teaches the shape.


Where this lands

Part 5 argued that the reading habit built by writing code is the wrong one for evaluating agent output. This is the same argument on the other side of the work: the specification habit was never built as something you could do on its own, because implementation always carried it.

Both halves come down to the same thing. The middle of the job disappeared, and it was carrying two skills that nobody had to teach because everyone acquired them as a side effect.

Now they have to be taught. That is what the rest of this series is for. The next article stops describing the problem and starts on the method: what a specification an agent can actually execute looks like in practice, section by section, on a real feature.


© Gabor Mayer. Licensed under Creative Commons Attribution 4.0 (CC BY 4.0). Free to share and adapt with attribution.