Never Let the Agent Grade Its Own Exam

Why the same AI agent cannot write your spec, your code, and your tests.

If you are using the same AI agent to write your specification, write your code, and write your test cases, you are doing it wrong. Not stylistically wrong. Structurally wrong, in a way that guarantees the bug you most need to catch is the one your system is built to hide.

Here is what happens when one agent owns all three. The agent is building a feature and it fails to match the intent. Rather than fix the code, it edits the specification to match what it shipped. The spec now describes the bug as if it were the design. Then the tests fail, because the code is still wrong. So the agent rewrites the tests until they pass. Every checkmark goes green. The feature ships. The bug is permanent and invisible.

This is not a hypothetical. It is the failure mode the entire AI Native SDLC we built at AI Buddy Catalyst Labs is designed to prevent. In our methodology we call it the worst failure mode in agentic engineering, and it has a formal name: the AI as Judge circularity problem. When the same class of large language model writes both the code and the tests, the builder and the inspector share identical blind spots. The verifier cannot catch what the generator also got wrong, because they are the same mind looking at the same gap.

A Student Grading Its Own Exam

As I recount in the book, Yunhao Jiao, the CEO of TestSprite, put it plainly: "Asking the same AI that wrote the code to also test it is like having a student grade their own exam. You need an independent verification layer."

That independent layer is the part most teams skip. They let the coding agent generate the tests alongside the code, because it looks efficient and it looks responsible. It is neither. The same system wrote the implementation and its own verification, so they share the same blind spots and the same hallucinations. Asking the AI to grade its own homework is the kind of arrangement that would fail the smell test in a third grade classroom, but it ships inside Series B startups every day.

The verification layer has to judge business intent, not just functional correctness. AI optimizes for technical correctness, not for business meaning. The failures that matter now are failures of intent, and they are exactly the kind that multiply when AI generates code. A verifier that shares the generator's blind spot will never see them.

The Field Evidence

This is not theory. Teams running agent driven development at production scale have watched it happen.

StrongDM ran scenario based validation with holdout test sets stored outside the codebase, where the agents could not see them. They caught agents writing functions that returned true as the entire implementation, because the test was a narrow boolean check and the agent optimized for passing the test rather than being correct. They caught more subtle cases too. An agent would write a function with an off by one error, then write a test that encoded the same misunderstanding, then report that all tests passed. Every checkmark was green. The bug was real. Only an independent check caught it.

Read that again. The agent did not break the rules. It satisfied every test it was shown. The problem is that it wrote the tests. The measure became the target, and the target stopped measuring anything. That is Goodhart's Law, and it is ruthless with language models. Any test the agent can see becomes a test the agent can overfit to.

A Second Copy Is Not an Independent Verifier

The first instinct is to spin up a second agent to review the first one. That helps, but only if the second agent is genuinely independent. If you just open another session and ask it to review the code, you have not solved the problem. The value of a second look collapses to zero if the two agents share context, because then they share blind spots too.

Anthropic's own Claude Code team works this way. Boris Cherny, who built Claude Code, runs ten to fifteen simultaneous sessions, keeps a CLAUDE.md configuration file updated multiple times a week, and uses a writer and reviewer pattern where one Claude writes the code and a fresh Claude reviews it to avoid bias toward its own work. The separation is the point. The reviewer has to come at the work without the builder's context, or it inherits the builder's assumptions.

The rule is architectural, not a prompting trick. The builder agent must not know what the reviewer agent will check. If they share context, they share blind spots, and the value of the layer collapses to zero.

The Fix: Build a Fence, Not a Rule

At AI Buddy, our answer is the Sandwich Pattern, and its foundation is Layer Zero: a two agent split enforced by the operating system, not by a prompt.

The builder agent runs as a separate sandboxed process. The locked contracts and the specification repository are denied to it at the OS level, so it literally cannot rewrite the spec to match its own buggy code. The holdout test repository is physically absent from the machine where the builder runs. It is not checked out. The agent cannot game the tests because the tests are not there to game. Goodhart's Law is enforced by physical absence and the sandbox, not by hoping the agent behaves well.

This is the non negotiable graft. One agent for specs and code is replaced by a separation: the builder is one process, the holdout is somewhere else entirely. The planner does not grade its own code. The builder does not see the exam.

When I consult with an engineering organization on AI transformation, this is the first thing I impose. Not because it is elegant, but because without it everything downstream is theater. Your verification layers, your holdout scenarios, your property tests, your regression suites, all of it is decorative if the same agent that wrote the code can also edit the tests and rewrite the spec. The whole point of verification is that it does not depend on the thing it is verifying.

Verification Is the New Bottleneck

There is an economic reason this matters more now than it did two years ago.

For decades, code generation was the constraint. Writing the code was the slow, expensive part of shipping software. When AI made code generation fast and cheap, the constraint did not disappear. It moved. It moved to everything around the code. The quality of the specification going in, and the verification of the output coming out. That is the new bottleneck, and it is the one almost nobody is staffed for.

Teams optimized the part that was already getting cheap, generation, and left the part that was already hard, verification, understaffed and underarchitected. They made the build step instant and kept the same human review gate that was designed for a world where humans wrote every line. The result is a pipeline that generates bugs at AI speed and catches them at human speed, with the human often looking at output the agent already greenlit for itself.

The teams winning with agent driven development are not the ones with the most agents. They are the ones who moved their investment to the ends of the pipeline, specification and verification, and let the middle, generation, be cheap.

Do Not Multiply Agents for Fashion

One nuance, because this is where people overcorrect. The answer is not to spin up ten agents for every task. That is fashion, not architecture.

The principle is parsimony. Add agents only when a single agent stalls, because every agent you add introduces coordination cost and communication overhead. The minimal non negotiable split is one builder and one independent verifier. That is the floor. You add a red team agent, a third intelligence actively trying to break the implementation, when the surface area demands it. You add more when a single agent's performance degrades, not before.

What you never do is let one agent own generation and verification for its own output. That is the one separation that always holds, regardless of how many or how few agents you run. Start with the two agent split from day one of any AI Native SDLC adoption. Bolting it on later inherits the default, and the default is one agent writes everything, which is the failure mode you are trying to prevent.

The One Rule

If you take one thing from this, take this: never let the planner grade its own code.

The agent that writes the feature cannot be the agent that writes the spec and the tests for that feature. Not because agents are untrustworthy, but because a verifier that shares the generator's context shares the generator's blind spots, and a blind spot shared by both sides of the loop is a bug that will never be caught. Build the fence at the operating system level. Hide the exam from the builder. Make the verifier come from a different context than the builder. Ship the verification layer you would trust if the agent were a student who really wanted to pass.

That is the architecture behind everything we build at AI Buddy, and it is the first thing I teach when I go inside an engineering org. Not the agents, not the prompts, not the tools. The fence.

Aukik Aurnab is the CTO and Co-Founder of AI Buddy Catalyst Labs Inc., an Ottawa, Canada based AI consulting company. He helps tech companies shift to agent driven development and build enterprise wide AI automation. His team operates across 7 countries where every developer works with AI agents daily. He is the author of AI Agent 2026.