Tests First: Building the Safety Net with AI

Tests in the prompt: this works

There is clean measurement for this. A 2024 conference paper compared three situations: task description only; task description plus supplied tests; and the same again with a repair round fed by the failing tests. Two models, two established task sets, plus 1,100 competitive programming problems.

The gain is unambiguous, and the weaker the model, the larger it gets.

But there are two limits, and the authors spell them out themselves.

The first: more tests is not always better. As the number of supplied tests grows, "in certain cases code that was previously correct breaks". The cause is information lost in a long context. There is an optimum, and past it things get worse.

The second matters more: a solution that passed the supplied tests frequently failed the hidden tests. Which means the model optimises for the test it was handed, not for correctness. That is not a defect, it is the nature of the task — but you have to know about it.

Tests the agent writes for itself: this does not work

This part is sobering, and it shows the exact opposite of what you would expect.

A 2026 measurement examined six models on the same 500-task set drawn from real repositories. Two extremes:

• One model wrote itself a test on 83% of the tasks. Its resolution rate: 74.4%.
• Another did so on 0.6% of them. Its resolution rate: 71.8%.

2.6 percentage points apart — while one of them tests almost always and the other almost never. And when the heavy testers were forbidden by prompt to write tests, the outcome barely moved: a net shift of around ten tasks out of five hundred.

The explanation is the paper's real lesson: the "tests" an agent writes are largely not checks but observation tools. According to the measurement, lines that print a value consistently outnumber the assertions. This is debug printing dressed up as a test.

So the difference is not whether a test exists. It is who gets to say what the right answer is. If the model writes the test for its own code, it is checking its own interpretation.

When the test misleads the model: the test that sets a bug in concrete

Here is the most important mechanism, and it is what gives this post its title.

A 2026 study looked at what happens when the existing, buggy code sits in the prompt and you ask for a test against it. The result: the model writes an assertion for the faulty behaviour instead of exposing it. Buggy code "steers the model towards tests that confirm the faulty behaviour rather than reveal it".

And the effect cuts both ways: not only does the number of misguided tests rise, the generation of effective, bug-finding tests drops as well.

The authors measured the remedy too: put the specification in the prompt, not the code — what it is supposed to do. This "substantially reduces misguided tests while significantly increasing effective ones".

This is why "tests first" is not a matter of style. It is the only known way to keep a generated test from setting an existing bug in concrete: if the code does not exist yet, the model has nothing to copy.

One nuance, for fairness: the same study also established that most of the tests generated against buggy code, and passing on it, pass on the fixed version too — meaning a lot of tests simply pin down no behaviour at all. Earlier literature overstated this.

How tight is the net?

It is worth knowing where a test stops working as a safety net — and there is hard data on that from an academic setting.

A 2026 conference paper went through 77 patches by hand, taken from the output of three agents, on the best-curated industry task set. The results:

• 7.8% of the patches counted as "correct" while failing the developers' own test suite.
• 29.6% of the "probably good" patches produced behaviour different from the real fix.
• Overall, the reported resolution rates are inflated by 6.4 absolute percentage points.

The reason is simple: the benchmark only uses the test files that the original pull request modified — the rest of the functionality stays untested.

If that is how it goes on a carefully assembled academic set, it will not be better in your own repo. A test measures only what it measures.

When does coverage mean anything?

There is fresh measurement on this too: eleven models, thirteen configurations, 8,268 generated test suites, 101,123 unique test cases.

The finding in one sentence: the usefulness of coverage and mutation score is heavily situational.

• If the code can be assumed correct — a regression situation, where the test preserves existing behaviour — then these metrics give a meaningful signal, especially when comparing models.
• If the code may be buggy and finding that bug is the point, they are no longer reliable signals.

That fits everything above exactly. A generated test makes a good regression net. It is not a bug-finding instrument.

The good news: not flaky

A common fear, and the measurement gives it a reassuring answer. In a study run on industrial and open-source database systems, new tests were generated for more than 5,000 test files, and the share of tests that behaved unpredictably ("flaky") ran between 0.00% and 0.29%.

The main cause, where it did occur: the test relied on an ordering that is not guaranteed — in 72 cases out of 115, that is, 63%.

So the problem is not that generated tests fail unreliably. It is that they reliably catch nothing.

The funnel: 18 out of 100 generated tests

There is an industry reference for this, with published ratios. From the automated test-repair system of a large social media company:

Generated test case:                       100
Compiles:                     100 × 0.75 =  75
Runs reliably:                100 × 0.57 =  57
Increases coverage:           100 × 0.25 =  25   <- only this helps
Accepted by an engineer:       25 × 0.73 = ~18

(Our own calculation from the published ratios.)

Eighteen tests out of a hundred. And that came out of a system that ran every generated test, repeatedly, and let only the coverage-increasing ones through.

That is the key message against the "just generate tests" advice: generation is cheap, the filter is the work.

Why a failing test is worth so much

There is a 2024 study on how much it is worth for a model to repair its own code from the errors. The answer: corrected for compute cost, the gain is "modest, varies substantially across datasets, and is sometimes absent altogether".

The bottleneck, though, is not the repair ability but the diagnosis. When a stronger model supplied the feedback, results improved significantly. And the authors add: even the best models fall far short of what human debugging can achieve.

Which yields the single most important practical rule: the name and message of a failing test are external feedback, not feedback coming out of the model. Precisely the thing that unblocks the bottleneck. That is why one good failing test is worth more than ten rounds of "try again".

The recipe: a gate at four levels

1. AT THE PROMPT LEVEL
   Not "write a test for this function", but:
   "write a validateEmail function. Examples: [email protected]
    true, invalid false, [email protected] false. After implementing
    it, run the tests."
   -> the example IS the spec. Keep the code OUT of the prompt.

2. AT THE SESSION LEVEL
   A fixed goal that something checks after every round.
   "Show me the test output" - not the claim that it is done.

3. A DETERMINISTIC GATE
   A script that blocks the end of the round until it is green.
   The only item here that constrains rather than suggests.

4. A SECOND OPINION
   A separate reviewer, on a FRESH context. Whoever wrote it
   is biased in favour of their own code.

AND THE TWO-MODEL PATTERN the vendor itself recommends for TDD:
   one writes the test, the other the code that passes it.

An open question worth saying out loud

We looked, and we found no measurement of whether a repo's existing test coverage improves an agent's results on that same repo.

What we do have: a test placed in the prompt helps; a test the agent writes does not; the benchmark's test suite is too narrow. The link between coverage as a property of the repo and an agent's success rate is for now an assumption without measurement. Logical, plausible — but not demonstrated.

Common misconceptions

• "I generate tests, and the net is in place." Eighteen out of a hundred stay usable even in a strictly filtered industrial pipeline.
• "If the agent writes itself a test, that is a good sign." Measurement says it barely changes the outcome, and what it writes is mostly printing, not assertion.
• "The tests are green, so the code is good." In an academic measurement, 7.8% of patches counted as correct while failing the developers' own test suite.
• "I hand over the buggy code and have it write a test for it." That is how you record the faulty behaviour. Give it the specification, not the code.
• "Generated tests fail unreliably." Their flakiness was under 0.3%. The trouble is the opposite: they reliably catch nothing.

What to take away

A test is worth something when you are the one saying what is correct — an example placed in the prompt improves things by 7–30%, the agent's own test by practically nothing. And once the code exists, a generated test tends to record the existing bug instead of finding it. Hence the order: the failing test first, the fix after. Generation is cheap — the filtering is the work.

Setting the first job for an agent is what the first task with a coding agent post walks through; reviewing the finished code is the subject of the reviewing AI-written code post, step by step.