The failure mode isn't bad code

The failure mode of AI-assisted development isn't bad code. Bad code gets caught. It fails loudly, or it fails the first time someone runs it, or a reviewer reads it and stops.

The failure mode is plausible code with a confident summary, where the summary is produced by the same process that wrote the code. The work looks done because the description of the work says it's done, and the description was written by the thing whose work is being described.

There's one response to that, and everything below is an instance of it: get evidence from somewhere other than the process that produced the artifact.

We've written from this spine twice already, so those two are links rather than sections. Write the control that fails is about proving a check is capable of failing before trusting the fact that it passes, and came out of a lint rule that passed continuously because it had never been able to match the thing it was looking for. Verify against the live system, not the source that produced it is about asking the running database what's true instead of reading the migration that was supposed to make it true.

Four more, from the current build of one system, none of them written up before.

Two correct tests can still contradict each other

One of our records has a derived status. A process that dies mid-operation leaves a row that still says running forever, so what a user should see is computed from elapsed time rather than read raw off the row.

Two screens read that status. One derived it correctly. The other returned the stored value.

Same record, same user, same session: failed on one screen, running on the other.

Both paths had tests. Both suites passed. Neither test compared its answer to the other's. Every test asked "is this path correct?" and none asked "do these two paths agree?" That's not a vacuity problem, which is the failure the lint-rule post is about. Both of these tests were doing real work. They were just each doing it alone.

The fix was one shared derivation used by all three read paths, plus a test that the paths agree, plus a control asserting they also agree in the ordinary case, so the agreement test can't pass by both paths being broken the same way.

The part worth sitting with is how it was found. Not by a test. Not by review. By looking at the running product and noticing two screens disagreeing about the same row.

Generalizing the implementation isn't generalizing the shape

A bug of this kind had already been fixed once. We'd extracted a single shared function and written a sweep that checks every reader calls it. That worked, and it holds.

It also couldn't see the contradiction above.

The sweep was written against the specific shared function, not against the category: a value that has to be derived rather than read raw. So a second derivation of exactly the same kind was invisible to a check built to prevent exactly that class of problem.

Extracting a shared implementation and writing a check for it produces something that reads as complete. It's complete for the instance it was written against.

The fix was to make the sweep table-driven over derivations, so adding one is a row rather than a new sweep, with controls per derivation in both directions. The check now knows what kind of thing it's checking, instead of knowing one thing.

Bookkeeping is not behavior

Three consecutive tests asserted something about an operation rather than the operation's effect. A return value. An outcome string. A count reporting what had been dropped. In each case the property that mattered was whether a person could subsequently do the thing.

The third one was hiding a real defect.

An access-transfer flow was tested by asserting that it returned consumed along with the right record id. It never asserted that a browser holding the result could then read the record.

It couldn't. The mechanism returned a hash of a credential, and restoring access needed the credential itself. A hash can't be turned back into what it hashed. The flow could not do the one thing it existed to do, and every test passed, because every test was checking the paperwork.

The fix that generalizes: a test covering a flow that grants, transfers or revokes access has to terminate in an access attempt and assert that it succeeded or failed. That's now a check that runs rather than a convention people remember, with a registry of such flows and a sweep that fails the build when a new one shows up unclassified.

It's worth being honest about the limit. The check confirms that a read was attempted in the same test. It doesn't confirm that the read used the material the transfer produced. The class is narrowed, not closed, and writing it down that way is more useful than claiming it's solved.

Describing intent instead of behavior

User-facing copy was wrong twice in two turns, both times the same way: the mechanism was described by what it was for rather than what it did.

An eviction notice told users they could reopen a lost item using the original link. The original link no longer worked. The correction told them that emailing themselves a link preserved access. That link expired in fifteen minutes and was single use.

Two in a row was enough to write down, so the rule went into our own documentation, with both examples and a pointer to the test that would keep it honest. That note claimed the test would "fail until the copy is updated."

It wouldn't. That test asserts the absence of certain words, so it fires when someone rewrites the copy, not when someone forgets to. The note described what the test was for.

That's the fourth instance of the pattern, and it's sitting inside the note written to prevent it. Nothing caught it but reading it again.

Describe what a mechanism does, not what it's for. The two feel identical while you're writing, because the intent is real. It's the thing you were trying to build. It just isn't evidence about what got built.

The spine

None of these were caught by the artifact explaining itself.

The contradiction was caught by a second screen. The blind spot in the sweep was caught by moving from an instance to a category. The flow that couldn't do its job was caught by attempting the access instead of reading the receipt. The wrong copy was caught by reading it again, cold, and asking what it actually claims.

That's the whole discipline, and it's not new. What's changed is the volume and the confidence. When an agent produces a thousand lines and a summary saying the thousand lines are correct, the summary is not evidence. It's the same source as the code, wearing different clothes.

Get the evidence from somewhere else.

Written by Synthetixis, an AI-native product studio. More on what most AI software gets wrong.