Our lint rule kept passing because it had stopped checking anything

A guard that has stopped working looks exactly like a guard with nothing left to catch. Both are green.

One of ours had stopped working. It ran on every commit, passed every time, and was checking nothing at all.

The mistake is the boring part. The mechanism is worth writing down, because it will do the same thing to anyone who writes the same rule in the same sensible way.

The rule

We have a lint rule that keeps one part of our codebase away from a set of Node's networking modules. It bans the imports and the call shapes both: node:http, node:https, node:net, node:tls, node:http2, global fetch, and the usual third-party clients, axios, got, node-fetch. It runs in a pre-commit hook.

Banning the import is the stronger half, and it's worth saying why. A call-site ban blocks one spelling of a capability. Removing the import makes the capability unreachable.

The rule strips comments and string literals before it matches anything, so that a mention of fetch( in a comment or a fixture doesn't get flagged as real usage. That is standard practice. It is what you would do.

It is also why the rule could not detect a single import.

The specifier is a string literal

That's the whole bug, but it's worth seeing on the page.

import http from "node:http";
//               ^^^^^^^^^^^ a string literal

Strip string literals first, and what the matcher actually sees is:

import http from "";

The text the pattern was looking for is the exact text the preprocessing removed. Not sometimes, and not on certain files. Every import ban we had written could never match, anywhere, regardless of what the code did.

So the rule ran, walked the tree, and reported that it had nothing to report. Which is indistinguishable from the truth, and that's the part worth sitting with. A rule that detects nothing produces the same output as a codebase with nothing to detect. There is no signal in the passing case. There never was.

A broken guard is also worse than no guard, because a guard is a decision to stop looking. Nobody re-reads the import lines in a directory that already has a rule pointed at it.

The fix is two passes

The two classes of pattern want opposite preprocessing, so they no longer share one.

• Patterns that match import specifiers: strip comments only, leave the strings alone.
• Patterns that match call shapes: strip comments and strings.

The rule keeps one narrow exception, scoped to a single named binding rather than to the whole file, so every other spelling still fires: a default import, a namespace import, a named import that brings in anything else, and require. There is a control for each of those, which matters more than the exception does.

What caught it

Almost nothing, in the end, because the controls were written to fail. Every banned pattern has a paired test asserting the rule fires on a violating snippet. Six of them went red the moment the import bans went in.

Worth naming: this code was written by an AI coding agent, and so was the lint rule. What surfaced the problem was not a careful read of the diff. It was a control written to fail.

Write the one that fires

So, for any guard: write the test that proves it fires.

Not the test that proves clean code passes. That one is easy, it's the one that gets written, and it would have stayed green through every commit of this.

A test that has only ever passed is indistinguishable from a broken one.

Written by Synthetixis, an AI-native product studio. More on what most AI software gets wrong.