Refute, Don't Review

I asked it to check my work. It checked that my work existed.
- A reviewer who agreed with everything, including the bug

Two weeks ago I wrote about forty "obsolete" ticket verdicts that looked airtight. A second pass of agents, told to prove each one wrong, overturned thirteen. Last week I wrote about a rewrite we talked ourselves out of by making the AI argue the other side.

Both turn on the same move. I use it in enough places now to call it a method. This is the field guide.

The Move

Ask an agent "is this right?" and it looks for reasons to agree. It usually finds them. Ask a fresh agent "prove this wrong" and it looks for the gap.

Four details make it work:

Fresh context. The refuter isn't the agent that did the work, and doesn't see its reasoning.
"Prove this wrong," not "check this." The instruction sets what it goes looking for.
Default to not confirmed. If it can't find conclusive evidence the claim holds, the answer is no. The burden of proof sits on the claim.
Give it the original requirement, not the first agent's summary. A summary is already framed by the answer. The ticket, the spec or the bug report is what the claim has to satisfy.

Everything below is that move pointed at a different kind of claim.

Variant 1: Agent Verdicts

That's the Sep 28 story. The misses were mostly scope misses: proof that part of the work was done, read as proof that all of it was.

It happened again on a smaller batch this summer. Three tickets in a batch had come back marked "no work needed." A refute pass found two of the three were wrong. One had been called "already fixed." Part of it was. The rest still had live gaps, and the comment justifying the exclusion was wrong about the code it described, which is why the gap existed.

Two of three is a small sample. I don't read it as a rate. I read it as the same failure showing up in a second place.

Variant 2: The Merge Gate for "Nothing Uses This"

An absence claim is the riskiest verdict an agent makes. "Nothing calls this." "Safe to delete." A search that finds nothing looks the same as a search aimed at the wrong place.

So when one of my agents opened a change that only deleted code, the review wasn't a general review. It was a single reviewer briefed to refute the absence claim, defaulting to not confirmed. It didn't refute it. That result is what earned the merge.

The preparation mattered too. A parallel cleanup of the same code, merged first, had kept three exports because their names appeared elsewhere in the project. They weren't the same things. Each was declared locally by the one file that used it, with an incompatible shape. A name search can't tell those apart, so all twelve deleted symbols were checked one at a time. The reviewer's own find: six stylesheets the first commit had left orphaned.

Variant 3: Your Own Claims, Before You Write Them Down

The claims I trust least now are the ones an agent writes into a plan, a comment or a commit message. Those outlive the session. A wrong "why" in a permanent document becomes the next person's bug.

One plan on a client's platform shows what refuting a design before any code looks like. Four independent review passes went at it adversarially before anything was built. The plan keeps a "Retracted Claims" log. It lists six. Two were retracted during the original investigation. The other four were knocked down by that review:

"There are exactly two places a timestamp becomes a string." There were five.
"Both paths produce the same format." They didn't, confirmed by running them.
A risk flagged as the biggest unknown was verifiable the whole time. The source was in the repository.
A promised one-step fix had nothing to hook into.

The plan's direction survived review unchanged. Its mechanics didn't. That's the right place to learn it: in a document, not in production.

The opposite case cost real time. On a money-path ticket, an agent asserted a timezone hazard from reasoning alone. Then it "corrected" itself with a second mechanism it also never measured. Hours went into relitigating a claim nobody had tested. A passing test proves the fix. It doesn't prove the explanation. The explanation needs its own evidence, or it gets written down as unverified.

Variant 4: Make "Nothing Found" Find Something

That same incident had a second failure. A probe for the timezone problem returned zero matches, and the agent read that as confirmation. The row it queried had never been inserted. One count query would have shown it.

The fix is a positive control: before you believe a search that found nothing, show the same search finding something you know is there. On the deletion-only change above, the clean type-check only counted after a probe that imported a deleted symbol produced an error. The instrument could see the thing it was looking for.

A control proves the probe works. It doesn't prove you pointed it everywhere. One sweep across fifteen repositories was positive-controlled and found zero readers of a data column. It was never run against the column's twin on the other side of the system, which had nine wired readers and a write endpoint. That cost two reversals in three weeks. Who checks the checker is its own post, coming in a few weeks.

Variant 5: Three Ways This Design Is Wrong

My rule: before an agent builds a non-trivial design, the plan gets a written section titled some version of "three ways this design is wrong." The agent has to attack its own design before it writes code.

The entries are concrete. One plan named a data-isolation hole it could open, a payload that might not carry the data it needed, and a query fan-out that could get expensive. Each came with a mitigation or a check to run first.

It's the cheapest variant, and the one I can't point to a clean catch for. I keep it because the others keep finding things.

Variant 6: Refute Your Own Tests

A test that can't fail proves nothing. The way to find one is to break the code on purpose and see if the test notices.

A few examples from this summer:

A review replaced an agent's entire fix with a simpler wrong expression. All 13 tests still passed. The test data made the right value and the wrong value identical.
A one-character change on the line that picked a money figure left all 61 tests green. For an account holding exactly zero, it would have brought the original bug back.
A test named for the fix passed with the fix deleted. The expected total and the wrong total were both 150.

In one review, two agents disagreed about whether a check could catch a mutation. One reasoned about it and agreed with the author. The other ran it. The one that ran it was right.

One caution: a mutation check only proves what the test setup ran. If your tests run with different settings than production, a caught mutation in tests can still slip through in production. One of my projects hit this with a database driver setting.

When It's Worth the Cost

A refute pass costs another agent and more of your attention. I spend it here:

Before bulk or irreversible actions. Closing forty tickets, deleting code, dropping anything.
On absence claims. "Zero callers," "no longer used," "nothing found."
On anything that becomes permanent text. Plans, comments, commit messages.
On money and data. Anything where a wrong answer can't be taken back.

I skip it for display work. A layout tweak or a label change gets built, tested for the reported bug and handed to a person, who confirms it in 30 seconds. When the logic moves real money, everything has to be right. When it's a display of that data, other people and a fully integrated environment catch problems faster and cheaper than I can.

One Developer, One Chat Window

You don't need a fleet for any of this.

Open a new chat. Paste the original requirement and the claim. Ask it to prove the claim wrong, and to say "not confirmed" unless it's certain.
When the AI says "nothing uses this," ask it to find something that does use it with the same search.
When it explains why a bug happened, ask what it measured.
When it writes a test, delete the fix and run the test.

Each takes a few minutes. Each has caught something for me that a friendly review didn't.

The shift is small. Stop asking the AI whether it's right. It usually says yes. Ask a fresh one to prove it wrong, and believe the answer when it can't.