← Zahir Khan · Artifact · August 2026

The Agent Verification Checklist

Ten checks I run against any AI agent's claim of "done" before I believe it. Point it at a pull request, a status report, or an agent transcript. I write this from a software delivery seat, and most of it carries over to any report you are asked to believe.

Every one of these comes back to the same failure. An agent, or a person, or a vendor says a thing is finished, and the only evidence is the sentence itself. That is a claim wearing verification's clothes. So the test I apply to every item here, including the list itself, is whether I can name the exact thing that would prove it false. If I can't, it doesn't belong.

Two things this list refuses to pretend. Evidence costs time, and a standard nobody can afford gets quietly dropped, so I run this against the claims I would have to explain if they turned out to be wrong. And a receipt can lie. Attaching evidence is the start of the argument rather than the end of it, which is why half these items are about the quality of the receipt instead of its existence.

There is one reason agents need this more than people do. When a person overstates what they finished, you can hold them to it afterwards, and that accountability does some of the work for you. An agent gives you nobody to hold. The verification has to carry the whole load on its own.

Tick an item only when you can name the receipt, and write it in the box. An item with a tick and no receipt doesn't count, which is the whole argument of this page applied to the page itself.

1. Reproduced before touchedIf this is a fix, I saw the failure myself, in a log line or a raw response or a screenshot, before any code changed. An agent reporting that something was broken is not a reproduction of it.
2. Live-probed, not narratedEvery claim about something a user can see has a real tool call behind it, and the call has to show correctness rather than existence. A page that loads can still be showing the wrong number. An API that answers can still be answering with garbage. I want the response with the value visible in it, or the query result, not a sentence describing what should have happened.
3. One claim, one checkEach item on the done list resolves with a single yes or no test, and it gets checked as the work lands rather than all at once at the end. If I can't say which exact test answers an item, it isn't specific enough to trust yet, so it gets split until it is.
4. The plan existed in writing firstThere is a plan I can still open, dated before the first change, and somebody or something other than the builder left a mark on it. If no such record exists, this item fails. Catching a bad approach early is cheaper than catching a bad result late.
5. A genuinely different check, not the same one twiceOn anything I would have to explain if it broke, the second look came from a different angle or a different reviewer. The same process asking itself the same question and agreeing with itself proves nothing.
6. Checked where it actually runsThe evidence came from the real environment, at the version that is live now, after the change landed. A green run on an old build, a cached page, or the right answer from the wrong branch is evidence about something else entirely.
7. Told what must NOT happenAt least one item on the list is a failure if it becomes true. A door that should stay locked, a regression that shouldn't come back. A checklist of only positives misses this whole class of bug.
8. Re-read against the original askAfter everything checks out technically, I go back to what was asked for at the start and confirm nothing on that original list got quietly dropped along the way.
9. Every "looks fine" has a receipt behind it"Should work." "Looks fine." "No errors." Those are good summaries of something I can open for myself. They are a red flag as the whole answer. When the phrase is doing the work the evidence should be doing, I go and look.
10. Deferred items say so out loudAnything I genuinely can't verify right now gets marked unverified, with a plan for when it gets checked. It stays out of "complete" even when checking it was inconvenient.

Everything you tick or type is saved to this browser only, using localStorage. Nothing is sent anywhere, and reloading the page won't lose it.

Where this comes from

I run a home server and a set of automated workflows for my own projects. Market screeners for my own use, side builds, small sites like this one. The failure mode I keep guarding against isn't the agent doing the wrong thing. It's the agent, or the report, or the pull request, confidently telling me it did the right thing with nothing behind the sentence. The stakes are lower on my own projects than they are in my day job, and the failure is identical. On my own projects I am allowed to talk about it.

About this page

An AI agent drafted this page, working from my practice and my edits, and I ran the list above against it before publishing. Item 2 held up, because the checkbox behaviour here was driven in a real browser with the console watched, rather than described to me in a summary. Item 5 failed on the first attempt. The review I asked for came back from the same model family as the agent that wrote the page, which is the same process agreeing with itself, and it took a second opinion from a different vendor to notice that two items on this list were saying nearly the same thing. Item 7 caught me as well. I had written down what must not appear on this page, then wrote one of those things anyway, and a reviewer found it before I did.

Tell me I'm wrong

If one of these ten is wrong in your experience, I want to know which one and why. Find me on LinkedIn. A list that argues every claim should be falsifiable ought to be willing to take the same treatment.

← Back to zahirkhan.com