An agent's report is not evidence.

I stopped taking
an agent's word for it.

It was not the first time, only the most recent.

In 2025 agents rarely did everything they were asked. Sometimes the code arrived with every setting fixed in place, where it needed to be adjustable. Ask for a line of text in green and you got the text, in black. Worse, an agent would sometimes invent an answer instead of reading the code in front of it, and the next agent would take that invention as fact and build on it. One wrong guess could snowball through a whole run.

Models are far better now, and a false "done" still costs what it always did. So every workflow I build now starts from one rule:

"The harness must assume agents will cheat. The harness must verify and never trust agent prose."

A harness is everything wrapped around the agent: the instructions it works from, the checks its work has to pass, and the rules about what it may and may not touch. Mine never takes the agent's word for anything.

Proof has to come
from the work.

Once you stop believing the report, the work itself has to be the evidence.

When an agent says it finished some code, the evidence is the code. When it says it finished some writing, the evidence is the writing. A summary of what the agent thinks it did is still only a claim.

It also puts a job on the person running the agent: know the model, and point it narrowly, at a clear specification, the actual code or a fact from the organization's shared knowledge base. An agent working from a vague brief fills the gaps itself, then reports the gaps as finished work.

Even with good evidence, the report itself still gets checked, by a second agent. I call it an honesty review: it checks what the agent says about the work, one claim at a time. It catches things like a task reported as complete when the project notes it was supposed to update were never touched.

Two habits make it effective. The agent answers in a fixed format, listing each thing it was asked to do and the evidence that it did it, so a gap is obvious. And the agent is told up front that an honesty review will judge that answer. The difference is noticeable, even though I have not counted it. An agent that knows its account will be checked makes sure the account is accurate.

I wrote that into my own working rules in May, in words I would still use:

"Do not trust anything the agents do, they will skip steps and lie. You must keep them honest."

Here is the honesty contract every agent on one of my larger builds receives word for word, with that project's own tools taken out so you can use it as it stands:

1. Every claim in your final report includes proof you can see: the command and its output, the file and line, the saved change. Never write "I did it" or "I checked it" without showing what proves it.
2. If a step fails, report exactly how: the error text, the code it returned. Do not paper over it, and do not say someone else must act unless you tried to fix it first and show why each attempt failed.
3. If you did something other than what the plan said, say so separately: what the plan said, what you did instead, and why.
4. Never stop at "still waiting". Finish the run, or explain in a structured report why it cannot finish.

A "blocked" is accepted only when it shows what was tried, the exact error each attempt produced, and the one thing the agent cannot get for itself. Anything short of that goes back to the agent.

Checking everything was expensive
until the agents learned.

All this checking costs something, and I learned how much on a cloud migration: moving a company's systems in Microsoft Azure from one setup to another.

One workflow had to copy out a description of everything the company ran in Azure, survey what it found, and write blueprints that could rebuild it all somewhere new. The first version ran one honesty review and one adversarial review, where a second agent attacks the result looking for weaknesses. The results kept coming back incomplete, and a full day went by with no visible progress.

So we tripled the checking: six agents reviewing each export. At first that cost about ten times as much as a single agent working alone, and most of that went into the agents learning how to do the job properly.

Without the harness, an agent failed to finish about 40% of the time. Nothing a business depends on can run at that rate.

What brought the cost down was a rule I added to the instructions every agent in the project reads. When an agent meets the same problem a second time, the fix has to be written down, either as a better way of doing the task or as a fact in the project's wiki. Before that rule I watched agents fix the same thing over and over. After it, the system started learning. Errors people find get written in the same way.

The other change was in the rhythm of the work. At first we tested a few migration tasks at a time, and every step after them hit the same failures fresh. So we changed it to run a whole batch, collect every failure, fix them all at once, and then run the batch again.

Once the agents had learned the job, the checking could adapt. The workflow now adds extra review wherever an agent cuts corners and pulls it back wherever agents do the job well. The cost settled at about twice that of a single agent working alone, for work that gets finished.

Tests written by agents
need checking too.

Cheaper checking still depends on what the checks are. The most familiar check of all is the automated test and that is where the next problem was hiding: agents write most of the tests.

"All green does not mean you are testing the right thing."

AI writes far too many tests. Some check something a person said in a meeting two weeks ago. Some check that the product works. From the outside they look the same, and a pile of the first kind can stay green while the product breaks.

So every so often the tests need clearing out, the same way a team prunes features nobody uses. Most can go. This is the instruction I used on my own:

"This repo is full of low-signal unit tests. Delete every one that wouldn't catch a real bug our E2E tests miss. Fan the work out across parallel subagents."

Unit tests check one small piece of code on its own. End-to-end tests run the product the way a real user would. After the cleanup, three rules go into the instructions so the next agent does not write the same tests again:

1. Never write unit tests after you write code.
2. Highly prefer E2E tests as the sole testing mechanism. Use them to verify complex features work. At the end of E2E tests, produce a verifiable and repeatable artifact.
3. If you must test a system in isolation, first write down all the ways it could fail, then write the code.

The second rule is the whole idea in one line. The proof is not a green tick. It is something a test leaves behind that another person, or another agent, can check again.

The same idea closed out the Azure migration. Before anyone started testing by hand, a script compared the settings on the old side and the new side. Everything had to match, and every difference had to be one we expected, such as a new account ID. A difference nobody expected was a failure, whatever the agent had reported. Only then did people start testing.

Some checks
never come out.

As the agents get better, most of that checking shrinks. A few pieces stay no matter how good the models become.

Checks against layering stay. Left alone, an agent records one decision, then a second that reverses it, then a third that changes the second, and nothing says which one is current. I wrote about that problem in A second brain needs an owner to build it.

Guards against dangerous actions stay. Before an agent can change something in Azure, or send data that changes another system, or do anything else you would never want an AI doing unsupervised, a check runs first and blocks it. The guard never asks the agent whether the action is safe. It simply refuses.

And the organization's own ways of working stay: the procedures a model could never know, and anything that gives the business its edge. Those are the reason the harness exists at all.

People set the goal
and sign off the result.

So where do people fit? I want the automated checking to go as far as it possibly can without a person involved.

Anthropic said on 28 August that misbehaviour in a model which can be measured can be reliably fixed, while subtle or rare failures may have no benchmark to catch them at all. I agree, and it tells me where people belong: at the beginning and at the end.

At the beginning, the agent interviews the person. I call it reverse prompting: the model asks the questions, the person answers them, and the goal is settled before anything gets made.

In the middle, the agents check each other: honesty reviews, adversarial reviews, scoring against a written standard, and scheduled reviews that go back over past work and tidy up what was learned.

At the end, the person shapes what was made and signs it off, and can expect it to be the best the task allows. That is where I caught the website rebuild.

The guardrails, tests and review around all of this are where most of the design work goes.

One rule holds all of it together.

"an honest blocked is always better than a fake claimed done."

A fake "done" is the worst outcome there is, because the agent is not being honest with you. Fake blockers are real too, though. Push too hard on "stop if you are unsure" and an agent learns to call everything blocked. The instructions have to reward the true answer, whichever way it falls.

So when an agent on your team tells you it has finished, what does it have to show you, and who checks it before anyone believes it?