There's a job most companies don't have a title for yet. The person who makes an AI system trustworthy enough to leave running. Plenty of people can build one. Far fewer will put their name to it.
That's most of what I do.
Community Tech runs on a system I built for exactly this. It watches public tender feeds, does background research, drafts documents for me to approve and puts its own code through review, on a schedule. I've been building it since January. Getting it to produce work instead of plausible-looking mush is most of what that time went on. Here's what stuck.
The model that wrote it is the worst judge of it
Ask an agent whether the code it just wrote is finished and it will tell you yes. It isn't lying to you. Every justification for every decision it made is still sitting in front of it, so from where it's standing the work looks obviously correct.
So nothing in my setup is signed off by the thing that built it. When a piece of work is finished, a fresh agent starts with no memory of writing it, reads exactly what changed, reads what the job was meant to achieve, and goes looking for what's wrong. Any real finding means the task isn't done. Fix it, then a new reviewer, again, until a reviewer that didn't build the thing can't find anything left.
That single change caught more real problems than everything else on this list put together. It's also the one people skip, because it feels like paying twice for the same work.
Two numbers from my own logs. Across one run of twenty six separate pieces of work, five came back from independent review with a real defect in them, after their own tests were already green. And on one stubborn job the review loop went eleven rounds and turned up ten genuine defects, three of which were created by the fixes for the earlier ones. Had I stopped at the first clean-looking fix, I'd have shipped three problems that didn't exist when I started.
One more rule from the same place: if a fix hasn't worked twice, the approach is wrong rather than the wording. Left to itself an agent will happily try the same fix a third time with a variable renamed.
One company's model checks the other's work
Models from different companies don't fail in the same places. They were trained differently and they're confident about different wrong things. So I have one company's model check the other's work, and the reviewer starts fresh every round.
That rule lives in the code rather than in my head. If I set the reviewer to be the same thing that did the building, it ignores me and picks something else. Whatever my reason was that day, it isn't as good as the reason not to.
Underneath that sits one more rule. A reviewer that comes back clean has to show it actually looked: which files it opened, how much of the change it covered. Below a minimum the round is recorded as unreliable rather than as a pass. Otherwise "clean because the code is fine" and "clean because it never really read it" arrive looking identical.
When several reviewers raise the same thing in different words, those get grouped together and labelled, because three reviewers independently flagging one line is a much stronger signal than three separate bullet points. What it deliberately doesn't do is raise the severity because three of them said it.
And when the builder and the reviewer are still arguing about the same point after three rounds, it stops and asks me. That exit is there because the alternative is the two of them quietly agreeing on whatever ends the conversation fastest.
"Done" has to mean something you can check
The failure that took me longest to spot is the confident summary of something that never happened.
One of mine reported that a message had gone out that morning, with a time on it. It had found the switch that permits sending, seen the switch was on, and reasoned forward from there. Nothing had ever been authorised to send. No message had left the system in three weeks. The switch was real. The send wasn't.
Existence is not behaviour. Finding a function, a setting or an endpoint proves it exists. It doesn't prove anything ever calls it.
So any claim about how the software behaves now has to trace the whole path, from where a thing is defined to where it's actually called in real code rather than in a test, and cite the file and the line number so I can check it in ten seconds. The word "verified" is only allowed after reading the code that runs. Everything else gets labelled as a guess, and guessing is fine as long as it says so.
There's an automatic version of that too. A review finding is only allowed to block a piece of work if it quotes the code it's complaining about, character for character, and that quote is then checked against exactly what the reviewer was shown. That record is saved before the review starts, so it can't be quietly recalculated afterwards to make a finding fit. If the quote isn't in there, the finding drops to advice and stops being able to block anything.
The same suspicion applies to the agent's own sign-off. A "complete" that contains the words "partially", "still failing" or "couldn't run the tests" is demoted automatically. And a passing test run only counts while the code still matches what was tested. Edit anything afterwards and the proof expires.
A green tick is a claim, not a result
If the model that wrote the code also wrote the tests, a green tick tells you those two agree with each other. There's more on that in the bugs that pass review.
I narrowed one line in a function that moves an item between folders in a password vault. In production that change would have wiped a real stored credential. The suite stayed green: 885 tests out of 885, including one whose whole purpose was to prove that the password survives a move. It passed because the test wasn't talking to a real vault. It was talking to a pretend one, and the pretend one answered from its own memory rather than from what it had just been handed.
So when a test claims to cover something that matters, I break the code on purpose and watch that test go red. If it doesn't go red, it wasn't covering anything.
Every rule in my setup is a scar
The instruction file my agents load is an incident log.
On the fifth of August one of them told me a server was refusing connections and asked me to go and sort it out by hand. A second automation picked that up and passed it on as urgent. The server was healthy the entire time. My connection was being blocked, and a blocked connection prints exactly the same message as a machine that's dead. It cost me a morning.
There's a section about that now. Reproduce the failure from somewhere else before you call anything down. Stop your own retries before you time anything, because plenty of blocks reset every time you knock, so a retry loop keeps you locked out and makes "I waited ten minutes and it still failed" meaningless.
Then there's the research document. Some of the rules I've just described started life in a document I had AI write for me. When I went back and checked its sources properly, eight of its figures were wrong. One statistic was credited to the wrong company. Another described ordinary human-written code and was being quoted at me as evidence about AI-written code. The corrections sit in that file now rather than being tidied away, because a document about not trusting confident summaries should probably keep the receipt.
Both are the same mistake. I took a confident summary at face value because it arrived formatted and sure of itself. So when something goes wrong now, the fix goes in writing as well as in the code, with the date and what it cost, because the next agent has never met me and will make exactly the same assumption.
Nothing goes out until I've said yes
I learned this one the embarrassing way, by having a calendar invite I'd never approved turn up in someone's diary.
Now anything a person can see, an email, a message, a meeting invite, needs my approval of the exact words, the exact recipient and the exact time before it moves. The daily runs that draft anything a person will read can flag it, queue it and stop themselves, but there's no route from them to an outbox. And "sent" is only true once something has gone and read the message back out of the Sent folder.
That invite is the only thing on this page that reached somebody outside. The rest cost me time, which is the trade all the checking is there to make.
If you're putting an agent anywhere near a customer, that boundary is the part to build first, before the clever bit.
Keep the noise down
Running a full test suite dumps a few thousand lines into the conversation, almost all of it passing tests nobody needs to read. The model loses the thread of what it was working on, and the quality of everything after that drops off.
So every project gets a quiet version of that command. One line if everything passes. Only the failures, in full, if it doesn't. The detail still goes to a file for anyone who wants it.
Most of the "it got worse halfway through the afternoon" problem I used to have was that. Me, filling its head with logs.
The part that's still a person's job
None of this is an argument about whether AI is any good. It writes most of my code now and I wouldn't go back. The difference is that nothing I hand over has only ever been looked at by the thing that wrote it.
What it hasn't replaced is the person who decides what "good enough" means before a customer finds the edge case for you, who works out where a human has to step in, and who can explain why the thing did what it did when it goes wrong at four on a Friday. That's a real job. It just doesn't have a name yet, and most people are learning it by accident after a pilot nearly takes something down.
If you've had an app or a system built largely with AI and you're not certain what's underneath it, that's what our AI Code Rescue service is for. We'll read it properly and tell you plainly what we find, including when the answer is that it's fine and you should leave it alone. Get in touch and we'll set up a chat.