Insights

Your human in the loop may have stopped looking

Putting a person in charge of signing off AI output doesn’t make it safe. A tool nobody ever disagrees with is a tool nobody is reading.
28 Jul 2026
AI, Risk and governance
A dark outline of an eye, closed, with three blue lashes, inside a dotted circular arrow.

A human in the loop only counts if the person has time to read, evidence to argue with, and clear rules on when to accept, check or escalate. Watch the override rate. If it falls to nothing, it's more likely that people have stopped looking than that the tool has become perfect.

“There’s a human in the loop” has become the standard answer to almost every question about AI risk. It’s a reasonable answer. It’s also one of the least tested.

If the person approving the output clicks yes on everything without reading it, that isn’t oversight. It’s a signature.

An old problem with a new tool

In 1983 the psychologist Lisanne Bainbridge wrote a short paper called “Ironies of Automation”. Her point was that when you automate most of a job, you leave people with the parts that are hardest to do, while giving them less and less practice at doing them. They spend most of their time watching a system that’s usually right. Then, on the rare occasion it’s wrong, they’re expected to spot it immediately and step in.

Aviation has learned this the hard way more than once. The same thing applies to a claims handler reviewing AI recommendations, or a solicitor checking an evidence pack the system assembled.

The risk is gradual. On day one, people check everything, because the tool is new and they don’t trust it. By month six, it’s been right a few hundred times in a row. Checking starts to feel like a waste of time. Support quietly turns into automation, and nobody made a decision to let that happen.

Treat disagreement as a good sign

Most AI dashboards treat overrides as a problem. The tool suggested something, a person changed it, so the tool must be wrong.

I’d read it the other way. A healthy override rate tells you people are still thinking. If the rate falls to nothing, it’s worth asking whether the tool has become perfect or whether people have stopped challenging it. I know which one I’d bet on.

So report the override rate as a sign of health, not a defect. And be worried when it goes quiet.

Design the job so checking is real

Five things help keep a person properly involved.

Make the tool show its working. For every answer, show where it came from, and mark clearly anything it couldn’t confirm. A reviewer can argue with a sourced page. They can’t argue with a confident summary.

Be clear about what the person is there to do. Spell out when they can accept a recommendation, when they must check it, and when they must pass it to someone else. Give them a simple way to reject or override it, and make sure doing so doesn’t create extra work for them.

Set rules for evidence. One rule I like: the tool can stop you acting on a single source, but it can never start you acting on one. Anything that leads to a letter, a visit or a legal claim needs two independent sources.

Check the checkers. Every so often, ask reviewers to record their own view before they see the recommendation. Occasionally include a case you know is wrong and see whether it gets caught. Neither is about catching people out. Both tell you whether the oversight you’re relying on is actually there.

And change what the role is called and how it’s judged. A “checker” rubber-stamps. A “reviewer” is expected to form a view. Don’t attach a productivity target to that role in the early stages, or you’ll be rewarding speed over judgement.

Who answers when it's wrong?

The last question is the one that matters most. When the tool gets something wrong and a customer is affected, who is accountable? If the answer is “the person who clicked approve”, but that person had forty cases an hour and no time to read them, the accountability is only on paper.

Design the role so the person named can actually do the job. That’s what a human in the loop is supposed to mean.

Relying on human review and not sure it's working?

Let’s look at what your reviewers actually see, how much time they get, and whether the oversight you’re paying for is really there.

Share this insight

Related insights
A faint grid of binary zeros and ones on cream, with a single blue magnifying glass to the right.
AI, Digital

Scotland’s biggest law firms have cracked SEO. Almost none are ready for AI search.

Scotland’s biggest law firms have cracked SEO, but almost none are ready for AI search. What we found across the ten largest firms, and the five fixes that move a firm from invisible to quotable.
2 Oct 2026
Three nested rounded frames on a cream ground, scattered with dark dots and rings, a few of them picked out in blue.
Marketing, Risk and governance

The riskiest AI in your marketing team is the tool nobody approved

How marketing teams can use AI without breaking data-protection rules or brand trust. Right-size the oversight, name an owner, and start with the tools already in use.
29 Sep 2026
Many tangled dark lines, two of them blue, drawn together from the left into one straight line that ends in a single dot.
AI, Marketing

Before AI optimises your campaigns, fix what you’re measuring

AI can optimise campaigns and share credit across channels, but only if the data underneath is joined up and honest. How to get attribution AI can actually use.
28 Sep 2026

Let's grab a coffee.

If something's stuck, or you'd like a second opinion on a plan, tell me about it. No hard sell.