Insights

Before you buy the AI tool, ask who it gets wrong

The price, the platform and the programming language matter far less than the output. Six questions to ask before anyone signs.
2 Sep 2026
AI, Risk and governance
Three rows of outlined figures on cream, with three of them filled in solid blue.

Price and platform matter far less than output. Before anyone signs, ask what the tool gets wrong and who it gets wrong for, whether it has been tested independently on data like yours, what you need in place to make it work, and who in your own organisation answers when it fails.

Early in my LSE programme we were asked what we’d most want to know about an AI tool that screens job applications. The options included cost, who else uses it and how it was built.

I picked a different one: who are the 5% it rejects, and what happens if they were your best candidates?

I’m not that bothered by the price or the programming language. I’m bothered by the outputs. That’s where the value is, and it’s where the damage is too.

A cautionary example

Epic’s sepsis prediction tool was used widely in American hospitals. The developer reported strong performance. In a study published in 2021, researchers at the University of Michigan tested it independently on more than 38,000 hospital stays. It failed to identify 67% of the patients who developed sepsis. It also raised alerts for 18% of all patients, which is a lot of noise for staff to wade through.

The tool worked. It just didn’t work nearly as well as people believed, and nobody checked until it was already in use.

Most business tools aren’t life and death. The pattern is the same, though. Claims made in a sales process are tested on the vendor’s data, in the vendor’s conditions, measured the way the vendor chose.

Be wary of big claims

I recently read a press release for an AI customer service agent that promised to handle conversations “without human intervention”. The company’s own materials explained that disputes and debt queries were passed to people. That’s sensible design. It’s also not what the headline said.

Gartner has estimated that only around 130 of the thousands of vendors selling “agentic” AI are the real thing, and predicted that more than 40% of agentic AI projects will be cancelled by the end of 2027. Rebadging ordinary chatbots and automation as agents is common enough that Gartner has a name for it: agent washing.

None of this means you shouldn’t buy. It means you should ask better questions.

Six questions to ask before anyone signs

01What specific problem does it solve, and how will we measure it? If the answer is general, like "productivity", ask what number will change and by how much.

02Has it been tested independently, on data like ours? A benchmark is only useful if it looks like your world. Ask what the test was, how recent it is, and whether the system could have seen the answers during training.

03What does it get wrong, and for whom? Every tool makes mistakes. You need to know whether those mistakes fall evenly or land on particular groups of customers or colleagues.

04What happens when it's wrong? Can a customer challenge the result? Can staff override it easily? Is there a record of what it did?

05What do we need in place for it to work? Data quality, system access, training, process changes. Vendors tend to assume these are sorted. They rarely are.

06Who answers when it goes wrong? Not the vendor's support desk. The named person in your organisation who owns the result.

Slow down at the point of purchase

Sales conversations are designed to create urgency. A limited offer, a competitor who’s already signed, a demo that looks effortless. That’s exactly the moment to slow down.

I’ve been sold large platforms on promises they didn’t keep, and the lesson stuck. A tool that falls short of expectations hasn’t always failed. But expectations that were never tested are a risk you chose to take.

Ask who it gets wrong. If nobody can tell you, that’s your answer.

Weighing up an AI purchase?

Get in touch before you sign. A second opinion on what the tool gets wrong is worth more than another demo.

Share this insight

Related insights
A faint grid of binary zeros and ones on cream, with a single blue magnifying glass to the right.
AI, Digital

Scotland’s biggest law firms have cracked SEO. Almost none are ready for AI search.

Scotland’s biggest law firms have cracked SEO, but almost none are ready for AI search. What we found across the ten largest firms, and the five fixes that move a firm from invisible to quotable.
2 Oct 2026
Three nested rounded frames on a cream ground, scattered with dark dots and rings, a few of them picked out in blue.
Marketing, Risk and governance

The riskiest AI in your marketing team is the tool nobody approved

How marketing teams can use AI without breaking data-protection rules or brand trust. Right-size the oversight, name an owner, and start with the tools already in use.
29 Sep 2026
Many tangled dark lines, two of them blue, drawn together from the left into one straight line that ends in a single dot.
AI, Marketing

Before AI optimises your campaigns, fix what you’re measuring

AI can optimise campaigns and share credit across channels, but only if the data underneath is joined up and honest. How to get attribution AI can actually use.
28 Sep 2026

Let's grab a coffee.

If something's stuck, or you'd like a second opinion on a plan, tell me about it. No hard sell.