Insights

What would convince your biggest sceptic?

Most AI pilots end in a demo and a shrug. Design the measures before you build, and decide in advance what would make you stop.
26 May 2026
AI, Projects
A see-saw on a dark pivot, weighed down on the left by a solid blue block, with an empty outlined box raised on the right.

Decide what would convince your biggest sceptic before you build. Measure the current process, however messy the data. Then watch use, overrides and errors month by month, report against measures the business already uses, and agree up front what would make you stop.

Most AI pilots don’t fail loudly. They drift. Nobody switches them off, nobody scales them up, and a year later they’re still “in pilot”, quietly costing money.

The numbers aren’t encouraging. S&P Global Market Intelligence found that 42% of companies had abandoned most of their AI initiatives in 2025, up from 17% the year before. The average organisation scrapped 46% of its proofs of concept before they reached production. An MIT study published the same summer reported that around 95% of the generative AI pilots it looked at had no measurable effect on profit and loss. People have argued about that study’s method, and fairly so. But few people who’ve worked in a large organisation were surprised by the headline.

In my experience the technology is rarely the reason. The reason is that nobody decided, at the start, what success would look like.

Start with the sceptic

The best question I’ve come across for this comes from Nadine Soyez: what would convince a sceptic that this worked?

It’s a better question than “what are our KPIs?” because it forces you to think about evidence rather than activity. A sceptic doesn’t care how many people logged in. They care whether the thing you said was broken is now less broken.

If you can’t answer it, you’re not ready to build. Go back and get clearer about the problem.

Measure before, even if it's messy

You can’t show improvement without a starting point. Measure the current process before anything changes: how long it takes, how often it goes wrong, what it costs. The data will be imperfect. Measure it anyway. A rough baseline is far more useful than a perfect one you never collected.

Watch the early signals, not only the results

Business results take time to show up. Cost, cash, customer satisfaction. You need them, but they arrive late.

In the meantime, watch the signals that tell you which way things are heading. Are people using it? How often do they change or override what it suggests? Are errors and exceptions going down? Is the work getting quicker? The direction matters more than the exact number.

Boil that down to three questions to ask every month. Is it being used? Is it improving? Is it changing how people work? From those you make one decision: continue, pause or stop.

Stopping isn’t failure. If adoption has flatlined, the problem has changed, or the tool only works in carefully controlled conditions, a clean stop is cheaper than another six months of drift.

Report what the business cares about

Automation rate is a tempting number. It goes up, it looks good on a slide, and it tells you almost nothing about value. Report against the measures the business already uses, such as cash collected, cycle time or complaints, and don’t claim a saving the team doing the work hasn’t agreed.

Then add a counter-measure. If a tool makes a task cheaper, it also makes it easier to do more of it, including the wrong version. A tool that speeds up debt chasing should be judged alongside how often the wrong customer gets chased and how many complaints are upheld. Otherwise you can hit your target and make things worse.

Finally, agree a stop rule up front. Pick three measures, give each one an owner, and agree that if none of them has moved in a year, you’ll stop. Writing that down before you start is uncomfortable. It’s also the thing most likely to keep the project honest.

What would convince your sceptic?

If you’re not sure how you’d prove a pilot worked, get in touch. We’ll design the measures before anything gets built.

Share this insight

Related insights
A faint grid of binary zeros and ones on cream, with a single blue magnifying glass to the right.
AI, Digital

Scotland’s biggest law firms have cracked SEO. Almost none are ready for AI search.

Scotland’s biggest law firms have cracked SEO, but almost none are ready for AI search. What we found across the ten largest firms, and the five fixes that move a firm from invisible to quotable.
2 Oct 2026
Many tangled dark lines, two of them blue, drawn together from the left into one straight line that ends in a single dot.
AI, Marketing

Before AI optimises your campaigns, fix what you’re measuring

AI can optimise campaigns and share credit across channels, but only if the data underneath is joined up and honest. How to get attribution AI can actually use.
28 Sep 2026
A neat grid of identical dark dots on cream, with one dot in blue, ringed to stand out from the rest.
AI, Marketing

Personalisation that helps, not personalisation that pesters

A practical guide to personalised lead generation and engagement with AI: the data foundation, lead scoring, guardrails, and the line between helpful and creepy.
27 Sep 2026

Let's grab a coffee.

If something's stuck, or you'd like a second opinion on a plan, tell me about it. No hard sell.