Close cycles cut in half. 85% of invoices with no human touch. We looked for independent proof behind figures like these inside ERP systems and found none. Here is what the evidence does show, and how to test an AI before you pay for it.
Draft for review | 18 November 2026 | 10 minute read
This is the last of four articles on what AI is really doing to finance and operations. Each one takes a single question, answers it with evidence, and can be read on its own.
In February 2024 the payments company Klarna announced that its new AI assistant was doing the work of 700 full-time customer service staff. The number travelled around the world. It appeared in board papers and conference slides as proof of what AI could do.
In May 2025 Klarna’s chief executive told Bloomberg that the company had gone too far. Cost, he said, had been “a too predominant evaluation factor”, and the result was “lower quality”. Klarna started hiring people for customer service again.
The 700 figure may well have been an honest count of the work the AI handled. What it did not show was the quality of that work, which is the part the company later said had slipped. Nobody outside the company had checked either one.
That is the problem with most AI numbers in business today. They are not necessarily wrong. They are unchecked. And the gap between those two things is where expensive decisions go wrong.
For this series we searched for a specific kind of evidence: a published, independent measurement of AI agents working inside an ERP system, with a clear method, a baseline and a sample size. In any module. In any country.
We did not find one.
What we found instead were vendor case studies. The ones we reviewed for this series report close cycles cut by 40% to 60%, more than 85% of transactions matched automatically, invoices processed 75% faster, and three to six times the money back in the first year. Almost none say how these were measured, what the starting point was, or how many companies were involved.
Others have noticed the same thing.
None of this proves that the ERP figures are wrong. Many may be true for the customer they describe. But a figure can be true for one company and still tell you very little about yours.
Unaudited does not mean false. It means nobody outside the sale has checked.
Take a typical claim: “Our AI processes 85% of invoices with no human involvement.” Every word in that sentence hides a choice. Before the number means anything, you need five answers.
Two of these deserve a closer look.
Compared to what is the question most often skipped. Many finance teams already match a large share of invoices automatically, using simple rules that check the purchase order, the delivery and the price. If an AI tool reaches 85% where your rules already reach 80%, it has earned five points, not eighty-five. A number with no starting point cannot tell you what the AI added.
Counted how matters because “processed” is not the same as “processed correctly”. Part 1 of this series showed how a tool that reads 98% of fields correctly can still get one whole document in nine wrong. Then there is “hours saved”, usually worked out by multiplying the number of documents by an assumed number of minutes each. That is an estimate wearing the clothes of a result.
It would be easy to stop here and conclude that AI does not work. Plenty of headlines say so.
The most quoted is from a 2025 MIT report: 95% of companies saw no measurable return from generative AI. It is cited as often as any vendor figure, and with as little checking. The report was based on 52 interviews, 153 survey responses collected at industry conferences, and a review of about 300 public announcements. The outcomes were reported by the companies themselves, and what counted as success varied from one to the next.
That does not make the finding wrong. It makes it the same kind of number as the vendor claims: interesting, and unaudited. If you would not accept “85% with no human” without asking five questions, do not accept “95% failed” without asking them either.
Not every source is equal. It helps to think of evidence as a ladder. The higher the rung, the harder the number is to bend.
The middle rungs are not empty, and what they show is consistent. Here are the most useful studies we found, with what each one can and cannot tell you.
Put these together with what Part 1 reported from regulated UK firms, where 80% had AI agents in use but only 2% let them act broadly on their own, and an honest picture appears.
If outside evidence is thin, measurement inside your own business is not a nice extra. It is the only proof you will have. The good news is that it does not take a dashboard of fifty metrics. Two numbers, read together, tell you most of what you need.
1. Correction rate. Out of everything the AI prepares, how much do your people change before they approve it? An entry recoded, an amount fixed, a match broken.
2. Reversal rate. Out of everything that was approved, how much was later undone? A reversed journal, a credit note, a reopened match, an adjustment from your auditors.
Each number alone can fool you. A falling correction rate looks like success. It may also mean your reviewers have stopped looking, the risk Part 1 described when 300 entries land before lunch. The reversal rate is what catches that, because errors that slip through approval tend to come back later, at a higher cost.
Three practical points make these numbers useful.
Notice what is not on the list: hours saved, documents touched, number of agents running. Those measure activity. They tell you the AI is busy, not that it is right.
All of this leads to one simple rule. Before you pay for an AI tool, make it beat what you already have, on your own work.
You already hold the perfect test. Last quarter’s documents have been processed, reviewed and closed. You know the right answer for every one of them. That makes them an answer key.
Here is how to run it.
A vendor confident in its product should welcome this test. If one will not run it, that tells you something too.
Four articles, one thread.
AI has made producing work cheap, but checking it costs what it always did. The ERP is not going away, because it is the referee that catches what AI gets wrong. The AI can be steered by what it reads, so its permissions matter more than its instructions. And the numbers used to sell it have, so far, not been checked by anyone outside the sale.
None of this is a reason to wait. AI will change the back office, and in parts it already has. But the companies that gain from it will not be the ones that believed the biggest number. They will be the ones that tested the tool on their own work, kept a person who knows the job in charge, and measured what happened afterwards.
In finance, nobody accepts a number because the person who produced it says it is right. That is what audits are for. AI deserves the same treatment.
AI and the Back Office · The full series