This is the third of four articles on what AI is really doing to finance and operations. Each one takes a single question, answers it with evidence, and can be read on its own.
In October 2026 the security firm Barracuda published an email its researchers had caught in a real attack. It looked like an ordinary message about an invoice.
Inside it was a block of text that no person would ever see. It was not written for a person. It was written for the AI assistant that many companies now use to read and summarise their email. The hidden text told the AI to add an urgent action to its summary: change this vendor’s payment details.
Think about what the employee sees. Not a suspicious email. A neat summary, written by their own trusted assistant, saying that a vendor’s bank details need updating today. The request did not seem to come from a stranger. It seemed to come from inside.
That is the subject of this article. It is a new kind of weakness, it is specific to AI, and it lands exactly where finance teams are most exposed.
A person reading an invoice knows the difference between their manager’s instruction and the words printed on the invoice. One is an order. The other is just information.
An AI model does not have that separation. Your instruction, the company’s rules and the contents of the document all arrive the same way, as one long stream of text. If a sentence inside the document reads like an instruction, the AI may follow it.
Imagine a new clerk who obeys every note he finds, including a note tucked inside a supplier’s envelope that says “pay this to a different account”. No sensible person works that way. An AI assistant, by default, can.
The technical name is prompt injection. Two things make it serious.
Three well-known business platforms have been tested by security researchers in the past eighteen months. In each case the researchers got in through an ordinary business form or record.
Be clear about what these are. All three were demonstrations by researchers, reported to the vendors. None is known to have been used by criminals. But they prove the method works on real, widely used products, and the door in each case was a text box open to the public.
Laboratory tests point the same way. In one widely used academic test that includes banking tasks, a simple attack of this kind succeeded about half the time against one of the leading AI models of the time.
Part 2 of this series described where AI agents are going: inside your ERP, working with your permissions, preparing entries and payments. Now put two facts side by side.
The agent can act. It can draft a payment, update a supplier record, match an invoice.
The agent reads outsiders’ documents all day. Supplier invoices. Customer purchase orders. Bank advices. Emails. Onboarding forms. That is its job.
No other part of a business takes in so much text written by outside parties and sits so close to the money. And the attack that pays best is a quiet one. Not stealing data. Changing a supplier’s bank account. Adding a new vendor. Approving one more invoice.
Finance teams already know this fraud. The email that says “our bank details have changed” is one of the oldest tricks used against them. Hidden instructions are the same trick, aimed at the AI instead of the clerk. And in the system’s records, the change looks like normal work done by a trusted assistant.
The attacker needs no password and no access. Their only cost is the cost of sending you an invoice.
Concerned, and calm. The evidence supports both.
The lesson from that last case matters. Before worrying about exotic attacks, make sure every AI tool connected to your systems has limited access and that its keys can be cancelled quickly.
In every case above, the AI was able to do far more than its task required. An agent that reads invoices needs to read invoices. If it cannot change a bank account, no hidden instruction can make it.
The safest design lets the AI that reads outside documents do only one thing: fill in a draft. It cannot send, pay or change anything. Fixed rules in your ERP, and a person where needed, decide what happens to that draft. Researchers who tested a strict version of this design cut successful attacks in one test from 300 to none.
Changing a supplier’s bank details. Creating a new supplier. Releasing a payment. For these three, keep the old rule: confirm by calling a number you already hold, not one given in the message. This control is decades old. It works against hidden instructions for the same reason it works against forged emails: it happens outside the channel the attacker controls.
Stolen data and false payments both have to go somewhere. Limit the addresses, accounts and systems the agent is able to reach. When Salesforce fixed its flaw, this is what it did.
Hidden text is hidden from people, not from software. Tools that inspect the file itself, and not only the words a person would see, catch far more. In one 2026 study, a detector that examined the structure of PDF files caught nearly all hidden instructions. A detector that read only the extracted text missed three in four.
One advantage in our markets
Saudi Arabia, the UAE and India are moving invoices onto structured electronic formats checked by the tax authority. A structured e-invoice has fixed fields for amounts, dates and tax numbers, which leaves less room for hidden text than a PDF or a photo sent on WhatsApp. This is our reasoning, not a tested result, and free-text fields and attachments remain open. But the more of your invoices arrive as structured data, the smaller this problem becomes.
Notice how much of this article rests on tests. Researchers tested three products and found holes. A laboratory test showed the attack working half the time. A study showed one filter missing three hidden instructions in four.
That is how you learn what a technology really does: someone independent tests it and publishes the result, good or bad.
Now ask the same of the promises. When a vendor says its AI closes the books 50% faster, or processes 85% of invoices with no human, who tested that?
Next in the series · Part 4
We looked for independent proof that AI agents deliver what is claimed inside ERP systems. We did not find any. The final part sets out what the evidence does show, the two numbers that reveal whether an AI tool is working, and the simple test every AI should have to beat before you pay for it.