Can AI type your invoices into a spreadsheet? We tested it on 24 made-up invoices

The short answer: our test supports AI extraction with a person checking flagged invoices and totals. An AI model copied 18 of 20 test invoices into spreadsheet fields perfectly in all 3 of its runs, including fake scans, invoices with simulated handwriting, multi-page invoices and euro and Canadian-dollar invoices. It got the other 2 wrong the same way every time. The one that matters most: it copied an invoice whose line items did not add up to its subtotal, and never warned us. Our simple rules script caught that one. So if you use AI for this, add a sum check to your spreadsheet and have a person look at anything that fails it or gets flagged.

All 24 invoices, the vendors and the customer in this test were invented. Nothing here is a real business's data.

What we tested

We made 24 fake invoices from 6 invented suppliers (a paper supplier, a coffee roaster, a plumbing parts shop, a bakery packaging firm in Europe, a Canadian print shop and an art supply store). They included:

• 6 clean, computer-made PDFs
• 4 simulated scans (slightly tilted, grainy, two with a coffee ring)
• 3 with simulated handwriting: a "paid 10/12 chk 4411" note, an "Approved for payment" note, and a quantity of 12 crossed out with "10" written next to it
• 3 multi-page invoices with 28 to 34 line items
• 2 credit memos, one with amounts in parentheses
• 2 foreign invoices (euros with comma decimals, and Canadian GST plus PST)
• 2 duplicates of invoices already in a sample ledger, one with a reused invoice number
• 2 traps: one whose line items add to $260.70 while the printed subtotal says $248.50, and one dated 03/04/2026, which could be March 4 or April 3

We ran each invoice through Anthropic's Claude Sonnet 5.5 model 3 times, asking for 11 fields (vendor, invoice number, document type, invoice date, due date, currency, subtotal, tax, tax type, total, line items), plus the related invoice number on credit memos, and four warning flags: duplicate, totals don't match, ambiguous date, and needs a person.

For comparison, we wrote a plain rules script, the kind of thing a spreadsheet-savvy bookkeeper or a basic OCR tool does: read the text, find the fields by pattern, check the math, look up the ledger. We wrote it while looking at only 4 of the clean invoices, locked it, and only then made the other 20. But the same assistant also wrote the program that made the invoices, so the script was built knowing these layouts and labels (such as "TOTAL CREDIT", "Credit Memo" and GST/PST). That makes it a best case for rules. Those 20 are the fair test, so the numbers below are for those 20 unless we say otherwise. (Both methods got all 4 of those clean invoices right.)

What happened

Invoices fully correct, out of the 20 fair-test invoices. The AI's pass or fail never changed between its 3 runs, so each AI count holds for every run.

* One of the rules script's 3 scan failures was descriptions only: two typos in item names, with every number correct.

Counted by attempt, the AI passed 54 of its 60 tries on the 20 invoices.

The comparison we set before the test: on the 18 messy held-out invoices, the AI passed 48 of 54 attempts (89%), the rules script 14 of 18 (78%). Because the AI's pass or fail never changed between runs, its per-attempt rate is the same as its per-invoice rate, 16 of 18. The gap is 2 invoices out of 18: the AI failed 2 and the rules script failed 4.

Where the AI was strong. It got every simulated scan right in all 3 runs, copied every line-item row correctly in every run, and handled both foreign formats. On the crossed-out quantity, it did what we asked in all 3 runs: entered the printed 12, flagged it for a person, and wrote a note saying 12 had been crossed out and 10 handwritten. It caught both duplicates, including the reused invoice number with a different date and amount, and it left the 03/04/2026 date blank and flagged it rather than guessing (our instructions used that same date as their example; see Limits).

Where the AI failed.

• The total that didn't add up. On the trap invoice, the AI copied the printed subtotal, tax and total correctly, and correctly copied all 3 line items. It did not raise the "totals don't match" warning in any of its 3 runs, even though our instructions told it to. The line items were $12.20 more than the subtotal. If you were relying on its warning, you would have entered this invoice without a second look.
• A date it shouldn't have blanked. On a credit memo dated 10/15/2026, it left the date blank and called it ambiguous in all 3 runs. It isn't ambiguous: there is no 15th month. In two runs its own note worked out that no 15th month exists, then flagged the date anyway. This is a safe mistake (a person would fill it in), but it's a mistake.
• One false alarm. On one run of a 28-line invoice, it flagged a totals mismatch and wrote that the lines added up to $2,137.60. The lines it extracted add up to $2,137.65, exactly the printed subtotal.

Where the rules script failed. Mostly on the scans and handwriting, where it depends on OCR. On one scan, every number was right, but two item names had OCR typos ("Oat mitk" for "Oat milk" and "2 tb bag" for "2 lb bag") and nothing was flagged. We count that as a failure because item names are part of the answer, but it's a description-only miss. It missed the subtotal on another scan and the invoice number and date on a third (and flagged both for a person), and read the crossed-out 12 as "40" (also flagged). On clean, computer-made PDFs it was perfect.

The totals trap. The rules script caught the invoice whose line items didn't match its subtotal. The AI missed it in all 3 runs. Checking the math is exactly what a rules script is good at, and it's why we recommend a sum formula below.

Consistency. The AI gave the same pass or fail on every invoice in all 3 runs. Its two failures were repeated every time, so running an invoice twice would not catch them.

Cost. The 72 AI runs cost $1.11 at Anthropic's list price on 23 Oct 2026 ($2 per million input tokens, $10 per million output tokens): 368,382 input tokens ($0.74) plus 37,684 output tokens ($0.38), or about 1.5 cents per invoice per run. The rules script ran on our own machine at no API cost. We did not measure time saved.

What we'd tell an owner

1. AI extraction is worth trying if your invoices are messy. Scans and handwriting are where it pulled ahead in our test.
2. Don't trust its warnings alone. It missed the one invoice whose numbers didn't add up in all 3 runs, while our rules script caught it. Add two formulas to your sheet: line items minus subtotal, and subtotal plus tax minus total. Anything that isn't zero gets a human look. That check would have caught our trap, because the AI copied the line items correctly.
3. Have a person review anything it flags, and fill in what it leaves blank. In our test, a flag sometimes meant a real problem and sometimes didn't.
4. If your invoices are clean PDFs from the same few suppliers, a rules-based tool may be enough. Ours was perfect on those, but it was written for these exact layouts, so treat that as a best case.

Accounting tools already offer this. QuickBooks Online says it "automatically pulls dates, amounts, vendor, and last four digits of the credit card" from bills you email it (Intuit help), and Xero says that when you photograph a receipt, it "captures the amount, date, and other details automatically," and that you "double-check and approve the info before Xero creates a financial record" (Xero). Those are the vendors' claims. We didn't test either product; we tested a general AI model through its API.

Limits

This was a small, synthetic test: 24 invented invoices, one AI model, one set of instructions, one day. The counts aren't rates you can expect on your own invoices. Our scans were mild and our handwriting was a font, not a person's pen; real phone photos and faxes can be much worse. We wrote the invoices, the answer key, the AI instructions and the rules script ourselves. The rules script was written knowing these exact layouts and labels, so it's an optimistic baseline for rules; a rules tool meeting new suppliers' layouts would likely do worse. The AI instructions used 03/04/2026 as their example of an ambiguous date, which is the exact date on our date trap, so that trap was likely easier for the AI than an unseen date would be. Each tricky situation appeared only once or twice, so one miss is a big share of a small group. One handwritten note on our test images ran off the edge of the page; the AI said so, and it didn't affect any field. Results could differ on another model, another prompt, or a rerun.

How we ran this test

We generated the invoices with a script, rendered the scans and handwriting as images, and wrote down the correct answer for every field before running anything, including what counted as correct on each trap. We fingerprinted every file (inputs, answer key, AI instructions, rules script and scoring scripts) before the first invoice went to the AI, and logged every later change. The AI received each PDF or image directly and answered in a fixed JSON format, with its optional "thinking" turned to the lowest setting this model allows and its effort setting at "high". A script did all the scoring except for the handwritten notes, which were judged by the site's AI writing assistant against a rubric set in advance, not blind (all 9 AI runs scored full marks). The full evidence, raw AI responses and deviations log are kept with the test.

• Evidence. Download the evidence files (zip): the invoices, answer key, AI instructions, rules script, raw API responses, scoring scripts, and SHA-256 fingerprints. Rerunning the scoring scripts reproduces the reported results. The run script makes new paid API calls and needs your own key.

Test designed and scored with the site's AI writing assistant. Reviewed and approved by Ross C, Oct 24, 2026.

Related reading

• AI vs. Simple Rules for Appointment and Payment Reminders: We Tested Both on 30 Invented Customers: rules vs. AI on 30 invented reminder records.
• How to Evaluate an AI Pilot: Keep, Fix, or Stop: a pilot succeeds when quality, review time, exceptions, safeguards, and cost are included.
• 5 Tasks Your Small Business Should Automate Before Anything Else: where to automate before you buy another tool.
• What Data Should You Put Into an AI Tool?: what to share with AI tools and what to keep out.

Sources

All web sources were checked on October 23, 2026. Vendor sources are the vendors’ own claims, and we did not test them.

Vendor claims (not tested)

1. QuickBooks Online, “Email receipts and bills”. Intuit help
2. Xero, Smart Document Capture (UK). Xero

Model, API, and pricing

3. Anthropic pricing. Claude Sonnet 5.5 costs $2 per million input tokens and $10 per million output tokens. Checked October 23, 2026. Anthropic pricing
4. Thinking settings. Anthropic thinking
5. Structured outputs. Anthropic structured outputs
6. PDF support. Anthropic PDF support