How to automate invoice and document data entry, properly

Retyping information from documents is the single most common thing businesses ask us to automate. Invoices, purchase orders, delivery notes, application forms, remittance advice. The shape is always the same: a document arrives, a person reads it, and a person types what it says into something else.

It is a good candidate. The steps are consistent, the volume is usually meaningful, and the result is checkable. But these projects go wrong in predictable ways, almost always because the extraction was treated as the whole project when it is closer to a third of it.

Here is how one is actually built.

Step 0: Work out what you are really solving

Before anything technical, be precise about the goal, because it changes the design.

Speed? People are waiting on data, and the delay costs something downstream.

Accuracy? Typing errors are reaching your records and being discovered late.

Capacity? Volume is growing and you would rather not add headcount.

Cost? The hours are simply expensive.

These pull in different directions. Optimising for accuracy means more review and less speed. Optimising for speed means accepting more automated decisions. Pick the primary one; the others become constraints rather than goals.

Also count the real hours honestly, including the parts nobody mentions: finding the document, chasing the supplier when something is missing, fixing errors found later. The correction work is frequently larger than the typing.

Step 1: Understand what actually arrives

Collect the last month of documents. All of them, not a curated sample.

Then count:

• How many distinct layouts? Not suppliers — layouts. Two suppliers using the same accounting package produce the same layout.
• How many are digital PDFs versus scans or photos? A digital PDF has selectable text and is far easier. A scan is an image and needs OCR first, with accuracy that depends on the scan.
• How do they arrive? Email attachment, portal download, shared folder, post. This determines the intake mechanism, which is its own piece of work.
• What proportion are awkward? Multi-page, multiple invoices in one file, credit notes, foreign currency, handwritten annotations.

This exercise is unglamorous and it is the most valuable hour in the project. People consistently guess low on layout count. Discovering the true number at the start changes the design; discovering it during the build changes the budget.

Step 2: Decide exactly what you need out

List the fields, and for each one write down three things: whether it is required, what a valid value looks like, and where it goes.

For a typical invoice:

That validation column is doing the heavy lifting later. Write it before you build.

Be ruthless about "required". Every field you insist on is a reason a document can get stuck. If you do not use it downstream, do not extract it.

Step 3: Extraction

Now the part people think of as the project.

For a digital PDF, text and its position on the page are directly available. For a scan, OCR runs first to produce text, and everything downstream inherits whatever errors it made.

From there, two broad approaches, and most real systems use both:

Template matching. For a known layout, you know where things are. Fast, cheap, highly accurate, and completely dependent on the layout not changing. Worth doing for your top few suppliers if they are a large share of volume.

Model-based extraction. A model reads the document and returns structured fields. Handles unfamiliar layouts, which is its entire value, and this is where "AI" enters the project.

The sensible design is a hybrid: templates for the high-volume known formats, the model for everything else. Cheaper to run and more predictable where it matters most.

One thing to insist on regardless: the system should record where on the page each value came from. When a reviewer sees a flagged total, being able to jump straight to the relevant part of the document turns a two-minute check into a five-second one. This single feature does more for adoption than any accuracy improvement.

Step 4: Validation, which is where the value is

The extraction gives you values. Validation tells you whether to believe them, and it is the difference between a system people trust and one they double-check.

The strongest checks do not involve AI at all:

Arithmetic. Do the line items sum to the subtotal? Does subtotal plus tax equal the total? Invoices are self-checking documents, and this catches a large share of extraction errors for essentially no effort. If the maths reconciles, the numbers are almost certainly right.

Cross-reference. Does the supplier exist in your records? Does the purchase order number match a real order? Are the quantities within the tolerance of what was ordered?

Duplicate detection. Have you already recorded this invoice number from this supplier? This one matters. Duplicate payments are a real and expensive failure mode, and automation makes them more likely, not less, because retries and re-sends happen.

Plausibility. Is this ten times the usual amount from this supplier? Is the date two years old? These will not always be wrong, but they should be looked at.

Model confidence. Useful as one input among several, not as the primary check. A model can be confidently wrong, and arithmetic cannot.

Step 5: The review queue

Some documents will not pass. The question is what happens to them, and the answer determines whether this project succeeds.

A good review queue:

• Shows the extracted values next to the original document, with the source location highlighted.
• Says why it was flagged, specifically. "Total does not match line items: extracted 1,240.00, computed 1,420.00" is actionable. "Low confidence" is not.
• Allows correction in place, then resumes the workflow.
• Learns from corrections, at minimum by flagging recurring problems with the same supplier so the underlying cause gets fixed.
• Has a named owner and a target turnaround, so the queue does not silently grow.

Be realistic about the target. A well-built system on reasonable documents should send a minority of items to review, and that minority is where your remaining human time goes. The saving comes from not touching the majority, not from touching none of them.

Step 6: Getting it into the destination system

The last step is the one that gets underestimated, and it is a genuine integration project of its own.

Questions to settle:

• Does the destination have an API? If it is import-file-only, you inherit batching, ordering and error-reporting problems.
• What does it do with a duplicate? Reject, overwrite, or create a second record? You must know before you connect.
• Can you match to existing records reliably? Supplier names never match exactly. "Acme Ltd", "ACME Limited" and "Acme" are one supplier to you and three strings to a computer. Matching rules need writing down.
• What is the status flow? Does it arrive as a draft for approval, or posted?
• What happens if the write fails halfway? Half-created records are worse than none.

That last point is why the operation should be safe to repeat: identify each document by supplier plus invoice number, check before creating, and a retry becomes harmless.

What a realistic first phase looks like

A sensible scope:

1. One intake channel. Email attachments to a dedicated address, say.
2. One document type. Supplier invoices only — not credit notes, not statements.
3. Your top suppliers by volume, which is usually a small number covering a large share of documents.
4. Full validation on those, because validation is what makes it trustworthy.
5. A review queue for everything else, including unrecognised suppliers.
6. Write into one system.

That handles most of your volume, and everything outside the scope goes to the queue exactly as it does today. Nothing gets worse. Then you extend based on what the queue actually contains, which is far better information than anything you could have guessed at the start.

What to measure once it is live

• Proportion passing without review. Should rise as edge cases get handled.
• Errors that got through. The number that matters. Sample the automatically-processed ones periodically; do not assume.
• Review queue time, and whether it is growing.
• Hours actually saved, against your baseline from step 0.
• Cost per document, including model usage, so you know how the economics scale with volume.

Measure the second one deliberately. It is the one nobody sets up, and it is the one that tells you whether to trust the system.

The honest caveats

Accuracy depends on your documents, not on the vendor. Be sceptical of headline percentages measured on someone else's clean data. Test on fifty of your own.

Scans are harder, and you do not control the quality. If they are a large share of your volume, test that specifically.

Model usage costs scale with volume. Get the per-document figure.

Someone must own the queue. Without an owner it grows, then gets ignored, then the system is abandoned.

It will not be perfect, and it does not need to be. It needs to be better than the current error rate, with mistakes that surface quickly. Manual data entry is not error-free either — it is just that its errors are familiar.

We built a FastAPI backend that moves business data between systems with cloud file storage behind it; the extraction was the interesting part and the validation and matching rules were the part that made it usable. If you have a pile of documents someone retypes every week, that is worth half an hour of conversation — book a free consultation and bring five real examples, including a messy one.