Document AI: OCR, extraction and validation

Businesses run on documents: invoices, forms, contracts, receipts, statements. Turning them into reliable, structured data is one of the most useful things machine learning does, and one of the easiest to get subtly wrong. A well-built document AI pipeline treats extraction as a chain of steps, each checked, rather than one clever model.

The pipeline at a glance

1. Ingest and normalise. Accept PDFs, scans and photos; convert to consistent images; fix rotation and resolution.
2. Layout analysis. Find the structure: text blocks, tables, headers, stamps, signatures.
3. Text recognition (OCR). Read the characters, unless the PDF already contains real text.
4. Extraction. Turn text and layout into fields: invoice number, date, total, line items.
5. Validation. Check the fields against rules and reference data.
6. Review and routing. Send uncertain results to a person.
7. Feedback. Use corrections to improve the system and the tests.

Step 1: get the input right

Quality at the start decides quality at the end.

• Prefer native PDFs with embedded text over scans, since you can skip OCR and avoid its errors.
• For scans, deskew, denoise and use adequate resolution. Photos of documents need perspective correction.
• Detect the language and script early.
• Keep the original file and a page-level record so every extracted value can be traced back to where it came from.

Step 2 and 3: layout and OCR

OCR returns text with positions, not just words. Keep the coordinates, because meaning often depends on layout: a number under a "Total" heading is not the same as a number in the item list. Tables are the classic hard case: recognise the grid, then read cell by cell, or preserve enough positional information to rebuild it.

Expect OCR errors that look plausible: "O" and "0", "1" and "l", missing decimal points, merged columns. These matter most in numbers and identifiers.

Step 4: extraction

Several approaches exist, and they combine well:

• Templates and rules for stable, high-volume forms: fast, predictable, brittle when layouts change.
• Trained extraction models that use text plus layout to label fields, robust across formats but needing labelled examples.
• Language models prompted (or fine-tuned) to return the fields as structured output, flexible for varied documents. They need the same care as any generated output: schema validation, and never trusting a value that is not in the document.

Whatever you use, return, for each field, the value, its source location and a confidence.

Step 5: validation is where reliability comes from

Extraction gets you close; validation catches the rest.

• Format checks: dates parse, amounts are numbers, identifiers match their pattern.
• Arithmetic checks: line items sum to the subtotal; subtotal plus tax equals the total.
• Cross-checks: the supplier exists in your master data; the purchase order number is real and open.
• Grounding check: the extracted value actually appears in the document text.

def validate_invoice(inv):
    errors = []
    lines_total = round(sum(l["amount"] for l in inv["lines"]), 2)
    if abs(lines_total - inv["subtotal"]) > 0.01:
        errors.append("line items do not add up to the subtotal")
    if abs(inv["subtotal"] + inv["tax"] - inv["total"]) > 0.01:
        errors.append("subtotal plus tax does not equal total")
    if inv["supplier_id"] not in KNOWN_SUPPLIERS:
        errors.append("unknown supplier")
    return errors

Step 6: human in the loop

Set thresholds: high-confidence, fully validated documents flow straight through; anything failing a check or below a confidence cut-off goes to a review queue with the source region highlighted next to the extracted value. Good review tools make correction fast, which is where the productivity comes from.

Measuring the system

• Field-level accuracy per field, not just per document, since one bad total matters more than a typo in a note.
• Straight-through rate: the share of documents needing no human touch, at an accuracy you can defend.
• Error types: OCR mistakes, wrong field, missing field, hallucinated value.
• Performance by document type and quality, since averages hide failures on poor scans.

Build a test set of real documents (with permission and privacy safeguards), with corrected values, and re-run it after every change.

Privacy and safety

Documents often hold personal and financial data. Limit who can see them, encrypt storage, log access, retain only what you need, and remove or mask sensitive fields where the task allows. Treat text inside documents as untrusted input if a language model reads it, since instructions can be hidden in a document.

Common mistakes

1. Skipping validation and trusting confident-looking output.
2. Discarding coordinates after OCR.
3. Measuring accuracy per document only.
4. No review path for uncertain results.
5. Testing only on clean scans.

A checklist

• Do we use native text where it exists?
• Can every value be traced back to its place in the source?
• Do arithmetic and reference-data checks run on every document?
• Is there a review queue with confidence thresholds?
• Is there a labelled test set covering poor-quality inputs?