Get a free Shopify store audit

Invoice Data Extraction With AI: OCR vs. LLM Methods

AI invoice data extraction uses OCR+layout models or LLM vision. Compare Azure, AWS, Google, OpenAI, and Anthropic, plus the only public accuracy benchmark.

Invoice Data Extraction With AI: OCR vs. LLM Methods — ASSOSIATIX Journal

Invoice data extraction with AI means pulling vendor names, totals, line items, and dates out of invoice PDFs and images automatically, using one of two approaches: prebuilt OCR-plus-layout models from Azure, AWS, and Google that return fixed fields with confidence scores, or multimodal LLM vision from OpenAI, Anthropic, and Google that reads the page image against a schema you define. Public benchmark data shows native image input beats converting the page to text first.

What "invoice data extraction with AI" actually covers

There are two genuinely different families of tooling behind this phrase, and conflating them is where a lot of vendor-blog comparisons go soft.

The first family is prebuilt OCR-plus-layout models: Azure AI Document Intelligence's `prebuilt-invoice` model, Amazon Textract's `AnalyzeExpense`, and Google Cloud Document AI's OCR and extraction processors. These were built specifically to read invoices and expense documents, and they return a fixed, standardized set of fields rather than whatever you ask for.

The second family is multimodal LLM vision: GPT models using OpenAI's Structured Outputs, Claude reading a PDF as a document content block, and Google Document AI's Custom Extractor running on a generative foundation model. These read the document image against whatever JSON schema you supply — in other words, extracting data from a PDF with AI rather than against a fixed template — which makes them more flexible for non-standard layouts but, as the one public benchmark on this shows below, with different accuracy and reliability trade-offs depending on the model and document quality. Choosing between these two families for a specific document set is exactly the kind of scoping our AI document processing service is built around. For a broader look at how OCR and LLM-based document processing compare outside invoices specifically, see our guide to AI document processing.

The fields every method is built to extract

Amazon Textract's `AnalyzeExpense` operation is a useful concrete example of what a prebuilt model returns, because it standardizes field names across vendors instead of returning whatever label happens to print on the page. Per Textract's documentation, header-level fields include:

  • `INVOICE_RECEIPT_ID`, `VENDOR_NAME`, `PO_NUMBER`, `DUE_DATE`
  • `TOTAL`, `AMOUNT_DUE`, `SUBTOTAL`, `TAX`, `DISCOUNT`, `SHIPPING_HANDLING_CHARGE`

Line items come back as a grouped `EXPENSE_ROW`, with per-item fields `ITEM`, `QUANTITY`, `PRICE`, `UNIT_PRICE`, and `PRODUCT_CODE`. Every field — header or line item — ships with a `Confidence` score and a bounding-box `Geometry`, so a downstream system can flag anything below a chosen confidence threshold for human review instead of trusting every extracted value equally. Fields that don't map to the standard taxonomy are returned under a generic `OTHER` type rather than being dropped.

This per-field confidence score is specific to the OCR-plus-layout family. It does not carry over to LLM vision extraction in the same form — a distinction worth keeping in mind when comparing the two approaches below.

Prebuilt OCR+layout models: what Azure, AWS and Google actually support

Each vendor's prebuilt model has its own documented ceilings, and they matter more than accuracy claims when you're deciding whether an option is even usable for a given document set.

Azure's `prebuilt-invoice` model (Document Intelligence v4.0, GA) supports invoices, utility bills, sales orders, and purchase orders in 27 languages. For PDFs and TIFFs, it processes up to 2,000 pages — but on the free (F0) tier, only the first two pages of any document are processed. File size is capped at 500MB on the paid (S0) tier and 4MB on the free tier. Image dimensions must fall between 50x50 and 10,000x10,000 pixels, and the minimum extractable text height is 12 pixels on a 1024x768-pixel image (roughly 8-point type at 150 DPI). Password-locked PDFs have to be unlocked before submission — Azure won't do that for you.

Amazon Textract's `AnalyzeExpense` runs either synchronously, for a single document you want results from immediately, or asynchronously via `StartExpenseAnalysis` followed by `GetExpenseAnalysis`, for larger documents or batch processing where you can poll for a result rather than hold a connection open.

Google Cloud's Document AI offers both an OCR processor and extraction processors in this family; the extraction side is covered under the LLM vision section below, since Google's current extraction processor runs on a generative foundation model rather than a traditional layout model. Document AI's processors (both the OCR processor and the Custom Extractor) share a page-count structure worth knowing before you architect around them: a maximum of 15 pages per online, synchronous request, up to 200 pages for batch (offline/asynchronous) requests, and up to 30 pages for online requests if you enable `imageless_mode`. That means a single-page synchronous call and a 150-page nightly batch job against the same processor behave very differently, and a workflow built around one won't automatically work for the other.

LLM vision extraction: how OpenAI, Anthropic and Google's generative extractor work

The second family reads the document as an image and returns whatever structure you ask for, with each vendor handling schema enforcement and document limits differently.

OpenAI's Structured Outputs constrains a model's reply to a JSON Schema you supply, either through function calling or through a `json_schema` response format set on `text.format`. Structured Outputs is available starting with GPT-4o, and schema-adherent `json_schema` output with `strict: true` is specifically supported on `gpt-4o-mini`, `gpt-4o-2024-08-06`, and later models — older snapshots can use the looser JSON mode instead, which produces valid JSON but doesn't guarantee it matches your schema.

Anthropic's Claude processes PDFs as a `document` content block, and because of that, PDF support is subject to the same limits as other vision tasks. All active Claude models support PDF input. The first-party API caps a single request at 128MB and 600 pages — or 100 pages if the model's context window is under 1M tokens. PDFs must be unencrypted and unpassworded. For large or frequently reused documents, Anthropic's Files API lets you upload once and reference the file by ID instead of resending it with every request.

Google's Document AI Custom Extractor can run on a generative foundation model — currently `pretrained-foundation-model-v1.5-pro`, built on Gemini 2.5 Pro and GA, which supports up to 30 pages per minute for online (synchronous) requests. The catch is scope: generative extraction on this processor officially supports only English, and is only available in the US, EU, `northamerica-northeast1`, and `asia-southeast1` regions. A non-English invoice set rules this specific path out immediately, regardless of accuracy.

None of these three vendors' documentation describes per-field confidence scoring for LLM vision extraction the way Textract and the OCR-layout family do — that's a meaningful operational difference if your downstream process depends on flagging low-confidence fields automatically.

The choice between function calling and a structured `text.format` response on the OpenAI side isn't just a style preference, either. OpenAI's own guidance is that function calling fits better when you're connecting the model to tools, functions, or data inside your own system — for example, a database lookup to confirm a vendor's PO number — while a structured `text.format` response fits better when the model's output is the end product you want to display or hand off, such as a JSON payload destined straight for an AP queue.

What the only public accuracy benchmark actually shows

Most of what gets published on OCR-versus-LLM invoice accuracy is vendor marketing. The one academic benchmark we found is from Fraunhofer IAIS and the Lamarr Institute (Berghaus et al., arXiv 2509.04469, August 2025), and it's worth citing specifically because it tested something concrete: eight multimodal LLMs — OpenAI's GPT-5 chat, mini, and nano; Google's Gemini 2.5 Pro, Flash, and Flash-Lite; and the open-source Gemma 3 12B and 4B — in zero-shot prompting on three open invoice and receipt datasets.

The paper compared two processing strategies: feeding the model the document image directly, versus first converting the page to markdown text with the open-source tool Docling and feeding the LLM that text instead. Native image processing beat the Docling-converted-text route on every dataset and every model tested. Gemini 2.5 Pro scored highest with native images across all three: 87.46% overall accuracy on scanned receipts (ICDAR-2019-SROIE), 96.50% on clean synthetic invoices (the Donut dataset), and 92.71% on scanned invoices with real-world artifacts like stamps and handwriting (the inv-cdip dataset).

The more interesting finding may be the bottleneck it exposes. On the clean-invoice dataset, the Docling-converted-text route capped nearly every model — regardless of size — in an 84-85% accuracy band, while the same models reading native images spread from roughly 46% to over 96% depending on the model. The paper's authors read that compression as evidence that the OCR-and-markdown conversion step, not the LLM's own reasoning, was the primary bottleneck for the text-based pipeline.

It's worth being plain about what this benchmark does and doesn't tell you. It's a zero-shot evaluation on three specific, openly available datasets, run by one research group, on one snapshot of models available in mid-2025. It is not a guarantee of how any of these models will perform on your own invoices, which will have their own layouts, languages, and scan quality. Treat it as evidence that native image input is worth trying before a text-conversion step, not as a forecast of the exact number you'll get.

Limits that actually rule an option out before accuracy does

Before accuracy ever enters the decision, the hard constraints above tend to eliminate options on their own:

  • A non-English invoice set immediately rules out Google Document AI's generative Custom Extractor, which officially supports only English for generative extraction.
  • A free-tier Azure evaluation will silently only read the first two pages of a scanned PDF — a limit that's easy to miss until a multi-page invoice comes back with half its line items missing.
  • Very large single documents or batches point away from a single synchronous call: Textract's asynchronous `StartExpenseAnalysis`/`GetExpenseAnalysis` path, or Claude's Files API with a reusable `file_id`, exist specifically for that case.
  • Password-protected or encrypted PDFs have to be unlocked before they reach either Azure's prebuilt model or Claude — neither will do that for you.
  • Document dimensions and file size have hard ceilings on Azure (50x50 to 10,000x10,000 pixels; 500MB paid, 4MB free) and on Claude's first-party API (128MB per request), so a very large scanned TIFF or an oversized batch request needs to be split or compressed before it's worth testing accuracy at all.
  • A single-page, real-time lookup and a large nightly batch job aren't interchangeable on Google Document AI either: 15 pages per synchronous request (30 with `imageless_mode` enabled) versus up to 200 pages for an asynchronous batch request.

Where this fits in an AP automation workflow

AI invoice processing doesn't stop at extraction — it's the first stage of an accounts-payable automation pipeline, not the whole thing. Whatever method pulls the fields off the invoice still needs to feed a validation and reconciliation step — matching the extracted PO number and totals against what's actually in your purchasing system — before anything gets posted to your ERP. The confidence scores from an OCR-plus-layout model, or a reviewer sign-off on low-confidence LLM extractions, typically determine what flows straight through versus what a human checks first.

If you want help scoping which extraction method fits your documents and wiring it into that validation step, talk to our team.

That kind of pipeline — extraction, validation, posting — is also where a lot of the operational complexity in AI workflow automation actually lives, beyond the extraction step covered here.

Frequently asked questions

What's the difference between OCR-based and LLM-based invoice extraction?

OCR-plus-layout models like Azure's `prebuilt-invoice` or AWS Textract's `AnalyzeExpense` are purpose-built for expense documents and return a fixed set of standardized fields, each with a confidence score and bounding box. LLM vision models like GPT with Structured Outputs or Claude reading a PDF document block extract whatever fields you define in a custom schema, but generally without the same per-field confidence scoring.

How accurate is AI invoice data extraction?

The only public academic benchmark we found, from Fraunhofer IAIS and the Lamarr Institute, measured zero-shot accuracy across eight multimodal LLMs on three open datasets. The top performer, Gemini 2.5 Pro reading native images, scored 87.46% on scanned receipts, 96.50% on clean synthetic invoices, and 92.71% on scanned invoices with real-world scan artifacts. Those numbers are specific to that paper's datasets and models — not a guarantee for your own documents.

Can AI extract line items from invoices, not just totals and headers?

Yes. Amazon Textract's `AnalyzeExpense`, for example, returns line items as a grouped `EXPENSE_ROW` with fields for item description, quantity, unit price, total price, and product code, each with its own confidence score. LLM vision models can extract line items too, as long as your schema defines them, though the Fraunhofer benchmark found line-item and unstructured fields (like IBANs) were generally harder to extract accurately than simple header totals.

Which AI tool is best for extracting data from PDF invoices?

It depends on your documents, not just on accuracy. Language coverage, page and file-size limits, and whether your PDFs are password-protected rule options out before accuracy becomes the deciding factor — for example, a non-English invoice set rules out Google's generative Custom Extractor outright, and a free-tier Azure account only reads the first two pages of a scanned PDF regardless of how accurate the model is.

Do I still need OCR if I'm using an LLM to read invoices?

Not as a separate step you run yourself. Claude and GPT read a PDF or image directly as part of their multimodal input — Claude's documentation describes PDFs as being processed as images, subject to the same limits as other vision tasks. The Fraunhofer benchmark specifically found that adding a separate OCR-to-markdown conversion step (via Docling) before the LLM reads the text actually reduced accuracy compared to feeding the model the native image directly.

Sources

Tell us what you're building.

Share a few details and we'll reply within 1–2 business days with questions, a scope and a quote.