Intelligent document processing (IDP) is software that reads business documents such as invoices, forms, contracts and claims, works out what each document is, pulls the fields you need into structured data, checks them, and sends anything uncertain to a person. OCR only turns an image into text; IDP adds the classification, extraction, validation and review steps that make that text safe to load into a system of record. Vision language models can now do much of the reading, which moves the hard part of the job: getting text off the page is largely solved, and deciding when to trust the result is where the real design work sits.
What is intelligent document processing?
Google's description of its own product is a good working definition: Document AI "takes unstructured data from documents and transforms it into structured data (specific fields, suitable for a database)". Microsoft positions Azure Document Intelligence as a service "that you can use to build intelligent document processing solutions", which is the honest framing: IDP is a category of solution you assemble, not a box you buy.
The documents in question are the ones that keep operations and finance teams busy: invoices, purchase orders, bank statements, delivery notes, forms, claims and contracts, arriving as attachments, scans and phone photos in layouts nobody controls. The goal is always the same: the right values in the right fields of your ERP, CRM or database, with an audit trail back to the source page.
How does intelligent document processing work?
Every serious IDP system runs some version of the same seven steps.
- Capture. Documents come in from inboxes, upload portals, scanners and shared drives. Files are normalized and multi-document PDFs are split; Google lists the ability to "split and classify documents by type. For example, a PDF file with multiple real documents".
- Digitize. OCR turns pixels into text with positions. Document AI does this "using OCR to get text, layout, and various add ons such as image quality detection (for readability) and deskewing".
- Classify. The system decides what each document is, because an invoice and a credit note need different handling. Azure's custom classifiers "identify document types before invoking an extraction model".
- Extract. Fields and tables are pulled into a defined schema: supplier, invoice number, dates, line items, totals.
- Validate. Rules check the result: totals add up, dates are real dates, the supplier exists in your master data, the purchase order number is open.
- Review. Anything with low confidence or a failed rule goes to a person, who sees the extracted value next to the region of the page it came from.
- Export. Approved data is written to the target system, with a link back to the source file for audit.
Steps 1 to 4 get the attention in vendor demos. Steps 5 and 6 decide whether the system can be trusted with real money.
What is the difference between OCR, IDP and LLM extraction?
OCR is one step inside IDP. LLM extraction is a newer way of doing the classification and extraction steps. They are not competing products so much as layers.
| OCR | Classic IDP (trained or prebuilt models) | LLM or vision language model extraction | |
|---|---|---|---|
| Output | Text, lines, words and positions | Named fields and tables, often with a confidence score per field | JSON that follows a schema you describe in the request |
| New layouts | Reads any layout, understands none | Prebuilt models cover common types; custom types need labeled samples | Handles unseen layouts from a written field description |
| Setup effort | Low | Labeling and training for custom documents | Prompt and schema design, plus a test set |
| Typical failure | Misread characters on poor scans | Fields missed on layouts unlike the training set | Plausible but wrong values, or values invented for fields that are not on the page |
| Best for | Search, archiving, feeding other steps | High-volume standard documents | Varied, messy or reasoning-heavy documents |
Most teams end up combining them: a layout or OCR service where it is cheap and reliable, a language model where documents vary, and one validation and review layer around both.
What did vision language models change?
Classic IDP handles a new document type by training on examples: Azure's custom models, for instance, are trained by using your labeled datasets. A general vision language model can instead be handed the page and a written description of the fields. Anthropic's documentation explains how this works for PDFs: the system converts each page of the document into an image, and the text from each page is extracted and provided alongside each page's image, so the model can use both the words and the visual layout, including "any text, pictures, charts, and tables".
The cloud IDP vendors have absorbed this rather than been replaced by it. Microsoft now describes Document Intelligence as providing "high-accuracy and reliable deterministic extraction from structured documents", while its sibling service Content Understanding offers "LLM-powered analyzers for complex, unstructured, and multimodal content". Google says Document AI "is built on top of products within Vertex AI with generative AI".
Two limits matter in practice. First, cost and size: Anthropic says a page typically uses between 1,500 and 3,000 text tokens depending on content density, with image tokens charged on top because each page is also an image, and caps a request at 600 pages, or 100 when the context window is under 1M tokens. Second, there is no universal winner: the OmniDocBench benchmark, accepted at CVPR 2025, evaluated both pipeline tools and end-to-end vision language models and reported their strengths and weaknesses across different document types. Test on your own documents, not on a vendor's samples.
How do you force extraction output into a schema?
With structured outputs, which both major model providers now offer. Anthropic says its implementation guarantees "schema-compliant responses through constrained decoding". OpenAI says its version "ensures the model will always generate responses that adhere to your supplied JSON Schema". That removes a whole class of failures: broken JSON, missing keys, a date where a number should be.
It does not make the values right. OpenAI states plainly that "Structured Outputs can still contain mistakes", and warns that the model "will always try to adhere to the provided schema, which can result in hallucinations if the input is completely unrelated to the schema". Anthropic notes that refusals and responses cut off at the token limit may not match the schema. For document work this suggests four habits:
- Make every field nullable, and tell the model to return null when a value is not on the page.
- Include a document type field with an "other" option, so an unexpected file is labeled rather than forced into an invoice shape.
- Ask for the page number or quoted source text for each critical value, so a reviewer can check it in seconds.
- Validate the output in code against your own data, never against the model's own claims.
The same schema discipline applies to invoices in particular, which have their own matching and approval logic; our guide to invoice processing automation covers that end to end.
How accurate is IDP, and how do you handle exceptions?
Accuracy is a number you measure on your own documents and then manage with thresholds. Microsoft's documentation is unusually clear about what a confidence score means: "Field confidence indicates an estimated probability between 0 and 1 that the prediction is correct. For example, a confidence value of 0.95 (95%) indicates that the prediction is likely correct 19 out of 20 times." It adds that confidence "can be used to determine whether to automatically accept the prediction or flag it for human review", and that not all document fields return a confidence score. For custom models it recommends targeting an accuracy score of 80% or higher, and close to 100% for sensitive cases such as financial or medical records.
Think about what 19 out of 20 means at volume: on 2,000 documents a month, a field at that confidence is wrong on about 100 of them. That is why exception handling is the real product. A working design has:
- Per-field thresholds. A bank account number needs a much higher bar than a delivery note reference.
- Cross-field rules. Line items must sum to the subtotal, tax must match the rate, the supplier's tax ID must match master data.
- A review screen people will use. The value, the highlighted source region and one click to accept or correct.
- A feedback loop. Corrections become test cases for the next model or prompt change.
Do not assume the vendor will supply the review layer. Amazon built a human review service, Augmented AI, that can send a Textract result to people when the inference confidence for a key falls within a range that you specify, but the same page now states that "Amazon SageMaker A2I is no longer open to new customers". Plan to build the queue yourself.
Which intelligent document processing tool should you use?
Start from your documents, not from the tool. The three big cloud services cover similar ground with different packaging:
| Service | What it offers for documents |
|---|---|
| Google Document AI | Enterprise Document OCR, Form Parser, Layout Parser, pretrained parsers, custom extractors, custom classifiers and splitters |
| Azure Document Intelligence | Read and Layout models, prebuilt models for invoices, receipts, bank statements, contracts, IDs and US tax forms, plus custom extraction and classification models |
| Amazon Textract | Text, forms and tables, Queries, AnalyzeExpense for invoices and receipts, AnalyzeID for US government IDs, and a lending workflow |
| LLM APIs with structured outputs | Extraction from any layout against a schema you define, with PDF and image input |
If most of your volume is a standard type the prebuilt models cover, start there. If your documents vary or need judgment, such as which contract clause sets the notice period, a language model with a strict schema is usually faster to stand up. Either way, budget more time for validation and review than for extraction.
Once documents are extracted, the text often becomes useful a second time as a searchable knowledge source; that is where a vector database comes in. For more ways teams put this to work, see our list of AI automation examples. And if you would rather have the whole pipeline built around your own documents, review rules and systems, that is the kind of document processing work we scope one process at a time. When we scope it, the first thing we ask for is a sample of real documents, including the worst scans you have.
Frequently asked questions
Is intelligent document processing the same as OCR?
No.
OCR converts an image of text into machine-readable text and stops there.
IDP uses OCR, or a vision model, as one step, then classifies the document, extracts specific fields, checks them against rules and routes exceptions to people.
A searchable PDF is an OCR result; a supplier invoice posted correctly to your accounting system is an IDP result.
Can ChatGPT or Claude do intelligent document processing?
They can do the reading, classification and extraction steps, and both providers' APIs accept PDFs and return schema-valid JSON.
A chat window does not give you the rest: intake from your inboxes, validation against master data, a review queue, an audit trail and export into your systems.
That surrounding system is where most of the build effort goes.
How do you measure IDP accuracy?
Build a labeled test set from your own documents, including bad scans and unusual layouts, and score it field by field rather than per document.
Then track three numbers in production: field-level error rate on a sampled audit, the share of documents that pass straight through without review, and how long reviews take.
A rising straight-through rate with a flat error rate is the sign the system is improving.