AI Agents & RAG

Verified against Claude · 2026-07-21

Extract structured data from messy documents without silently inventing fields

A prompt for turning unstructured documents into schema-validated JSON with an explicit null-versus-guess distinction and a confidence flag per field, for a pipeline that feeds directly into a database where a wrong value looks identical to a correct one.

ClaudeGPT-5.1InstructorLangExtract4 fillable variables

The prompt

Ready to copy — highlighted parts are example details you can swap.

You are extracting structured data from a vendor invoices (PDF, OCR'd text) into JSON that must validate against the schema below and feed directly into a downstream database — a wrong or invented value here is worse than a missing one, because it looks identical to a correct value once it's in the database.

TARGET SCHEMA
{ invoice_number: string, vendor_name: string, invoice_date: string, due_date: string | null, line_items: [{description, qty, unit_price, total}], subtotal: number, tax: number, total: number }

SOURCE DOCUMENT
the OCR'd text of a single invoice PDF, pasted in full including any garbled OCR artifacts

EXTRACTION RULES
- Extract only values actually present in the document, in the form they appear. Don't normalize a date format, currency, or unit unless the normalization rules below say to — normalization is itself a transformation that can introduce errors, so it needs to be an explicit, auditable rule, not something you decide to do on the fly.
Convert all dates to ISO 8601 (YYYY-MM-DD); leave currency amounts in the original currency, do not convert
- If a field required by the schema isn't present anywhere in the document, set it to null and add it to a separate "missing_required_fields" list — never fill it with a plausible-looking default just to satisfy the schema's type requirement.
- If a field's value is present but ambiguous — a name that could be split into first/last two different ways, a date format that could be read as either DD/MM or MM/DD — flag it in "ambiguous_fields" with the possible readings, rather than picking one silently.

CONFIDENCE HANDLING
For every extracted field, include a confidence of "high", "medium", or "low": high means the value appears verbatim and unambiguously; medium means the value required light inference, such as reading a total from a subtotal-plus-tax line rather than a single stated total; low means the value was inferred from context rather than stated directly. A downstream reviewer uses this to decide what to spot-check — don't mark everything "high" by default.

VALIDATION
After extraction, check your own output against the schema: correct types, required fields present even if null, no extra fields not in the schema. If your output fails this check, fix it before returning it — don't return invalid JSON and note the problem in prose instead.

OUTPUT FORMAT
Valid JSON matching the schema, plus three arrays: missing_required_fields, ambiguous_fields with alternate readings, and low_confidence_fields with the field name and why. If the document is unreadable, corrupted, or clearly not a vendor invoices (PDF, OCR'd text), say so instead of returning a JSON object full of nulls.

Customize

Optional — swap in your own details for the highlighted parts above.

Why this works

The pressure to invent a plausible value instead of returning null is not a hypothetical risk with structured extraction — it's a direct consequence of how JSON schema itself works. A schema that marks due_date as a required string field gives the model a structural incentive to produce some string rather than admit the document doesn't state one, because 'no due date visible' doesn't fit neatly into a required string type the way it fits into a sentence. Explicitly instructing the model to use null and a separate missing_required_fields list, rather than coercing the schema's type pressure into a guess, is what breaks that incentive — it gives the model somewhere honest to put 'not present' that isn't a lie dressed up as a value. Per-field confidence scoring is the mechanism that makes this pipeline actually deployable at volume. A team processing hundreds of invoices a day cannot manually review every field of every document — that defeats the entire point of automating extraction — but they also cannot ship financial data with zero review. Confidence flags let a human reviewer's attention go exactly where it's needed: spot-check the low-confidence and ambiguous fields, trust the high-confidence ones, which turns an all-or-nothing review decision into a targeted, sustainable one. Requiring the model to validate its own output against the schema before returning it catches a specific downstream failure mode: a JSON parser or database insert step further down the pipeline doesn't reason about a malformed field, it either throws an exception that halts the batch or, worse, silently coerces a wrong type. Catching a schema violation inside the extraction step itself, where the model that made the mistake can actually see and fix it, is much cheaper than catching it three systems downstream where nobody has the original document open anymore to figure out what should have been extracted.

What you get back

{ "invoice_number": "INV-88213", "vendor_name": "Acme Supply Co.", "invoice_date": "2026-06-14", "due_date": null, "subtotal": 1200.00, "tax": 96.00, "total": 1296.00, "line_items": [...] }, "missing_required_fields": ["due_date"], "ambiguous_fields": [], "low_confidence_fields": [{"field": "total", "why": "computed from subtotal + tax since no single stated total line was visible on the OCR'd page"}]

Verified against

Claude Sonnet 4.6 · 2026-07-21

GPT-5.1 2026-06 release · 2026-07-21

Changelog

  • 2026-07-21 Initial publish, verified against Claude Sonnet 4.6 and GPT-5.1 structured output on OCR'd invoice text.

Building this for real?

This is a free starting point. If you'd rather have AI agents & automation built and running for your business, that's Scult's day job.

EXPLORE AI AGENTS & AUTOMATION
All AI Agents & RAG prompts

Check your AI visibility

One URL in, a 0–100 score and the exact fixes out.

RUN THE CHECK

Browse all the tools

15 tools across six categories
13 of them never send your data anywhere

Free · No signup · No trial clock

SEE THE DIRECTORY