Verified against Unstructured.io · 2026-08-03
Get tables and figures out of PDFs without flattening them into useless text
A prompt for designing how a RAG ingestion pipeline handles tables, charts, and images inside PDFs — deciding what stays structured, what gets described by a vision model, and what gets skipped entirely — instead of routing everything through one generic text extractor that turns a table into unreadable word soup.
The prompt
Ready to copy — highlighted parts are example details you can swap.
You are designing how a RAG ingestion pipeline handles the non-plain-text content inside a annual financial reports and product spec sheets (PDF) — tables, charts and figures, and embedded images — deciding a specific extraction strategy per content type instead of routing everything through one generic text extractor that turns a table into a stream of numbers with no structure left.
DOCUMENT PROFILE
Financial reports contain multi-row-header tables (a metric broken out by quarter and by region in the same table) and bar/line charts; spec sheets contain simple single-header comparison tables and product photos
CONTENT TYPES PRESENT
multi-header financial tables, single-header spec tables, bar/line charts with axis labels, product photography, company logos on every page header
EXTRACTION STRATEGY PER TYPE
For tables:
- State whether tables get extracted to a structured format, such as a Markdown table with column headers preserved, versus flattened into a plain-text left-to-right, top-to-bottom read order. Flattening a table with more than a couple of columns produces text where a number is disconnected from the column header and row label that actually give it meaning — decide explicitly whether this document profile's tables are simple enough for flattening to be acceptable, or complex enough, with multi-row headers or merged cells, that structure-preserving extraction is required.
- State whether a table gets embedded and retrieved as one unit, so a query never gets half a table with no header, or whether individual rows can be independently retrieved, which is only viable if each row genuinely stands alone in meaning without its header row present in the same chunk.
For charts and figures:
- State whether these get a caption-style description generated by Claude (vision) generating a structured description per chart, and if so, what that description is required to state explicitly — the actual data or trend shown, not a description so generic it would be equally true of a hundred different charts.
- State what happens to the figure's own caption or label text if the source PDF has one — it should be preserved verbatim alongside the generated description, not replaced by it, since the caption often carries information the generated description won't reproduce, such as a figure number referenced elsewhere in the document.
For embedded images with no clear semantic role, such as logos or decorative elements:
- State the rule for skipping these rather than generating a description for every image indiscriminately — a vision-model description of a company logo is pure noise in a retrieval index and costs a real vision-model call for zero retrieval value.
RETRIEVAL IMPLICATIONS
Analysts ask both direct lookup questions ('what was APAC revenue in Q3') and interpretive questions ('why did the chart show a dip in Q2') — the pipeline needs to support both — for each content type above, state explicitly how a real query would need to reach it: does a question about a specific figure need the table's numeric content directly retrievable, or does it need the surrounding paragraph that already interprets the table in prose? Sometimes the answer is genuinely both, and the pipeline needs to keep them linked, not just co-located.
VALIDATION
Run 10 real analyst questions against the extracted content end to end and confirm the correct number/trend is retrievable and correctly attributed, not just that a chunk containing roughly the right area of the page was returned — state how you'll verify the chosen strategy actually preserved enough structure to answer a real question against a real extracted table, not just that extraction ran without an error.
OUTPUT FORMAT
1. The extraction strategy per content type, stated as a concrete rule.
2. How each extracted piece gets linked back to its surrounding text and page number.
3. One worked example: a specific table from the document profile, its chosen extraction format, and a sample query it needs to answer correctly.Customize
Optional — swap in your own details for the highlighted parts above.
Why this works
Whether a table needs structure-preserving extraction or can tolerate flattening is a real, testable distinction, not a stylistic choice, because flattening's failure mode is concrete: reading a multi-column table left to right and top to bottom in plain text produces a sequence of numbers with no attached column header by the time a chunking step or an embedding model sees them, so a retrieved chunk can contain the right number with no way for the model reading it to know which region or quarter that number belongs to. A single-header, two-column comparison table survives flattening fine because the row label sits right next to its value; a table with merged header cells spanning multiple quarters and regions does not, and pretending the two cases need the same extraction strategy is how a financial-reporting RAG pipeline ends up occasionally citing a real number attached to the wrong quarter with complete confidence. Requiring a chart description to state the actual data or trend shown, rather than accepting a generically accurate caption, targets a specific and common weakness of naive vision-captioning: a caption that would be equally true of a hundred different charts is technically not wrong, but it's also not retrievable against any question that needs the chart's actual content, because nothing in that caption distinguishes this chart from any other bar chart in the corpus. A description has to name the specific trend to be worth the vision-model call that produced it — otherwise the extraction step ran successfully and still produced something functionally useless for retrieval. Explicitly deciding to skip decorative images like page-header logos, rather than running every embedded image through the same vision-description pipeline, matters because indiscriminate captioning has a real, compounding cost: every page of every document in the corpus likely repeats the same logo, and captioning it burns a vision-model call and adds a near-duplicate, semantically empty chunk to the index on every single page, multiplying storage and noise for content that will never usefully answer a real question. A pipeline with no explicit skip rule pays this cost by default and only notices it later as an unexplained increase in vision-API spend and a retrieval index quietly padded with logo descriptions.
What you get back
Table: 'Q1-Q4 Revenue by Region' (multi-row header: quarter over region) -> extracted as a structured Markdown table, kept as one atomic chunk with its full header intact, linked to page 14 and the preceding paragraph that interprets it. Sample query: 'what was APAC revenue in Q3' correctly retrieves the whole table plus the row/column intersection, rather than an isolated number with no header context.
Verified against
Unstructured.io 0.18 · 2026-08-03
Claude Sonnet 4.6 (vision) · 2026-08-03
Changelog
- 2026-08-03 — Initial publish, verified against Unstructured.io 0.18 table extraction and Claude Sonnet 4.6 vision-based chart description.
Building this for real?
This is a free starting point. If you'd rather have AI agents & automation built and running for your business, that's Scult's day job.
EXPLORE AI AGENTS & AUTOMATION
