Document AI Pipeline

What's in this pack

The seven parsers cover every shape of "this file used to be for humans, now an LLM has to read it." Some specialize (Surya for OCR, MinerU for math papers); others are generalists (Docling, Unstructured, MarkItDown). Pick by file mix and accuracy budget.

Why this matters

LLMs are surprisingly bad at reading raw PDF text. The bytes that look like prose to your eyes are usually scattered glyphs with no reading order — pdfplumber and PyMuPDF return jumbled output that confuses the model. Tables come out as broken rows. Headers and footers leak into the body. Multi-column layouts read top-to-bottom of the left column then top-to-bottom of the right, which is meaningless to a transformer.

This pack solves that. Surya and Zerox use vision models to see the page like a human and reconstruct logical reading order. Docling and Unstructured run layout-aware pipelines that label each element (heading, paragraph, table, caption) so downstream chunking respects structure. MinerU is the only open-source tool that reliably extracts equations and matrices from scientific papers.

For Office files (PowerPoint decks, Word docs, Excel sheets), MarkItDown is the answer. Microsoft published it because their own internal Copilot retrieval needed clean Markdown from Office, and existing converters were terrible.

Install in one command

# Install the whole pack
tokrepo install pack/document-ai-pipeline

# Or pick the parser that matches your file mix
tokrepo install docling
tokrepo install surya
tokrepo install markitdown

Each TokRepo asset page lists the supported file types, GPU requirements (Surya and Zerox want GPU; Docling and MarkItDown run on CPU), and the chunking strategy that pairs well downstream.

Common pitfalls

• OCR vs PDF text layer: a PDF with a text layer doesn't need OCR. Run Docling first; if the text layer is intact, skip Surya entirely. OCR is 10-100x slower than text extraction.
• Tables silently broken: most parsers extract tables but flatten rows incorrectly. Always sample 10 random table outputs and eyeball them before trusting the pipeline.
• Reading order on multi-column: legal documents and academic papers in two columns trip up naive parsers. Docling and Surya handle this; pdfplumber does not.
• Image captions get lost: figures are often the most information-dense part of a paper. Make sure your parser keeps caption text linked to the figure, not floating elsewhere.
• Token cost on Zerox: Zerox calls a vision-LLM per page. A 200-page PDF can cost $1-2 in API fees. Cache aggressively and prefer Docling-then-Zerox-fallback over running everything through Zerox.

Relationship to other packs

This pack is the ingestion layer for retrieval. It produces clean text and structured elements; the RAG Pipelines pack chunks, embeds, and serves them. For web pages instead of files, switch to AI Web Scraping. For voice or video content, those flow through speech-to-text first (out of scope for this pack).

A common production stack is: MarkItDown for Office → Docling for PDFs → Unstructured chunking → vector DB → RAG pipeline. The boundaries between packs are clean enough that you can swap any single layer without rewriting the rest.