Paper Explained: D-RAC — Normalize Everything to PDF, Read the Pages Once, Then Chunk by ID
D-RAC ingests PDFs, Word files and slides for RAG by rendering everything to PDF, converting the pages to Markdown with one multimodal LLM pass, and then letting an LLM plan chunks using only element IDs. Explained from scratch.
Key points
- D-RAC renders any document to PDF and runs a single multimodal LLM pass that turns the pages into retrieval-oriented Markdown.
- In the Markdown conversion, each table row becomes a self-contained sentence and the heading hierarchy is kept.
- At the chunking stage, an LLM arranges element IDs rather than text.
- Rule-based extraction and OCR scramble reading order, flatten tables and lose the heading hierarchy.
- It processed 236 documents (795 pages) in 72 minutes with zero errors and cut chunking-stage output tokens by 95.7%.
Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion
Primary source — what this article is built on
undefined2026-09-21→undefined2026-09-26same month
Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown ConversionUday Allu, Abhivanth Sivaprakash, Pratik Singh et al. · 2026-09-21 · v1arXiv:2609.24220Paper page·PDFundefined
Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presentations, and scans -- whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, flatten tables, and lose heading hierarchy, while fully agentic chunking over extracted text incurs high token costs and hallucination risk. We present Document Retrieval-Aware Chunking (D-RAC), an extension of our Web Retrieval-Aware Chunking (W-RAC) framework to arbitrary document formats. D-RAC first normalizes any input document into PDF, exploiting the fact that virtually every format has a faithful, deterministic PDF rendering. A single multimodal LLM pass then converts rendered pages into retrieval-optimized Markdown -- rewriting tables as self-contained prose statements and preserving heading hierarchy -- after which chunking proceeds exactly as in W-RAC: deterministic parsing into ID-addressable units followed by lightweight LLM-based chunk planning over identifiers rather than text. Source text is never regenerated during chunking, preserving W-RAC's cost, determinism, and observability benefits while unlocking every renderable format as a first-class input. On the 236-document, 795-page PDF subset of the RAG-Multi-Corpus benchmark spanning five enterprise domains, D-RAC converts and chunks the entire corpus in 72 minutes with zero errors, producing 1,748 retrieval-ready chunks. Compared to agentic chunking with frontier LLMs, D-RAC reduces chunking-stage output tokens by 95.7%, cutting chunking cost by 77.8% (GPT-4.1 pricing) to 85.6% (Gemini 2.5 Pro pricing) and chunking time by 75%. D-RAC scales linearly to documents of 500+ pages.
What problem does this paper solve?
The paper is "Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion" (arXiv:2609.24220), published in September 2026 by the AI research team at Yellow.ai.
Here is the abstract in a nutshell. A RAG system over a company knowledge base has to ingest PDFs, Word documents, slide decks and scans, whose content is trapped in multi-column layouts and dense tables. Rule-based extraction and OCR scramble reading order, flatten tables and lose the heading hierarchy. Handing the extracted text to an LLM and asking it to rewrite everything into chunks ("agentic chunking") is expensive in tokens and invites hallucination. D-RAC renders any document to PDF, runs a single multimodal LLM pass that turns the pages into retrieval-oriented Markdown (each table row becomes a self-contained sentence, headings are kept), and then chunks by having an LLM arrange element IDs rather than text. On 236 documents (795 pages) it finished in 72 minutes with zero errors, and compared with agentic chunking it cut chunking-stage output tokens by 95.7%, chunking cost by 77.8–85.6%, and chunking time by 75%.
If RAG itself is new to you, RAG fundamentals will make the rest easier to follow.
An analogy: split the copyist from the indexer
Imagine a library receiving a pile of mixed material: handwritten notes, brochures, price sheets full of tables. A scanner that just sucks out the characters is cheap, but two-column pages get interleaved and tables turn into a stream of numbers. Asking a skilled librarian to copy everything out by topic gives beautiful results, but it is slow, costly, and copying errors creep in.
D-RAC splits the work between two people. A copyist reads each document once and produces a clean copy with proper headings. An indexer then looks only at the paragraph numbers of that clean copy and decides "paragraphs 3–6 go in one folder." Because the indexer never rewrites text, the step is fast and cannot introduce copying errors. And if you later want to regroup, you never have to redo the clean copy.
Why PDFs are hostile to RAG
The paper describes PDF as "a presentation format, not a semantic one" (§1). Characters sit at page coordinates; nothing says which line is a heading or which column to read first. Pull text out with PyMuPDF or similar and columns interleave while tables become positionally ambiguous fragments (§2.1). Layout models such as LayoutLM or Docling help, but still emit tables as grids (§2.2). The core issue: a cell value stripped of its row and column headers means nothing. If the number "20" ends up embedded on its own, the retriever cannot tell whether it is a policy term or a payment term.
You can get a feel for how context-free fragments become unreachable in vector space with the widget below.
Comments
Sign in to comment