Document Type Classification and Routing#

Docling has no built-in document-level classification or multi-pipeline dispatch. Pipeline selection is static and based solely on InputFormat (PDF, DOCX, IMAGE, etc.) — not on document content, document type, or per-page analysis. Users must implement any document-type detection and conditional routing themselves.

How Pipeline Selection Works#

DocumentConverter._get_pipeline() maps each input file's InputFormat to a FormatOption, extracts the pipeline class and options, then caches and reuses the instance. There is no hook for content-based routing — the pipeline class is fixed at DocumentConverter initialization time.

The two main PDF pipelines:

• StandardPdfPipeline — default for PDFs; staged layout → OCR → table → assembly.
• VlmPipeline — holistic page understanding via a Vision-Language Model; must be explicitly specified via PdfFormatOption(pipeline_cls=VlmPipeline, ...).

There is no automatic selection between them based on document content.

Region Labels Are Not Document Classification#

Docling's layout model emits 17 structural labels per detected region — Text, Table, Picture, Form, Key-Value Region, Checkbox-Selected/Unselected, Code, etc. 1 — but these are region-level structural labels, not a document-level type signal. No Docling component answers "what type is this document?" (invoice, legal filing, scientific paper, etc.).

The DocumentPictureClassifier enrichment model classifies individual picture regions into subtypes (chart, photo, barcode, etc.) and stores results in PictureItem.meta.classification 2, but this is also region-level and picture-specific — not document-level classification.

Common Pain Points#

Form pages in StandardPdfPipeline: Pages classified as FORM or KEY_VALUE_REGION by the layout model are linearized destructively — field labels and values dissociate with no degradation signal, and downstream RAG/LLM consumers receive plausible-but-wrong content 3. The conflict resolution rules in LayoutPostprocessor (e.g., KEY_VALUE_REGION vs TABLE overlap) are hardcoded and not configurable 4.

Scanned PDFs needing OCR: PdfPipelineOptions.do_ocr is a simple boolean 5 with no automatic detection of which PDFs have broken text layers (e.g., text drawn as Bézier curves). A parse_score quality signal exists in the preprocessing model but nothing in the current pipeline acts on it to auto-escalate to full-page OCR 6. A community proposal for OcrOptions.force_full_page_ocr_detection based on parse_score thresholds and text quality heuristics has been discussed but not yet shipped 6.

Multi-template VLM routing: Users wanting to route different region types (table/text/formula) to different VLM prompts face an architectural gap: StandardPdfPipeline supports external layout plugins but not VLM routing; the experimental ThreadedLayoutVlmPipeline combines layout + VLM but does not support external layout plugins because it directly instantiates LayoutModel instead of going through the plugin factory system 7.

Manual Classification Patterns#

Pre-classification before DocumentConverter (simplest approach): Run a document classifier upstream and select the pipeline before constructing DocumentConverter. For form-heavy documents, the officially recommended workaround is to use VlmPipeline 3:

from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import VlmPipelineOptions, VlmConvertOptions
from docling.pipeline.vlm_pipeline import VlmPipeline

vlm_options = VlmConvertOptions.from_preset("granite_docling")
pipeline_options = VlmPipelineOptions(vlm_options=vlm_options)

converter = DocumentConverter(
    format_options={
        InputFormat.PDF: PdfFormatOption(
            pipeline_cls=VlmPipeline, pipeline_options=pipeline_options
        )
    }
)

VlmPipeline requires a GPU and is slower (~2–4 pages/sec on high-end hardware) but handles complex form layouts substantially better than StandardPdfPipeline 3.

Text-quality heuristics for OCR escalation: Inspect parse_score from the page preprocessing model, text-to-bitmap ratios, and U+FFFD replacement-character counts before conversion. If indicators suggest a broken text layer, set do_ocr=True or switch to a full-OCR configuration 6.

Page-level layout sampling: Use LayoutPredictor from docling-ibm-models as a standalone component (no full pipeline required) to detect the label distribution across pages before committing to a pipeline. A high ratio of Form or Key-Value Region predictions signals that VlmPipeline may produce better results 8. The predictor accepts a PIL Image and returns dicts with label, confidence, and pixel-space bounding box coordinates 8.

Standalone Region Routing#

For custom scatter-gather architectures, LayoutPredictor can be used entirely outside DocumentConverter to drive per-region routing before any OCR or specialized processing runs 9. Key constraints to keep in mind:

• No handwritten label: the Text label covers both printed and handwritten regions; distinguishing them requires a secondary external classifier 8.
• Single model per pipeline: the custom layout plugin system (BaseLayoutModel, registered via setuptools entry points with allow_external_plugins=True) supports exactly one layout model per pipeline; ensemble or parallel layout models are not supported 10.
• ThreadedLayoutVlmPipeline plugin gap: the experimental combined layout+VLM pipeline bypasses the plugin factory, so custom layout models cannot be used with it 7.

Key Source References#