Document Layout Detection Models#

Docling's layout detection layer identifies and localizes structural elements on document page images using a family of RT-DETR-based object detection models. Two model families are available: Heron and Egret, each hosted on HuggingFace under the docling-project organization. All variants share the same 17-label taxonomy and a common LayoutPredictor API, making them interchangeable in the pipeline.

Architecture#

All models are loaded via LayoutPredictor in docling-ibm-models, which wraps HuggingFace's AutoModelForObjectDetection and RTDetrImageProcessor. The predictor exposes two inference methods:

• predict(image) — single-image inference, yields bounding box dicts
• predict_batch(images) — batched inference, more efficient for pipeline use

Each result dict contains keys label, confidence, l, t, r, b (pixel coordinates, top-left origin) 1.

Within the Docling pipeline, LayoutModel.predict_layout() calls predict_batch() and feeds results into LayoutPostprocessor for confidence thresholding, label remapping, cell-to-cluster assignment, orphan cell recovery, and overlap resolution 2.

Model Variants and Configuration#

Model specifications are defined as LayoutModelConfig instances (Pydantic models) and registered in LayoutModelType:

3

Default model is DOCLING_LAYOUT_HERON, set as the default value of LayoutOptions.model_spec in pipeline_options.py. To override, pass a different spec to LayoutOptions:

from docling.datamodel.layout_model_specs import DOCLING_LAYOUT_EGRET_XLARGE
# pass via LayoutOptions(model_spec=DOCLING_LAYOUT_EGRET_XLARGE)

All variants support CPU, CUDA, MPS, and XPU accelerators 4.

The 17-Label Taxonomy#

All model variants output the same canonical label set, defined in LayoutLabels:

5

Labels 0–10 originate from DLNv1/v2; labels 11–16 were added in DLNv2. The Text label does not distinguish printed from handwritten content — any differentiation requires a downstream classifier 6.

LayoutPredictor also accepts a blacklist_classes set to suppress specific labels at inference time 7.

Key Source Files#