Custom Enrichment Models#

Docling's enrichment pipeline is a post-document-assembly processing phase. After layout analysis, OCR, and table extraction produce a DoclingDocument, an ordered list of enrichment models annotates specific element types with additional information. All built-in enrichments are disabled by default because they require extra model inference. 1

Base Classes#

All enrichment models live in docling/models/base_model.py and extend one of two abstract bases, both of which inherit from GenericEnrichmentModel[T].

GenericEnrichmentModel declares three required abstract methods 2:

BaseEnrichmentModel#

BaseEnrichmentModel is the simplest variant: prepare_element returns the element unchanged if is_processable returns True. No image is involved. Use this for text-only enrichment.

BaseItemAndImageEnrichmentModel#

BaseItemAndImageEnrichmentModel is used when inference requires a cropped image of the element's bounding box. Its prepare_element implementation:

1. For PictureItem with an embedded image: returns it directly 3.
2. Otherwise: crops the element's region from the source page image at self.images_scale, optionally expanding the bounding box by self.expansion_factor 4.

The result is an ItemAndImageEnrichmentElement — a (item, image) pair ready for batch inference. This handles both rendered pages (PDFs) and embedded-image documents (Word, HTML) transparently 5.

Two class attributes control image extraction 6:

• images_scale: float — rendering scale (e.g., 2 in DocumentPictureClassifier, 1.67 in CodeFormulaVlmModel)
• expansion_factor: float = 0.0 — fractional bounding-box expansion (e.g., 0.18 in CodeFormulaVlmModel)

Execution Loop#

BasePipeline._enrich_document() iterates each model in self.enrichment_pipe, calls prepare_element() to filter and prepare candidates, chunks them into batches via elements_batch_size, and calls the model on each batch. The inner iterator must always be fully exhausted. 1

Implementing a Custom Enrichment Model#

The scaffold example in docs/examples/develop_picture_enrichment.py shows the full integration pattern:

1. Subclass BaseEnrichmentModel (for text-only) or BaseItemAndImageEnrichmentModel (for image+item):

   ⧉

   class MyEnrichmentModel(BaseEnrichmentModel):
       def is_processable(self, doc, element):
           return self.enabled and isinstance(element, PictureItem)
   
       def __call__(self, doc, element_batch):
           for element in element_batch:
               # run inference, annotate element
               yield element
   

2. Subclass the pipeline and replace self.enrichment_pipe 7:

   ⧉

   class MyPipeline(StandardPdfPipeline):
       def __init__(self, pipeline_options):
           super().__init__(pipeline_options)
           self.enrichment_pipe = [MyEnrichmentModel(enabled=True)]
   

3. Wire it into DocumentConverter via PdfFormatOption(pipeline_cls=MyPipeline) 8.

Note: The example scaffold uses BaseEnrichmentModel (no image crop) for simplicity. If your model needs a page-cropped image, subclass BaseItemAndImageEnrichmentModel and set images_scale. See DocumentPictureClassifier for a production example.

Production Implementations#

DocumentPictureClassifier sets images_scale = 2 12, filters to PictureItem instances 13, runs batched inference via a classification engine, and stores results in item.meta.classification as PictureClassificationMetaField 14.

Plugin Architecture for External Models#

Custom enrichment models can be injected in two ways 1:

1. Pipeline subclass (recommended for development)#

Subclass StandardPdfPipeline, override __init__, and reassign self.enrichment_pipe. Pass via PdfFormatOption(pipeline_cls=...). This is the approach shown in the scaffold example.

2. Setuptools entry-point plugins#

Register models via the "docling" entry-point group. The factory system uses pluggy's PluginManager to discover them. External plugins (modules not starting with "docling.") are only loaded when allow_external_plugins=True is set in PipelineOptions — disabled by default for security. 15

Handling Handwritten Text and Mixed-Content Documents#

BaseItemAndImageEnrichmentModel is the right base class for handwriting classifiers. A custom model follows the same pattern as DocumentPictureClassifier: filter TextItem regions, crop their page images, classify as printed vs. handwritten, and store a flag in element metadata 16.

Architecture note: An enrichment-based handwriting classifier runs after the full pipeline (layout → OCR → assembly). This means handwritten regions still pass through Docling's print-oriented OCR before being flagged. For scenarios where routing handwritten regions to a dedicated HTR tool before OCR is required, an external orchestrator approach using LayoutPredictor standalone is more appropriate. 16

Key Files#