Pipeline and Model Caching#

Overview#

DocumentConverter maintains a per-instance pipeline cache that allows initialized pipelines and their loaded ML models to be reused across multiple document conversions without re-loading weights. The cache scope is one converter instance — a new DocumentConverter() starts a fresh, empty cache and performs a full model load on first use.

Pipeline Cache: How It Works#

The cache lives in DocumentConverter.initialized_pipelines, a plain dict keyed by a (pipeline_class, options_hash) tuple:

self.initialized_pipelines: dict[
    tuple[Type[BasePipeline], str], BasePipeline
] = {}

The options hash is computed in _get_pipeline_options_hash by serializing the PipelineOptions model to a dict, converting to a string, and taking its MD5 digest:

options_str = str(pipeline_options.model_dump())
return hashlib.md5(options_str.encode("utf-8"), usedforsecurity=False).hexdigest()

Cache lookups happen inside _get_pipeline, which is called once per document via _execute_pipeline. A module-level threading.Lock (_PIPELINE_CACHE_LOCK) serializes concurrent initializations 1:

• Cache miss: the pipeline class is instantiated with the provided options, stored, and returned.
• Cache hit: the existing pipeline is returned as-is, logged at DEBUG level.

Because the key includes both class and options hash, two formats that share the same pipeline class but with different options each receive their own pipeline instance. Conversely, two formats sharing the same class and the same options reuse a single instance.

Model Loading: Inside the Pipeline#

Models are not cached independently — they are attributes of the pipeline object. Once a pipeline is cached, its models are implicitly cached with it.

For StandardPdfPipeline, _init_models is called from __init__ and loads all heavy models exactly once per pipeline instance 2:

These model objects are then handed to per-execution ThreadedPipelineStage workers as read-only references on every execute call — stages never mutate model state 3.

Factory-Level Process Cache (lru_cache)#

The model factories themselves (OcrFactory, LayoutFactory, TableStructureFactory, PictureDescriptionFactory) are cached at the process level using @lru_cache in docling/models/factories/__init__.py. This means factory objects (which discover and register plugin model classes via pluggy) are created only once per process, regardless of how many DocumentConverter instances exist.

BaseFactory.create_instance 4 is called at pipeline construction time to build the actual model object. The resulting model is owned by the pipeline, not the factory.

Lifecycle Summary#

Process
└── lru_cache factories (OcrFactory, LayoutFactory, …) ← process-level singleton

DocumentConverter (instance A)
└── initialized_pipelines dict
    ├── (StandardPdfPipeline, hash1) → pipeline1 ← holds OCR/layout/table models
    └── (SimplePipeline, hash2) → pipeline2

DocumentConverter (instance B)
└── initialized_pipelines dict ← separate; no sharing with A
    └── (StandardPdfPipeline, hash1) → pipeline3 ← new model load

Key implication: long-lived server applications should create one DocumentConverter and reuse it. Creating a new converter per request causes redundant model loads and significantly higher memory and startup time.

Explicit Pre-warming#

DocumentConverter.initialize_pipeline(format) can be called before any document is submitted to force a cache miss and model load eagerly (e.g., at application startup):

converter = DocumentConverter()
converter.initialize_pipeline(InputFormat.PDF) # loads models now, not on first convert

Important Caveats#

• No cross-instance sharing: the initialized_pipelines dict is an instance attribute. There is no global or class-level pipeline pool.
• Hash mutation bug: the pipeline options hash is computed at pipeline construction and stored in the cache key. A known issue (#3109) is avoided by copying code_formula_options in _init_models before mutating them, to prevent hash drift 5.
• Thread safety: _PIPELINE_CACHE_LOCK protects writes to initialized_pipelines, but the pipeline models themselves are designed to be read-only during execution.

Key Files#