PDF Parse Lifecycle Management#

Overview#

During PDF conversion, the StandardPdfPipeline maintains three categories of intermediate per-page artifacts in the Page datamodel:

1. _image_cache — A dict[float, Image] mapping render scale → PIL image, populated during the preprocessing stage 1.
2. _backend — A reference to the PdfPageBackend (pypdfium2 or docling-parse), which holds C-side page resources 2.
3. parsed_page — A SegmentedPdfPage | None carrying PDF text/bitmap cells extracted from the backend 3.

All three are expensive: _image_cache holds decoded bitmaps in RAM, _backend ties up native memory/file handles, and parsed_page stores the full cell list per page. The pipeline garbage-collects them as early as possible — unless downstream stages need them.

Decision Logic: What Gets Kept#

Two pipeline-level boolean flags gate resource retention. Both are computed once in _init_models() and held for the lifetime of the pipeline instance.

keep_images#

Set to True if any image-output feature is active 4:

generate_page_images OR generate_picture_images OR generate_table_images

When False, _image_cache is cleared to {} after assembly.

keep_backend#

Set to True if any enrichment model is active 5:

do_formula_enrichment OR do_code_enrichment OR do_picture_classification
OR do_picture_description OR do_chart_extraction

Enrichment models (post-assembly VLMs, picture classifiers) crop element bounding boxes from page images after assembly via BaseItemAndImageEnrichmentModel. If no enrichment flag is set, the backend is unloaded immediately after assembly and its reference is nulled.

generate_parsed_pages#

A separate PdfPipelineOptions field controlling parsed_page retention 6. Defaults to False. When False, parsed_page is set to None after assembly. Setting this to True is required for HeadingHierarchyModel's use_style inference, which needs parsed PDF cell heights 7.

Cleanup: Where and When#

Cleanup happens at two points in the pipeline execution:

1. Per-page cleanup (after assembly stage)#

_release_page_resources() runs as a postprocess callback on the assemble ThreadedPipelineStage — immediately after each page exits the assemble worker thread 8:

• Clears _image_cache if not keep_images
• Calls page._backend.unload() and sets _backend = None if not keep_backend
• Sets parsed_page = None if not generate_parsed_pages

This per-page, streaming cleanup frees memory while other pages are still flowing through upstream stages — critical for multi-page documents in the threaded pipeline.

2. Bulk cleanup (after all pages complete)#

_integrate_results() performs a second sweep over all collected pages after the pipeline drains 9. This is a safety net for pages that may have bypassed the per-page hook (e.g., failed pages, edge cases).

Flow Summary#

Key Files#

Common Scenarios#

Implication: Any enrichment feature (do_picture_classification, do_picture_description, do_code_enrichment, do_formula_enrichment, do_chart_extraction) prevents early backend unloading, keeping native PDF page resources alive through the enrichment pass. For memory-constrained environments, disable unused enrichment features.