Caption-Figure Linking#

Overview#

Caption-figure linking is the mechanism by which Docling associates CAPTION-labeled text blocks with their owning PictureItem or TableItem in the final DoclingDocument. It runs as part of the ReadingOrderModel stage inside _assemble_document() — after per-page layout is complete but before the document tree is finalized.

Two paths produce caption associations:

1. Reading-order prediction — ReadingOrderPredictor.predict_to_captions() assigns free-standing caption elements to nearby figures/tables based on adjacency in the sorted reading order.
2. Layout child elements — Caption clusters that the layout model nests directly inside a FigureElement cluster are added as child nodes via _add_child_elements().

The end result in both cases is a FloatingItem.captions list of RefItem pointers. At runtime, caption_text(doc) resolves each pointer and concatenates the text.

Data Model: FloatingItem and RefItem#

PictureItem and TableItem both inherit from FloatingItem, which carries three reference lists:

RefItem wraps a JSON pointer string (the $ref / cref field, e.g. "#/texts/3") that resolves to a concrete node in the document object graph via RefItem.resolve(doc).

FloatingItem.caption_text(doc) is the standard accessor: it iterates self.captions, resolves each RefItem, and concatenates .text. There are no equivalent helper methods for footnotes or references.

Pipeline Stage: Where Linking Happens#

Caption linking is orchestrated by ReadingOrderModel.__call__(), which runs sequentially during _assemble_document() in the StandardPdfPipeline. The call sequence is:

1. predict_reading_order(page_elements) → sorted_elements
2. predict_to_captions(sorted_elements) → el_to_captions_mapping
3. predict_to_footnotes(sorted_elements) → el_to_footnotes_mapping
4. predict_merges(sorted_elements) → el_merges_mapping
5. _readingorder_elements_to_docling_doc(...) → final DoclingDocument

All caption and footnote cids collected from the mappings are placed into a skippable_cids set 1. Elements in this set are not emitted as standalone body items during document construction — they appear only as children of their owning figure or table.

Caption Prediction Algorithm#

predict_to_captions() delegates per-page work to _find_to_captions() in docling-ibm-models. The algorithm uses reading-order adjacency, not pixel proximity:

1. Scan neighbors — For each CAPTION element, scan the sorted list backward and forward to identify adjacent TABLE, PICTURE, or CODE elements 2.
2. Uncontested assignment — If a caption has neighbors on only one side, assign it to those elements immediately 3.
3. Conflict removal — Strip already-claimed elements from competing captions to prevent double-assignment 4.
4. Re-assignment pass — Apply the single-direction logic again to any still-unassigned captions 5.
5. Deduplication — _remove_overlapping_indexes ensures each element maps to at most one caption; when multiple captions compete for the same element, the one with the smallest |cid distance| wins 6.

The output is a dict[int, list[int]] mapping each floating element's cid to a list of caption cids.

Footnote linking via _find_to_footnotes() is simpler: for each TABLE or PICTURE, it scans forward and collects all immediately consecutive FOOTNOTE elements — no backward search or conflict resolution.

Wiring Captions into the DoclingDocument#

_readingorder_elements_to_docling_doc() iterates sorted_elements in order, building the document tree. For each floating element:

• FigureElement — add_picture() creates a PictureItem. For each caption_cid in el_to_captions_mapping[rel.cid], _add_caption_or_footnote() inserts the caption as a child TextItem under pic, then pic.captions.append(new_cap_item.get_ref()) stores the RefItem pointer 7.
• Table — Same pattern: tbl.captions.append(new_cap_item.get_ref()) 8.
• CODE TextElement — Also participates: code_item.captions.append(...) 9.

Layout-detected caption children of a FigureElement cluster (nested at the layout stage, not via the mapping) are added via _add_child_elements(), which adds them as child nodes of the PictureItem but does not append them to pic.captions. These are structurally children, not caption-linked items.

_add_caption_or_footnote() calls out_doc.add_text(parent=parent, ...) — so the caption text node is both a child of the floating item in the document tree and referenced by the captions list.

Key Behaviors and Edge Cases#