List Marker Normalization#

List marker normalization is the process of detecting hierarchical or compound list-item markers — such as 9a., 3.a., 1.2.1, or 5.1 — and moving them out of the item's text field into the dedicated marker field of a ListItem, while setting enumerated = True. When this step is missed, the marker stays fused in the item text, producing doubled output in Markdown and other exports (e.g., 9a. 9a. Algorithm step).

This concern spans two independent pipelines:

The ListItem data model 1 stores normalized markers in the marker field alongside the text and enumerated flag 2.

PDF Pipeline: ListItemMarkerProcessor#

The primary rule-based classifier for PDF list markers lives in docling_ibm_models/list_item_normalizer/list_marker_processor.py. The ListItemMarkerProcessor class 3 recognizes two families of patterns compiled at init time 4:

• Bullet patterns — Unicode and ASCII bullet symbols 5
• Numbered patterns — simple single-level formats: \d+\., \d+\), (1), [1], Roman numerals, single letters a./A. 6

Gap: none of these patterns cover compound markers like 9a., 3.a., or 1.2.1. Items with such markers pass through undetected, leaving the marker text fused in ListItem.text. A development branch (feat/compound-list-item-markers) in a fork of docling-ibm-models adds the missing patterns, and PR #3849 temporarily pins Docling's dependency to that branch while the fix is upstreamed.

The processor's main entry point is process_document() 7, which calls update_list_items_in_place() 8 and optionally merge_markers_and_text_items_into_list_items() 9 to combine standalone marker-only TextItems with their following content items.

PDF Pipeline: Compound Marker Recovery#

PR #3815 proposed an optional ListNormalizationModel post-processing stage to recover compound markers that slip past ListItemMarkerProcessor. It was closed in favor of fixing the patterns upstream (PR #3849), but its design documents the normalization logic:

• Pattern: _COMPOUND_ORDERED regex captures dotted-decimal (1.1, 1.1.1) and digit+letter (9a, 9a., 3.a) prefixes at the start of ListItem.text 10.
• Conservative detection: only processes items whose marker field is empty (already-handled items are untouched). Triggers only in a numbered context — i.e., when the enclosing list group already has a recognized numbered marker or at least two sibling items match compound patterns 10.
• Normalization: sets it.marker to the extracted prefix, strips the marker from it.text, and sets it.enumerated = True 10.
• Opt-in: would have been enabled via PdfPipelineOptions().list_normalization_options.enabled = True 10.

The preferred resolution (PR #3849) adds compound patterns directly to ListItemMarkerProcessor in docling-ibm-models, avoiding post-processing overhead. Ground-truth impact from that change: markers like 3.a., 9a.–9d. and dotted-decimal section identifiers (5.1, 5.2) move from item text into the marker field across affected documents 11.

DOCX Pipeline: Multi-Level Marker Construction#

The DOCX backend (docling/backend/msword_backend.py) builds hierarchical markers natively from Word's numbering.xml, without a separate normalization stage. PR #2815 introduced the full multi-level support.

Numbering resolution (_get_numId_and_ilvl()): reads <w:numPr> directly from the paragraph XML; if absent, walks the paragraph's style inheritance chain to find an inherited numPr. Defaults ilvl to 0 when numId is found but ilvl is unspecified 12.

Abstract numbering lookup (_get_level_element()): given (numId, ilvl), walks numbering.xml through the <w:num> → <w:abstractNumId> → <w:abstractNum> → <w:lvl> chain to retrieve the level element 13.

Counter tracking (_get_list_counter()): maintains a dict[tuple[int,int], int] keyed by (numId, ilvl). When a parent level counter advances, all sub-level counters for that numId are reset 14.

Marker rendering (_build_enum_marker()): reads the w:lvlText template (e.g., "%1.%2") and substitutes %N placeholders with actual counter values. Falls back to a dotted-decimal 1.2.3. pattern for plain numeric formats 15. Templates with non-trivial prefix/suffix text (e.g., "Proposal %1:") are rendered verbatim 16.

Key Source References#