Multi-Column Layout Processing#

Multi-column PDFs represent one of the most challenging document types for MinerU. Problems arise at multiple stages of the pipeline — layout detection, footer boundary classification, and reading order sorting — each with distinct failure modes and backend-specific trade-offs. Understanding where each failure originates is essential for choosing the right backend or workaround.

The core layout model is PP-DocLayoutV2 (pp_doclayoutv2.py), an RT-DETR-based object detector used by both the Pipeline and Hybrid backends. Any region it fails to detect is permanently lost from all downstream steps (OCR, formula recognition, reading order) 1.

Known Failure Modes#

1. Footer Boundary Detection Swallows Column Body Text#

Symptom: Body text in one column is misclassified as footer when a real footer exists in the adjacent column at the same vertical position.

Root cause: The footer relabeling logic in _post_process_object_detection reclassifies any box whose top Y-coordinate is ≥ the footer anchor's top Y-coordinate and falls within the footer's horizontal scope. The horizontal scope check (_is_footer_x_scope) applies a full-width threshold of 0.7: if the detected footer spans ≥70% of the page width, all content below it is swept into footer classification regardless of column. A narrow, left-column-only footer can still trigger this if the threshold is not met, but a wide footer will always affect the entire row.

Affected scenario: Two-column academic PDFs where the left column ends with a footer/footnote and the right column continues with body text at the same vertical level — confirmed in a clinical trial PDF with a left-side footer misclassifying the right-column "Introduction" text (issue #5217).

Status: Fixed in PR #5219 2.

Workaround (pre-fix): Use --backend hybrid-engine --effort high. Content misclassified as footer is preserved in paras_of_discarded inside middle.json and can be manually recovered 2.

2. PP-DocLayoutV2 Misses Text Blocks in Dense Multi-Column Layouts#

Symptom: Entire text records are absent from the output with no gap visible in middle.json's para_blocks. Reproducible regardless of --mode txt or --effort level.

Root cause: PP-DocLayoutV2 uses per-class confidence thresholds — text and paragraph_title classes use the lowest threshold of 0.4 1. In dense, tightly-packed multi-record layouts (e.g., Chinese university admissions tables), the model produces confidence scores below this floor for certain regions, so no bounding box is generated. Because layout detection is the entry point for all downstream processing, the content is unrecoverable.

Scope: 5+ records missed across 20 sampled pages in a 600-page document; dense military academy and directed-recruitment sections are especially affected 1.

Workaround: Switch to VLM backend (-b vlm-engine) or hybrid-engine --effort high. The VLM backend performs layout detection end-to-end inside the vision-language model, bypassing PP-DocLayoutV2 entirely 1.

3. Cross-Column Paragraph Merging Failure in VLM Mode#

Symptom: Paragraphs that span across columns or page boundaries are not merged; each column segment appears as an independent block.

Root cause: The merge_prev flag in model version 2605 was corrupted during training, disabling cross-column and cross-page paragraph joining in VLM mode 3.

Recommendation: Use hybrid-engine instead of VLM. Hybrid mode uses line detection information to support cross-column and cross-page merging. The online SaaS already defaults to hybrid 3.

Reading Order: XY-Cut++ Sorter#

PP-DocLayoutV2 predicts reading order via a transformer decoder with bidirectional attention masks 1. For PPTX documents (not PDFs), MinerU also ships a standalone geometric fallback: the XY-Cut++ sorter (mineru/model/pptx/xycut_pp_sorter.py), based on arXiv:2504.10258.

The algorithm works in four steps 4:

1. Pre-mask cross-layout elements — identifies blocks wider than beta × max_width (default beta=2.0) that horizontally overlap ≥2 other blocks (e.g., page-spanning headers/footers).
2. Compute density ratio — decides whether to prefer horizontal or vertical cuts first.
3. Recursive XY/YX-Cut — recursively splits the region by the largest horizontal or vertical gap ≥ MIN_GAP_THRESHOLD=5.0 pixels; falls back to top-then-left sort when no valid cut exists.
4. Re-inject cross-layout elements by Y-position into the sorted result.

Limitation: The XY-Cut++ sorter is a purely geometric algorithm — it has no semantic type awareness. If column gaps fall below MIN_GAP_THRESHOLD or blocks span across columns without a clean cut, the fallback _sort_by_y_then_x is used, which reads blocks top-to-bottom ignoring column boundaries and produces wrong reading order for multi-column text.

Backend Selection Guide#

Decision rule: For most multi-column documents, start with hybrid-engine --effort high. Fall back to VLM only if hybrid still misses regions and you can tolerate the speed penalty and the paragraph-merging limitation.

Debugging Layout Detection Failures#

1. Isolate the page — rerun with -s <page_index> -e <page_index> to get a clean intermediate file.
2. Inspect middle.json — look for vertical gaps in para_blocks bbox coordinates where content is expected 1.
3. Check paras_of_discarded — footer-misclassified content appears here and can be manually recovered 2.
4. Verify the text layer — call mineru.utils.pdf_text_tool.get_page_chars(). If characters exist in the PDF text layer but are absent from para_blocks, the layout model is the culprit 1.
5. Rule out OCR — rerun with --mode txt. If records are still missing, the failure is definitively in the Layout Detect phase, not OCR 1.

Threshold tuning: DEFAULT_CLASS_THRESHOLDS are registered as adjustable tensor buffers. Lowering the text class threshold below 0.4 may recover borderline detections at the cost of more false positives 1.

Key Source Files#