MinerU Equation Extraction#

MinerU extracts two equation types — block-level (interline/display) and inline — producing LaTeX strings stored in middle.json. The mechanism differs significantly across backends:

VLM mode is actively being deprecated in favor of hybrid mode — see VLM Mode Deprecation 1.

Block Type Architecture#

All equation-related type constants are in mineru/utils/enum_class.py.

BlockType (block-level structure):

• INTERLINE_EQUATION = 'interline_equation' — display-mode (block) formula used by pipeline/hybrid
• EQUATION = 'equation' — standalone formula type emitted by the VLM backend; mapped to INTERLINE_EQUATION during processing 2

ContentType (span-level, inside lines):

• INLINE_EQUATION = 'inline_equation' — inline formula span
• INTERLINE_EQUATION = 'interline_equation' — display formula span
• EQUATION = 'equation' — legacy span type

ContentTypeV2 (used in content_list_v2.json output):

• EQUATION_INTERLINE = 'equation_interline'
• SPAN_EQUATION_INLINE = 'equation_inline'

In middle.json, display equations are indexed at the page level in interline_equations, while inline equations appear as spans (type inline_equation) embedded within text block lines 3.

Pipeline and Hybrid Backends: MFD → MFR Two-Stage Pipeline#

Both pipeline and hybrid backends use a two-stage process:

Stage 1 — MFD (Mathematical Formula Detection): PP-DocLayoutV2 detects display_formula and inline_formula regions on each page image. These are the only two formula-class labels in its 25-class schema 4. Detected bounding boxes are collected with an empty "latex" field placeholder.

Stage 2 — MFR (Mathematical Formula Recognition): The MFR model converts formula crops to LaTeX strings. The default model is pp_formulanet_plus_m; the alternative is unimernet. Selectable via MINERU_MFR_MODEL 5. The LaTeX output is written back into each detection's "latex" field, then propagated to span "content" in middle.json.

Formula number handling: Standalone formula_number blocks (e.g., (1), (2)) detected by PP-DocLayoutV2 are matched to adjacent display equations and appended as \tag{} to the equation's LaTeX content 6.

Inline equations in text: Inline formula bboxes from MFD are stored separately as page_inline_formula. During OCR, these regions are masked so character recognition is skipped; the MFR-generated LaTeX is used instead. The resulting spans appear as ContentType.INLINE_EQUATION within text block lines.

Entry points: batch_analyze.py (pipeline), hybrid_analyze.py (hybrid).

VLM Backend: Equation Handling#

The VLM backend processes layout and formula extraction in a single end-to-end model inference pass (Qwen2-VL / MinerU2.5-Pro). There is no separate MFD or MFR stage.

Block-level (interline) equations: The VLM emits blocks with "type": "equation". In vlm_magic_model.py, this is remapped to BlockType.INTERLINE_EQUATION and the span type is set to ContentType.INTERLINE_EQUATION. The "content" field is cleaned via isolated_formula_clean().

Inline equations: The VLM does not emit a distinct inline_equation block type. Instead, inline formulas are embedded in text block content using \( ... \) delimiters. vlm_magic_model.py parses these with a regex, splitting the content string into alternating ContentType.TEXT and ContentType.INLINE_EQUATION spans within the same line. This splitting is only triggered when \( and \) counts are balanced 7.

VLM-recognized block types include equation for display formulas; there is no explicit inline_formula block — inline math is always embedded in text content 8.

VLM Mode Deprecation and Hybrid Mode#

VLM mode (-b vlm-engine) is actively being deprecated in favor of hybrid mode (-b hybrid-engine / hybrid-auto-engine) 1.

Key reasons affecting equation extraction:

• The merge_prev marker in VLM model v2605 was corrupted during training, breaking cross-column and cross-page paragraph (and equation) merging in VLM mode 1.
• Hybrid mode uses PP-DocLayoutV2 for layout detection (including formula region detection) and applies line-detection information to support cross-column/cross-page merging — critical for multi-line display equations.
• The online SaaS service at opendatalab.com already uses hybrid mode internally 1.

Migration: Replace -b vlm-engine with -b hybrid-engine. Hardware requirements are identical 1.

For hallucination-free equation extraction, use pipeline mode — the MFR model is purely recognition-based and cannot hallucinate content the way a generative VLM can 1.

Key Source Files#