Block Type Filtering and Extraction Control#

Overview#

MinerU uses a two-stage filtering system to separate page furniture (headers, footers, page numbers, aside text, footnotes) from content blocks before producing markdown output. The gate is a hardcoded enum, NotExtractType, combined with split logic in the pipeline's MagicModel class. Filtered blocks are never silently dropped — they are preserved in discarded_blocks in the intermediate JSON so downstream consumers can still access them.

NotExtractType Enum#

NotExtractType (mineru/utils/enum_class.py, lines 120–134) is a plain Enum whose values are string block-type constants borrowed from BlockType. The full member list is:

Note: HEADER, FOOTER, PAGE_NUMBER, PAGE_FOOTNOTE are VLM-backend block types added in v2.5 1. They appear in NotExtractType because the hybrid backend also routes them. In the pipeline backend, these four types are handled directly by name in __build_return_blocks().

Stage 1 — preproc_blocks vs. discarded_blocks Split#

The first filter runs inside MagicModel.__build_return_blocks() (pipeline_magic_model.py, lines 199–220). It iterates every detected page block and routes by type:

• discarded_blocks: HEADER, FOOTER, PAGE_NUMBER, ASIDE_TEXT, PAGE_FOOTNOTE
• preproc_blocks: everything else (including captions, footnotes, text, titles, images, tables)

ASIDE_TEXT is placed in discarded_blocks here but is not a member of NotExtractType — it is always discarded unconditionally at this stage.

The caller retrieves both lists via magic_model.get_preproc_blocks() and magic_model.get_discarded_blocks(), then packages them into the page's page_info dict via make_page_info_dict() 2. Both appear in the intermediate JSON as preproc_blocks/para_blocks and discarded_blocks respectively 3.

Stage 2 — Markdown Rendering#

After layout analysis merges preproc_blocks into para_blocks, make_blocks_to_markdown() (pipeline_middle_json_mkcontent.py, lines 18–67) dispatches each block by type. Only blocks with an explicitly handled type produce output:

• TEXT, LIST, INDEX, ABSTRACT, REF_TEXT → plain text via merge_para_with_text()
• TITLE → markdown heading prefixed with # × level
• INTERLINE_EQUATION → LaTeX or image path
• IMAGE, TABLE, CHART → skipped in NLP_MD mode; rendered in MM_MD mode
• CODE → always rendered

Blocks whose rendered text is empty after stripping are silently dropped 4. Notably, captions and footnotes (IMAGE_CAPTION, TABLE_CAPTION, etc.) are not dispatched at the top-level here; they are rendered as sub-blocks inside merge_visual_blocks_to_markdown() when the parent IMAGE/TABLE/CHART block is processed 5.

NotExtractType in the Hybrid Backend#

hybrid_analyze.py materialises not_extract_list at module load time 6:

not_extract_list = [item.value for item in NotExtractType]
HYBRID_OCR_DET_TEXT_TYPES = set(not_extract_list)

This list is used in two ways 7:

1. OCR detection gating — _is_hybrid_ocr_det_candidate() checks whether a block's type is in HYBRID_OCR_DET_TEXT_TYPES to decide if OCR line-detection should run on it. The VLM-OCR hybrid path uses a narrower set (HYBRID_VLM_OCR_DET_TEXT_TYPES = only TEXT, TITLE, DOC_TITLE, PARAGRAPH_TITLE) 8.

2. VLM skip list — When OCR is not enabled, not_extract_list is passed to predictor.batch_extract_with_layout() or predictor.batch_two_step_extract() to tell the VLM to skip extraction for these block types 9. When OCR is enabled, None is passed instead so the VLM can process all blocks 10.

Key Source Files#