MFR Prediction#

MFR (Mathematical Formula Recognition) is MinerU's inference stage that converts detected formula regions (bounding boxes from MFD — Mathematical Formula Detection) into LaTeX strings. It runs inside the pipeline and hybrid backends and is one of the more memory-sensitive stages in the PDF parse workflow.

Model Implementations#

Two model backends are available, selectable via the MINERU_MFR_MODEL environment variable 1. If unset, the default is pp_formulanet_plus_m; the other option is unimernet.

Both classes expose the same predict() / batch_predict() interface and share the same batch-grouping utility 2.

Batch Size Tuning#

The caller in batch_analyze.py sets MFR_BASE_BATCH_SIZE = 16. The actual call is:

batch_size = batch_ratio × MFR_BASE_BATCH_SIZE # default: 1 × 16 = 16

3

Inside FormulaRecognizer.batch_predict, the requested batch size is halved before being handed to the batch grouper — a deliberate memory-risk reduction introduced in 1:

formula_requested_batch_size = max(1, batch_size // 2) # → 8 at defaults

4

UnimernetModel.batch_predict passes batch_size directly to build_mfr_batch_groups without halving 5.

Dynamic Batch Grouping (build_mfr_batch_groups)#

Formula crops are sorted ascending by pixel area before batching, so that similarly-sized images land in the same group. The grouping algorithm in mineru/model/mfr/utils.py works as follows:

1. Effective batch size — clamped to the largest power-of-two ≤ min(requested_size, total_count) 6.
2. Minimum dynamic batch size — max(16, requested_batch_size // 4). Groups smaller than this threshold are not created mid-stream 7.
3. Area-ratio downscaling — For each candidate batch, the mean area is compared against a base mean (the first batch's average). If ratio ≥ 4, the batch size is halved; at ratio ≥ 8 it halves again, etc. — all sizes stay powers of two 8.
4. Finalization — Trailing groups smaller than their predecessor are merged upward; a single oversized group is split into two power-of-two sub-groups 9.

This prevents large formulas from triggering OOM while keeping throughput high on pages with mostly small inline formulas.

Device-Specific Memory Behaviour#

UnimernetModel 10:

• Non-CPU devices (CUDA, MPS, NPU, MUSA): model is cast to float16 to halve activation memory.
• MPS / NPU / MUSA: initialised with attn_implementation="eager" (FlashAttention is not available on these backends).
• Pinned memory & non-blocking transfers: enabled only for CUDA — DataLoader(pin_memory=True) + tensor.to(..., non_blocking=True) 11.

FormulaRecognizer (PP-FormulaNet): PyTorch-based; tensors are moved to the configured device explicitly 12, with torch.no_grad() for all inference 13.

After the MFR stage, batch_analyze.py calls clean_vram(device, vram_threshold=8) to release fragmented GPU memory before the next stage 14.

Inference Locking#

run_mfr_inference() in model_init.py wraps every batch_predict call with PIPELINE_MFR_INFERENCE_LOCK — a threading.RLock() that guards against concurrent access in hybrid mode where pipeline and hybrid backends share the same model object 15.

The lock is disabled by default; enable it with:

MINERU_ENABLE_PIPELINE_INFERENCE_LOCKS=true

16

Known Issues & Operational Notes#

• NPU (Ascend 910B) OOM: Frequent OOM when GPU memory is shared with other processes. The maintainer recommendation is to run MinerU with exclusive device access 17.
• Hang at "Layout Predict: 0%" (also affects MFR stage): Reported on Ascend 910B after upgrading to v3.4.0 in pipeline/hybrid modes; VLM mode works. Workaround: use VLM backend or downgrade 18 19.
• Batch size on CPU: The internal halving in FormulaRecognizer and the area-ratio downscaling are the primary levers for reducing peak memory on CPU deployments where clean_vram has no effect.

Key Source Files#