Large Document Memory Management#

Processing large (500+ page) PDFs in MinerU's hybrid-engine and hybrid-auto-engine modes exposes two distinct memory problems that compound under continuous operation: PyTorch GPU/NPU memory fragmentation and ModelSingleton-induced memory leaks. Both cause OOM, but through different mechanisms and require different mitigations.

Root Causes#

1. PyTorch Memory Fragmentation#

Fragmentation — not actual exhaustion — is the primary OOM trigger during large-file processing. PyTorch reserves memory in blocks; after many alloc/free cycles across layout, MFR, and OCR inference stages, reserved memory far exceeds currently allocated memory, leaving no contiguous free block for the next allocation.

A representative error from a 60.96 GiB Ascend 910B NPU 1:

RuntimeError: NPU out of memory. Tried to allocate 54.00 MiB
(total capacity: 60.96 GiB; allocated: 508.55 MiB; free: 11.84 MiB; reserved: 566.00 MiB)
If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.

The clean_vram() utility in model_utils.py calls device-specific empty_cache() and gc.collect(), but only fires when total VRAM ≤ 8 GB 2. On high-VRAM devices (≥32 GB), this cleanup is skipped entirely unless MINERU_VIRTUAL_VRAM_SIZE overrides the threshold 3. clean_vram is called after layout inference, MFR inference, and OCR-det batches in batch_analyze.py 4 5 6.

2. ModelSingleton Memory Leak#

MinerU caches every loaded model in long-lived singleton dictionaries that are never evicted:

• AtomModelSingleton — Layout, MFR, OCR, Table models, keyed by model parameters
• HybridModelSingleton — MineruHybridModel instances
• Pipeline ModelSingleton in pipeline_analyze.py — keyed by language + formula/table flags

clean_memory() and torch.cuda.empty_cache() cannot free tensors still referenced by these singletons. Under continuous document processing, RAM and VRAM grow without bound until the process is killed 7 8. The maintainer has marked root-cause architectural restructuring as "no plan" 9.

3. Memory Cost Scaling#

Peak memory scales with the product MINERU_API_MAX_CONCURRENT_REQUESTS × MINERU_PROCESSING_WINDOW_SIZE. The default configuration (3 concurrent × 64-page window) is sized for ~32 GB RAM. A configuration of 10 × 128 demands ~7× more memory 7.

Mitigations#

Environment Variable Tuning (Immediate)#

For a 500+ page document on a high-VRAM NPU/GPU, the recommended starting combination is 1:

export MINERU_HYBRID_BATCH_RATIO=1
export MINERU_PROCESSING_WINDOW_SIZE=4
export PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:128
export MINERU_VIRTUAL_VRAM_SIZE=8

Operational / Architectural Workarounds#

Periodic service restart is the most reliable leak mitigation since singleton models accumulate across requests. A daily cron restart of mineru-api clears all held model references 8.

Multi-instance isolation: Deploy multiple mineru-api instances, each with MINERU_API_MAX_CONCURRENT_REQUESTS=1, behind mineru-router. Isolating each request to its own process bounds leak accumulation to a single document's lifetime 8.

Separate VLM process: Run mineru-vllm-server as an independent process so the heavy VLM inference engine can be restarted independently of the main API process 8. The VLM singleton (vlm_analyze.py) includes atexit shutdown hooks 11, unlike pipeline/hybrid singletons.

Effort level reduction: effort=medium disables image/chart analysis stages, substantially reducing peak memory for documents with many visual elements 1.

Device exclusivity: Hybrid-engine mode requires the GPU/NPU to be exclusively dedicated to MinerU; shared device access with other processes exacerbates fragmentation 12.

Related Issues & References#

Key source files:

• mineru/utils/model_utils.py — clean_vram, clean_memory, get_vram
• mineru/backend/pipeline/model_init.py — AtomModelSingleton, HybridModelSingleton
• mineru/backend/pipeline/pipeline_analyze.py — pipeline ModelSingleton, batch ratio logic
• mineru/backend/pipeline/batch_analyze.py — clean_vram call sites