Ascend NPU Support#
Overview#
MinerU supports Ascend 910B NPUs for PDF parsing in pipeline, hybrid-engine, and hybrid-auto-engine modes. The primary operational challenge on 910B is PyTorch memory fragmentation — not actual exhaustion — which causes OOM errors after processing multiple files sequentially. The NPU device is detected via torch_npu , and all batch ratio and VRAM cleanup logic applies uniformly across CUDA, NPU, and other accelerators.
Root Cause: Memory Fragmentation#
After many alloc/free cycles across layout, MFR, and OCR inference stages, PyTorch's NPU allocator reserves memory in blocks. Reserved memory grows far beyond currently-allocated memory, leaving no contiguous free block for the next inference call. A representative error on a 60.96 GiB 910B :
RuntimeError: NPU out of memory. Tried to allocate 54.00 MiB
(NPU 0; 60.96 GiB total capacity; 508.55 MiB already allocated; 11.84 MiB free; 566.00 MiB reserved)
If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
Why high-VRAM devices are worse: The clean_vram() utility calls empty_cache() and gc.collect(), but only fires when detected VRAM ≤ 8 GB. On a 60 GiB 910B, this cleanup is skipped entirely unless overridden . clean_vram is called after layout, MFR, and OCR-det stages in batch_analyze.py.
High batch ratios amplify the problem: The batch ratio logic in pipeline_analyze.py auto-detects VRAM and sets batch_ratio=16 for devices ≥ 32 GiB. On a 60 GiB 910B this is the default, which multiplies the inference batch size 16× and proportionally increases peak VRAM usage per stage.
Device exclusivity required: Hybrid-engine mode requires the 910B to be exclusively dedicated to MinerU. Sharing the device with other processes exacerbates fragmentation .
Quick-Fix: Environment Variables#
Set these before starting mineru-api or invoking the CLI:
| Variable | Value | Effect |
|---|---|---|
PYTORCH_NPU_ALLOC_CONF | max_split_size_mb:128 | Reduces allocator block granularity to limit fragmentation |
MINERU_HYBRID_BATCH_RATIO | 1 | Overrides the auto-detected batch ratio (default 16 on ≥32 GiB devices), reducing peak VRAM per inference stage |
MINERU_VIRTUAL_VRAM_SIZE | 8 | Forces clean_vram() to fire after every inference stage by spoofing a low-VRAM device |
MINERU_PROCESSING_WINDOW_SIZE | 4–8 | Reduces per-window page count, lowering peak per-batch memory |
MINERU_PDF_RENDER_THREADS | 1 | Limits concurrent PIL image loads that compound RAM usage |
Recommended combination for 500+ page documents on 910B :
export MINERU_HYBRID_BATCH_RATIO=1
export MINERU_PROCESSING_WINDOW_SIZE=4
export PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:128
export MINERU_VIRTUAL_VRAM_SIZE=8
Additional Mitigations#
- Reduce effort level:
effort=mediumdisables image/chart analysis stages, substantially reducing peak VRAM for visually-heavy documents . - Disable OCR batching:
export MINERU_LMDEPLOY_DEVICE=corexforces OCR detection to process images one at a time rather than in batches . - Periodic service restart:
ModelSingletoncaches never evict model instances —clean_memory()andtorch.npu.empty_cache()cannot free tensors held byAtomModelSingletonorHybridModelSingleton. A daily cron restart ofmineru-apiclears all held references . - Per-request process isolation: Deploy multiple
mineru-apiinstances withMINERU_API_MAX_CONCURRENT_REQUESTS=1behindmineru-routerto bound memory accumulation to a single document's lifetime .
Known Issues#
- Pipeline/hybrid hang at "Layout Predict: 0%" on v3.x after upgrading from v2.x: reported on BCLinux/aarch64 with 910B, VLM mode works but pipeline and hybrid modes stall . No confirmed fix as of the report date.
- ModelSingleton memory leak compounds fragmentation under continuous document processing; the maintainer has marked architectural restructuring as "no plan" .
Key Source Files#
| File | What's There |
|---|---|
mineru/utils/model_utils.py | clean_vram, get_vram, MINERU_VIRTUAL_VRAM_SIZE handling, NPU device detection |
mineru/backend/pipeline/pipeline_analyze.py | Batch ratio auto-detection logic (lines 353–366), ModelSingleton |
mineru/backend/pipeline/batch_analyze.py | clean_vram call sites after layout, MFR, and OCR stages |
mineru/backend/pipeline/model_init.py | AtomModelSingleton, HybridModelSingleton definitions |