Long-Running Service Stability#
mineru-api deployments processing documents continuously encounter three distinct stability failure modes over time: ModelSingleton memory leaks, GPU/NPU memory fragmentation, and vLLM prefix caching non-determinism. These are distinct from single-document memory issues — they compound across requests and worsen as the process runs longer.
1. ModelSingleton Memory Leak#
The most pervasive long-running issue. MinerU caches all model instances in long-lived singleton dictionaries that are never evicted across requests:
AtomModelSingleton— layout, MFR, OCR, Table modelsHybridModelSingleton—MineruHybridModelinstances- Pipeline
ModelSingletoninpipeline_analyze.py
Because these singletons hold live tensor references, clean_memory() and torch.cuda.empty_cache() cannot free them. RAM and VRAM grow without bound until the process is OOM-killed. This was reproduced reliably on a 128 GB system within ~2 hours under continuous load . The maintainer has marked root-cause architectural restructuring as "no plan" .
Aggravating factor: Peak memory scales with MINERU_API_MAX_CONCURRENT_REQUESTS × MINERU_PROCESSING_WINDOW_SIZE. The default (3 × 64) is sized for ~32 GB RAM. Configurations like 10 × 128 demand ~7× more memory and exhaust it correspondingly faster .
2. GPU/NPU Memory Fragmentation#
After many alloc/free cycles across layout, MFR, and OCR inference stages, PyTorch's allocator reserves memory in blocks, leaving the device with reserved >> allocated memory and no contiguous free block for the next inference call. This appears as OOM even though most VRAM is nominally "free":
RuntimeError: NPU out of memory. Tried to allocate 54.00 MiB
(total capacity: 60.96 GiB; allocated: 508.55 MiB; free: 11.84 MiB; reserved: 566.00 MiB)
If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
This error appears on Ascend 910B after processing tens of files sequentially . The clean_vram() utility that calls empty_cache() only fires when detected VRAM ≤ 8 GB — meaning high-VRAM devices (≥32 GB) skip cache cleanup entirely by default . Additionally, hybrid engine mode requires exclusive GPU/NPU access; sharing the device with other processes exacerbates fragmentation .
3. Prefix Caching Non-Determinism (VLM Mode)#
In mineru-api deployments using the vlm backend with vLLM 0.11.0 V1 engine on NVIDIA GPU, the first request after a service restart returns correct output, but all subsequent identical requests produce inconsistent results — e.g., missing images in table HTML output .
The cause is vLLM's prefix caching, which is enabled by default in MinerU for all device types except kxpu . The cache returns stale or reused KV states for subsequent requests on the same PDF. Disabling prefix caching restores per-request consistency .
Mitigations#
Operational (Highest Priority)#
| Mitigation | How |
|---|---|
| Periodic service restart | Daily cron systemctl restart mineru-api clears all singleton model references |
| Per-request process isolation | Deploy multiple mineru-api instances with MINERU_API_MAX_CONCURRENT_REQUESTS=1 behind mineru-router; bounds leak accumulation to a single document |
| Separate VLM process | Run mineru-vllm-server independently so the VLM engine can be restarted without restarting the main API |
| Exclusive device access | Dedicate the GPU/NPU entirely to MinerU; do not share with other processes |
Environment Variables#
| Variable | Value | Effect |
|---|---|---|
MINERU_VIRTUAL_VRAM_SIZE | 8 | Forces clean_vram() to fire after every inference stage |
MINERU_HYBRID_BATCH_RATIO | 1 | Reduces peak VRAM per inference stage on high-VRAM devices |
PYTORCH_NPU_ALLOC_CONF | max_split_size_mb:128 | Reduces allocator block granularity to limit NPU fragmentation |
MINERU_PROCESSING_WINDOW_SIZE | 4–8 | Reduces per-window memory footprint |
MINERU_PDF_RENDER_THREADS | 1 | Limits concurrent PIL image loads that compound RAM usage |
Disabling Prefix Caching#
Pass --enable-prefix-caching false to mineru-api (or set enable_prefix_caching=False via the Python API). The parse_unknown_args() function normalizes these flags and forwards them to vLLM . The vLLM config in utils.py translates enable_prefix_caching=False to the --no-enable-prefix-caching server flag .
Related Issues & Source Files#
| Issue | Topic |
|---|---|
| #5013 | ModelSingleton leak confirmed in v3.1.15+ hybrid-auto-engine |
| #5265 | Prefix caching non-determinism (vLLM 0.11.0, NVIDIA A10G) |
| #5224 | NPU fragmentation OOM after processing dozens of files |
| #5185 | Ascend 910B OOM on sequential file processing |
Key source files:
mineru/backend/pipeline/model_init.py—AtomModelSingleton,HybridModelSingleton, inference lockmineru/utils/model_utils.py—clean_vram,clean_memory,get_vrammineru/backend/vlm/utils.py— device config, prefix caching defaultsmineru/utils/cli_parser.py— CLI flag forwarding to vLLM