Long-Running Service Stability#

mineru-api deployments processing documents continuously encounter three distinct stability failure modes over time: ModelSingleton memory leaks, GPU/NPU memory fragmentation, and vLLM prefix caching non-determinism. These are distinct from single-document memory issues — they compound across requests and worsen as the process runs longer.

1. ModelSingleton Memory Leak#

The most pervasive long-running issue. MinerU caches all model instances in long-lived singleton dictionaries that are never evicted across requests:

• AtomModelSingleton — layout, MFR, OCR, Table models
• HybridModelSingleton — MineruHybridModel instances
• Pipeline ModelSingleton in pipeline_analyze.py

Because these singletons hold live tensor references, clean_memory() and torch.cuda.empty_cache() cannot free them. RAM and VRAM grow without bound until the process is OOM-killed. This was reproduced reliably on a 128 GB system within ~2 hours under continuous load 1. The maintainer has marked root-cause architectural restructuring as "no plan" 1.

Aggravating factor: Peak memory scales with MINERU_API_MAX_CONCURRENT_REQUESTS × MINERU_PROCESSING_WINDOW_SIZE. The default (3 × 64) is sized for ~32 GB RAM. Configurations like 10 × 128 demand ~7× more memory and exhaust it correspondingly faster 1.

2. GPU/NPU Memory Fragmentation#

After many alloc/free cycles across layout, MFR, and OCR inference stages, PyTorch's allocator reserves memory in blocks, leaving the device with reserved >> allocated memory and no contiguous free block for the next inference call. This appears as OOM even though most VRAM is nominally "free":

RuntimeError: NPU out of memory. Tried to allocate 54.00 MiB
(total capacity: 60.96 GiB; allocated: 508.55 MiB; free: 11.84 MiB; reserved: 566.00 MiB)
If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.

This error appears on Ascend 910B after processing tens of files sequentially 2 3. The clean_vram() utility that calls empty_cache() only fires when detected VRAM ≤ 8 GB — meaning high-VRAM devices (≥32 GB) skip cache cleanup entirely by default 4. Additionally, hybrid engine mode requires exclusive GPU/NPU access; sharing the device with other processes exacerbates fragmentation 3.

3. Prefix Caching Non-Determinism (VLM Mode)#

In mineru-api deployments using the vlm backend with vLLM 0.11.0 V1 engine on NVIDIA GPU, the first request after a service restart returns correct output, but all subsequent identical requests produce inconsistent results — e.g., missing images in table HTML output 5.

The cause is vLLM's prefix caching, which is enabled by default in MinerU for all device types except kxpu 6. The cache returns stale or reused KV states for subsequent requests on the same PDF. Disabling prefix caching restores per-request consistency 5.

Mitigations#

Operational (Highest Priority)#

Environment Variables#

Disabling Prefix Caching#

Pass --enable-prefix-caching false to mineru-api (or set enable_prefix_caching=False via the Python API). The parse_unknown_args() function normalizes these flags and forwards them to vLLM 7. The vLLM config in utils.py translates enable_prefix_caching=False to the --no-enable-prefix-caching server flag 8.

Related Issues & Source Files#

Key source files:

• mineru/backend/pipeline/model_init.py — AtomModelSingleton, HybridModelSingleton, inference lock
• mineru/utils/model_utils.py — clean_vram, clean_memory, get_vram
• mineru/backend/vlm/utils.py — device config, prefix caching defaults
• mineru/utils/cli_parser.py — CLI flag forwarding to vLLM