vLLM Configuration in MinerU#

MinerU's VLM backend (-b vlm-engine) supports two vLLM inference modes: vllm-engine (synchronous vllm.LLM) and vllm-async-engine (asynchronous vllm.AsyncLLM). Both are initialized inside ModelSingleton.get_model(), which applies device-specific overrides, sets conservative defaults, and passes all remaining kwargs directly to the vLLM engine.

Key Source Files#

Initialization Flow#

When ModelSingleton.get_model() runs for a vLLM backend 1:

1. Strip non-vLLM kwargs — batch_size, max_concurrency, http_timeout, server_headers, max_retries, retry_backoff_factor are removed before reaching vLLM 2.
2. Apply device-type overrides — mod_kwargs_by_device_type() injects hardware-specific params driven by the MINERU_VLLM_DEVICE env var.
3. Set defaults — gpu_memory_utilization and model path are injected if absent 3.
4. Inject logits processors — enable_custom_logits_processors() conditionally adds MinerULogitsProcessor 4.
5. Forward to vLLM — remaining kwargs go directly to vllm.LLM(**kwargs) or AsyncEngineArgs(**kwargs) 5.

Device-Specific Parameters (MINERU_VLLM_DEVICE)#

_get_device_config() maps the MINERU_VLLM_DEVICE environment variable to a config dict applied via mod_kwargs_by_device_type(). Overrides are only injected if the caller has not already set the key.

In server mode (vllm_server.py), False boolean overrides for enable-chunked-prefill and enable-prefix-caching are translated to --no-<flag> CLI arguments 6.

For async_engine, compilation_config is converted from a dict to a CompilationConfig object 7; for sync_engine it is passed as a plain dict.

GPU Memory Utilization Default#

set_default_gpu_memory_utilization() returns 8:

• 0.5 — default for most configurations
• 0.7 — when vLLM ≥ 0.11.0 and VRAM ≤ 8 GB

Applied to all vLLM backends and vllm_server.py unless the caller supplies gpu_memory_utilization explicitly 9.

Prefix Caching and Sampling Determinism#

Prefix caching is on by default in vLLM and MinerU does not disable it globally — only the kxpu device config opts out 10.

Known determinism issue: In long-lived API deployments using vLLM 0.11.0 V1 engine on NVIDIA GPU, the first request for a given PDF returns correct output, but subsequent identical requests produce inconsistent results (e.g., missing images in table HTML). Disabling prefix caching restores per-request consistency 11.

MinerU does not set vLLM sampling parameters (temperature, seed, use_beam_search, etc.) — these are fully delegated to vLLM's defaults.

Custom Logits Processors#

enable_custom_logits_processors() enables MinerULogitsProcessor when all of the following hold 12:

• A CUDA/NPU/GCU/MUSA/MLU/SDAA device is available
• VLLM_USE_V1 != 0
• vLLM ≥ 0.10.1
• Compute capability ≥ 8.0, or vLLM ≥ 0.10.2 (which relaxes the CC requirement)

Set VLLM_USE_V1=0 to force-disable custom logits processors.

Passing Custom kwargs to vLLM#

Any valid vllm.LLM constructor parameter can be forwarded through three entry points:

• Python API: Pass kwargs to ModelSingleton().get_model(), doc_analyze(), or aio_doc_analyze() 13.
• CLI / FastAPI: Unknown flags (e.g., --enable-prefix-caching false, --gpu-memory-utilization 0.8) are parsed by parse_unknown_args(), which normalizes dashes to underscores and coerces values to bool | int | float | str.
• vLLM server: vllm_server.py intercepts --model, --port, --gpu-memory-utilization, and --logits-processors, then passes everything else to vllm serve.

Known Issues#