vLLM Multimodal Cache (mm_receiver_cache) Bug#

Overview#

A compatibility bug between mineru_vl_utils and vLLM 0.21.0 causes the vlm-engine and hybrid-engine backends to emit malformed layout tokens (]]<|><|><|><|>) instead of valid layout JSON during Layout Detection. Every page returns 0 blocks, making all tasks using those backends fail completely. The vlm-http-client backend is unaffected. 1

Severity: P0 — 100% task failure rate for vlm-engine / hybrid-engine backends on vLLM 0.21.0. 1

Symptoms at a Glance#

Root Cause#

vLLM 0.21.0 introduced mm_receiver_cache (vllm/multimodal/cache.py), which requires that any mm_hash present in a prompt already have its multimodal features stored in EngineCore's cache before generate() is called. 1

The bug is triggered by VllmEngineVlmClient._predict_one_batch() in the external mineru_vl_utils package, which calls _render_vllm_cmpl_inputs() to pre-render prompts on the client side before passing them to LLM.generate(). In vLLM 0.21.0, LLM.renderer.render_cmpl converts a raw {"prompt": str, "multi_modal_data": {"image": [PIL]}} dict into a TokensPrompt containing mm_hashes and mm_placeholders — but the multimodal features are not written into EngineCore's mm_receiver_cache. When LLM.generate() receives the already-rendered TokensPrompt, it skips its own preprocessing, and EngineCore finds a cache miss (mm_features = None). The model sees empty image features and emits truncated/garbled tokens. 1

The call chain:

MinerUClient.batch_two_step_extract
  └── VllmEngineVlmClient._predict_one_batch
        ├── _render_vllm_cmpl_inputs() ← generates mm_hashes, does NOT fill EngineCore cache
        └── vllm_llm.generate(TokensPrompt) ← EngineCore cache miss → mm_features = None → garbled output

By contrast, passing the raw dict directly to LLM.generate() lets vLLM complete renderer + cache + inference in one internally consistent step. 1

Why vlm-http-client is unaffected: The HTTP path sends raw image data to mineru-openai-server, which runs vLLM's full preprocessing server-side in a single call — no client-side pre-rendering, no cache desync. 1

Why hybrid-engine is also affected: hybrid-engine routes its VLM inference step through the same VllmEngineVlmClient code path. 1

Root cause ownership: The VllmEngineVlmClient class lives in the external mineru_vl_utils package 2, not in the MinerU repo. The long-term fix requires an upstream update to that package.

Key Files#

Mitigations#

Option 1 (Recommended): Monkey-patch _predict_one_batch#

Create mineru/utils/vllm_compat_patch.py and call apply_vllm_engine_patch() at the top of vlm_analyze.py (after the MinerUClient import at line 39, before any VllmEngineVlmClient instantiation). The patch replaces _predict_one_batch with a version that skips _render_vllm_cmpl_inputs and passes raw dicts directly to LLM.generate(). 1

Properties of the patch:

• Idempotent — a _mineru_skip_render_patch sentinel prevents double-patching.
• Safe on older vLLM — on vLLM < 0.21.0, _render_vllm_cmpl_inputs was already a no-op; skipping it is equivalent.
• Scoped — only touches VllmEngineVlmClient; vlm-http-client is unaffected.

The patch must be applied before any VllmEngineVlmClient instantiation; vlm_analyze.py is the recommended injection point since all vlm-engine paths import it first. 1

Option 2: Downgrade / Pin vLLM#

Pin vllm==0.10.x. On that version, LLM.renderer.render_cmpl does not exist, so _render_vllm_cmpl_inputs takes the return raw_prompts branch and the cache desync cannot occur. 1

Option 3: Switch to vlm-http-client#

Launch mineru-openai-server separately and configure workers to use vlm-http-client. Confirmed working; trades in-process inference convenience for correctness. 1

Related Issues#

• #5267 — Primary detailed bug report with 27-experiment root cause analysis 1
• #5263 — Earlier symptom report showing identical garbled output 3