MLX VLM Engine#

The MLX VLM engine is MinerU's Apple Silicon inference backend for vision-language model document parsing. It wraps mlx-vlm — a third-party library that runs Qwen2-VL and compatible models natively on Apple's MLX framework — to deliver on-device VLM inference without a GPU server.

Platform Requirements#

The engine enforces two hard gates before loading:

1. OS and hardware: macOS 14.0 (Sonoma) or newer, Apple Silicon (arm64). This check is performed by is_mac_os_version_supported("14.0") in mineru/model/vlm/runtime.py and by _mlx_server_error() in the CLI layer.
2. mlx-vlm version: exactly >=0.7.0,<0.8.0. The CLI checks the installed version via importlib.metadata and also verifies that mlx_vlm.server is importable before starting the server 1.

Engine Name Mapping#

mlx is the user-facing engine name. It maps to the internal backend identifier mlx-engine 2. The required modules for the mlx engine are mlx and mlx_vlm 3.

Auto-Selection Behavior#

The MLX engine is never chosen automatically. resolve_vlm_engine() in selector.py treats non-Linux, non-Windows platforms as a llama-cpp default; MLX must be specified explicitly via config or CLI --engine mlx 4.

Similarly, the --engine auto path in vlm_server_cmd() considers only vllm, lmdeploy, and llama-cpp; mlx must be passed explicitly 5.

Runtime Initialization#

When mlx-engine is selected, _create_model() in runtime.py delegates model loading to mineru_vl_utils.mlx_compat.load_mlx_model(model_path) 6. The resulting (model, processor) pair is passed into MinerUClient.

Because MLX models are not thread-safe, the runtime attaches a threading.Lock (_mineru_execution_lock) to the predictor via _maybe_enable_serial_execution(). All inference calls go through predictor_execution_guard (sync) or aio_predictor_execution_guard (async) to serialize access.

VLM Server Mode (mlx-engine as HTTP backend)#

The CLI command mineru-kit vlm-server --engine mlx (or the mineru-openai-server alias) launches mlx_vlm_server.py. This thin wrapper:

1. Resolves/downloads the default model (MINERU_2_5_PRO_2605_1_2B) if --model is not supplied 7.
2. Calls mineru_vl_utils.mlx_compat.prepare_mlx_model_path for any needed model preparation 8.
3. Delegates to mlx_vlm.server.main with forwarded args, defaulting to host=127.0.0.1, port=8080, and max-num-seqs=8 9.

The resulting server is OpenAI-compatible; other MinerU components can connect to it via the http-client backend.

Packaging Gap: mlx-vlm Is Not a Declared Dependency#

mlx-vlm does not appear in pyproject.toml's dependencies, torch, full, or all extras 10. It is a runtime-only requirement that users must install manually:

pip install "mlx-vlm>=0.7.0,<0.8.0"

The version gate is enforced at runtime in _mlx_server_error() and will surface a clear error message if the wrong version (or no version) is installed 1.

Key Files#