Apple Silicon Support in Docling#

Docling runs on Apple Silicon (M-series) Macs using two distinct hardware acceleration paths:

• MPS (Metal Performance Shaders): PyTorch's mps device, used for layout detection and some image-classification models.
• MLX: Apple's native ML framework (mlx-vlm, mlx-whisper), used for VLM inference and ASR on Apple Silicon. This is the preferred path for VLM workloads on macOS.

The decide_device() function in accelerator_utils.py resolves which device to use. In AUTO mode, it probes CUDA → MPS → XPU → CPU in that order, so MPS is selected automatically on any Apple Silicon Mac without a CUDA GPU 1. Individual models can restrict which devices they accept via a supported_devices list that silently filters unavailable accelerators 2.

Per-Engine MPS/MLX Support Matrix#

MPS support is deliberately inconsistent across Docling's inference components:

Key point: For VLM workloads (formula/code enrichment, VLM-based table structure, caption enrichment), Docling uses MLX rather than raw MPS. The Transformers VLM engine explicitly refuses to run on MPS — if you force device=mps without an MLX model export present, the auto-inline engine logs a warning and falls back to the Transformers engine on CPU 11.

MLX Engine Details#

VLM (vision-language models): The MlxVlmEngine wraps mlx-vlm for Apple Silicon. All MLX model operations share a _MLX_GLOBAL_LOCK because MLX is not thread-safe 12. Engine selection happens in auto_inline_engine.py: on macOS with MPS detected, the auto-inline engine checks model_spec.has_explicit_engine_export(VlmEngineType.MLX); if the model declares an MLX export and mlx-vlm is importable, VlmEngineType.MLX is selected.

Models with explicit MLX support include the granite_docling preset, which maps VlmEngineType.MLX → ibm-granite/granite-docling-258M-mlx 13. To force MLX explicitly:

from docling.datamodel.vlm_engine_options import MlxVlmEngineOptions
code_formula_options = CodeFormulaVlmOptions.from_preset(
    "codeformulav2",
    engine_options=MlxVlmEngineOptions()
)

13

ASR: mlx-whisper is auto-selected when torch.backends.mps.is_available() and the mlx-whisper package is installed 14. Install with pip install "docling[asr]" for ASR or pip install mlx-vlm for VLM MLX support.

Known Issues and Limitations#

TableFormer V2 decoder-drift on long tables#

TableFormer V2 (TableStructureV2Options) has a documented quality regression affecting long, low-entropy tables. The seq2seq autoregressive decoder loses positional attention after approximately ~190 cells, causing overlapping sliding-window duplicate rows to be emitted. Reported environment: macOS 14, Apple Silicon, MPS 15.

The failure is silent — output is valid, well-formed table markup — but the content is corrupt. A 7,000-word document has been observed bloating to ~42,000 words with ~55% duplicate rows 15. Both do_cell_matching=True and do_cell_matching=False are equally affected.

Workaround: Use TableStructureOptions (V1) for documents with long tables, especially multi-page reference lists, fee schedules, and bibliographies 16.

This is an architectural issue in the decoder — not configuration-fixable — and requires model retraining to resolve.

MPS disabled in TableFormer (V1 and V2)#

Both table structure models override AcceleratorDevice.MPS → CPU at initialization time. The reason is documented only as "until we know why it makes things slower" 4 5. This is a known open issue, not a resolved one.

Transformers VLM engine does not support MPS#

If you manually set device=mps in AcceleratorOptions and use a VLM preset that has no MLX export, the Transformers engine will raise AcceleratorDeviceNotAvailableError because MPS is absent from its supported_devices list 6. Use device=auto to allow graceful fallback.

MLX global lock / thread safety#

MlxVlmEngine and HuggingFaceMlxModel both require a process-wide _MLX_GLOBAL_LOCK. Multi-threaded pipelines will serialize on this lock 12. Avoid using MLX-based VLM or Whisper models in high-concurrency scenarios.

Third-party models with CUDA-only tensors#

Some community VLM/OCR models hardcode .cuda() calls and cannot run on MPS or CPU without upstream changes. These models will fail early on Apple Silicon unless the upstream maintainer removes the CUDA-only assumption 17.

Performance Notes#

• TableFormer V2 + MPS inference: Apple Silicon MPS acceleration provides approximately ~14× speedup over non-accelerated execution for V2 table inference 16. However, the current code forces V2 to CPU anyway (see above), so this benefit is not realized without a code change.
• No published quality benchmarks (TEDS, precision, recall) comparing TableFormer V1 vs V2 exist as of mid-2026 16. V2 is described as "lightweight, optimized for CPU/GPU inference while maintaining high accuracy" but this is unverified.
• OcrMacModel (macOS Vision framework) is automatically preferred on macOS in the OCR auto-selection chain — it uses Apple Silicon natively and is faster than CPU-bound ONNX alternatives on M-series hardware 10.

Key Source Files#