Container Image Variants#

Docling-serve publishes hardware-targeted container images so users can deploy the service against the exact PyTorch build for their environment. All variants are built from the same Containerfile using a UV_SYNC_EXTRA_ARGS build argument that selects a different uv dependency group, each backed by a distinct PyTorch package index 1.

Available Images#

The following images are distributed on ghcr.io/docling-project/ and mirrored to quay.io/docling-project/ 2:

CUDA images (-cu128, -cu130) are not tagged latest — only explicit version tags (e.g., v1.18.0) and main are available, because CUDA versions are deprecated over time as PyTorch evolves. Always pin to an explicit version 3.

Dependency Group Wiring#

Each variant maps to a uv dependency group in pyproject.toml. The Makefile UV_SYNC_EXTRA_ARGS flag used per target is:

4

Note: cu126 has a Makefile build target but is not in the distributed images table above. Note that cu124 was previously supported but is commented out in both the Makefile and pyproject.toml, indicating it has been deprecated. All CUDA/ROCm builds are constrained to linux/amd64 5.

Pre-bundled ML Model Weights#

Every image variant bakes ML model weights at build time using docling-tools models download. The default model set is 6:

layout tableformer picture_classifier rapidocr easyocr

Models are stored at DOCLING_SERVE_ARTIFACTS_PATH=/opt/app-root/src/.cache/docling/models 7. This eliminates cold-start model downloads when the container starts.

Known gap: The rocm72 image bakes English and Chinese RapidOCR models but omits the Latin character set (latin_PP-OCRv3_rec_mobile). As of v1.25.0, stricter model validation surfaces this as a task failure rather than silently continuing — affecting users processing non-English Latin-script documents. Workaround: add a post_start hook to download the missing Latin models, or pin to v1.24.0 8.

Building Locally#

Use the Makefile targets to build any variant locally. The CONTAINER_RUNTIME variable defaults to docker but can be overridden for Podman 9. For example:

make docling-serve-rocm72-image # Build ROCm 7.2
make CONTAINER_RUNTIME=podman docling-serve-cpu-image # Build CPU with Podman

ROCm images require extra device mounts and env vars at runtime. The Makefile run-docling-rocm72 target shows the required flags: /dev/kfd, /dev/dri, video/render group-adds, and TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1.

Base Image & Runtime Details#

All variants share the same CentOS Stream 9 (quay.io/sclorg/python-312-c9s:c9s) base image and include 10:

• mimalloc (v3.2.8) preloaded via LD_PRELOAD for reduced memory fragmentation
• OMP_NUM_THREADS=4 set by default
• Service exposed on port 5001, started via docling-serve run

A docling-serve-slim variant (without pre-bundled weights) is planned but not yet available 11.