Overview#

Docling Serve is a FastAPI-based HTTP service that wraps the Docling document conversion engine and exposes it as a scalable, deployable API. It listens on port 5001 1 and is published as a pre-built container image at ghcr.io/docling-project/docling-serve-cpu (plus GPU-enabled variants).

The service is not a thin wrapper — it adds a full async task system with pluggable orchestration backends (Local, RQ, Ray) powered by the docling-jobkit library. Docling Serve owns the HTTP layer, task routing, and status polling; docling-jobkit provides the orchestrators, converter manager, and task/result data models 2.

The container is built from a CentOS Stream 9 base, bundles mimalloc for memory efficiency, and bakes in ML models (layout, tableformer, picture classifier, rapidocr, easyocr) at image-build time via docling-tools models download 3. Models are stored at DOCLING_SERVE_ARTIFACTS_PATH (default /opt/app-root/src/.cache/docling/models) 4.

API Endpoints#

The FastAPI application is defined in docling_serve/app.py. All conversion endpoints accept conversion options (pipeline preset, OCR settings, output formats, etc.) in the request body.

All endpoints depend on a BaseOrchestrator instance injected via FastAPI dependency injection. The orchestrator backend is selected at startup by DOCLING_SERVE_ENG_KIND and cannot be changed at runtime.

Orchestration Backends#

The orchestration backend is selected via the DOCLING_SERVE_ENG_KIND environment variable (default: local) 5. The factory in docling_serve/orchestrator_factory.py dynamically imports and configures the chosen backend 6, all from docling-jobkit.

DOCLING_SERVE_ENG_KIND=local | rq | ray

Local (default)#

Uses docling_jobkit.orchestrators.local.LocalOrchestrator — multiprocessing on a single machine. Key settings:

• DOCLING_SERVE_ENG_LOC_NUM_WORKERS (default: 2) — number of worker processes 7
• DOCLING_SERVE_ENG_LOC_SHARE_MODELS (default: false) — share ML models across workers 8

Suitable for development and single-node deployments. No external dependencies required.

RQ (Redis Queue)#

Uses docling_jobkit.orchestrators.rq.RQOrchestrator — distributed job queue backed by Redis. The API pod pushes tasks to a Redis queue; one or more worker pods consume the queue. Key settings 9:

• DOCLING_SERVE_ENG_RQ_REDIS_URL (required) — Redis connection URL
• DOCLING_SERVE_ENG_RQ_QUEUE_NAME (default: convert)
• DOCLING_SERVE_ENG_RQ_RESULTS_TTL / DOCLING_SERVE_ENG_RQ_FAILURE_TTL (default: 4 hours each)
• DOCLING_SERVE_ENG_RQ_REDIS_MAX_CONNECTIONS (default: 50)

Workers are started with docling-serve rq-worker 10.

Known issue (as of v1.24.0): The RQ engine passes HTTP source URLs directly to Docling without pre-fetching them via jobkit. This means unreachable URLs silently report task_status: "success" with empty errors arrays, unlike the Ray engine which materializes sources first and retries transient failures. See issue #648 11.

Ray#

Uses docling_jobkit.orchestrators.ray.RayOrchestrator — distributed orchestration via Ray Core with fair round-robin scheduling, autoscaling, per-tenant resource limits, and fault tolerance. This is the most advanced backend. Key settings 12:

• DOCLING_SERVE_ENG_RAY_ADDRESS (required) — Ray cluster address ("auto", "local", or a cluster URL)
• DOCLING_SERVE_ENG_RAY_REDIS_URL (required) — Redis for task state and pub/sub
• DOCLING_SERVE_ENG_RAY_MIN_ACTORS / DOCLING_SERVE_ENG_RAY_MAX_ACTORS — autoscaling bounds 13
• DOCLING_SERVE_ENG_RAY_MAX_CONCURRENT_TASKS (default: 5) — per-tenant concurrency cap 14
• DOCLING_SERVE_ENG_RAY_TENANT_ID_HEADER (default: X-Tenant-Id) — header for multi-tenant routing 15
• Timeout settings: DOCLING_SERVE_ENG_RAY_TASK_TIMEOUT, DOCLING_SERVE_ENG_RAY_DOCUMENT_TIMEOUT, DOCLING_SERVE_ENG_RAY_DISPATCHER_RPC_TIMEOUT 16

Validation at startup enforces that RQ requires eng_rq_redis_url and Ray requires both eng_ray_redis_url and eng_ray_address 17.

Deployment Configurations#

All deployment manifests live under docs/deploy-examples/ and are documented in docs/deployment.md 18.

Docker Compose (local GPU)#

Both expose port 5001 and are started with docker compose -f <manifest> up -d 19.

Kubernetes / OpenShift#

The RQ workers manifest deploys three separate Kubernetes workloads: the API server (1 replica), RQ worker pods (2 replicas by default), and Redis 20. Redis credentials are stored in a Kubernetes Secret and injected as DOCLING_SERVE_ENG_RQ_REDIS_URL and REDIS_PASSWORD 21.

The sticky sessions configuration is needed when using the local engine with multiple replicas, because async task state is held in-process — requests for a given task_id must reach the same pod 22.

The oauth-proxy variant secures the service with TLS encryption between all components and authentication via OpenShift's built-in OAuth server 23.

Docling-Jobkit Integration#

docling-jobkit is the orchestration library Docling Serve depends on, declared as docling-jobkit[rq,ray,vlm]>=2.0.0,<3.0.0 24.

What jobkit provides to docling-serve:

• BaseOrchestrator — abstract interface for all three backends
• LocalOrchestrator, RQOrchestrator, RayOrchestrator — backend implementations
• DoclingConverterManager / DoclingConverterManagerConfig — manages the Docling pipeline (VLM, OCR, layout, table structure, picture classification presets)
• Task, TaskSource — task data model
• StoredSuccessOutcome, StoredFailureOutcome — result data model
• S3PresignedConfig — S3 source/target connector configuration

The orchestrator_factory.py _build_cm_config() function maps all Docling Serve settings to jobkit's DoclingConverterManagerConfig 25, controlling VLM pipeline, picture description, code/formula handling, and OCR engine presets. S3 presigned URL configuration is built and injected here as well 26.

Jobkit also ships standalone CLIs (docling-jobkit-local, docling-jobkit-multiproc) for batch document processing without the HTTP service layer 27.