Redis Integration in docling-serve (Ray Backend)#

Redis is a mandatory external dependency when using the Ray orchestration backend in docling-serve. It is not optional or replaceable — startup validation raises an error if DOCLING_SERVE_ENG_RAY_REDIS_URL is not set when DOCLING_SERVE_ENG_KIND=ray 1. Redis serves four distinct roles in this backend: durable task state storage, pub/sub messaging, conversion result storage, and per-tenant fairness controls.

The rq backend also requires Redis (DOCLING_SERVE_ENG_RQ_REDIS_URL), but uses a separate key space. This article covers only the Ray backend, which uses Redis more extensively 2.

What Redis is Used For#

Task State Management#

The Ray orchestrator stores all durable task lifecycle state in Redis — not in actor memory. This means the full queue and in-flight task state survives process restarts. Key data structures (all implemented in docling_jobkit/orchestrators/ray/redis_helper.py):

• Task queues: tenant:{tenant_id}:tasks — Redis Lists (RPUSH/LPOP for FIFO dispatch)
• Active task tracking: tenant:{tenant_id}:active_tasks — Redis Sets (SADD on dispatch, SREM on completion)
• Task metadata & counters: task:{task_id} hash, tenant:{tenant_id}:limits hash, tenant:{tenant_id}:stats hash
• Execution lease: task:{task_id}:execution hash — written when a Ray actor claims a task; used to detect stale/crashed workers
• Dispatch state: task:{task_id}:dispatch — tracks dispatcher ownership with a TTL for lease expiry

Atomic dispatch uses Redis transactions (WATCH/MULTI/EXEC) to prevent race conditions when multiple dispatchers compete for the same task.

Pub/Sub Notifications#

When a task completes (success or failure), the coordinator actor PUBLISHes to the docling:ray:updates channel (configurable via DOCLING_SERVE_ENG_RAY_SUB_CHANNEL) 3. The docling-serve API process subscribes on a dedicated Redis connection (separate from the command connection pool) with TCP keepalive and health-check interval of 15 seconds. This subscription drives the WebSocket streaming and polling endpoints that clients use to receive results.

Result Storage#

Completed conversion results are stored under the key pattern {results_prefix}:task:{task_id}:result using SETEX with a TTL 4. Defaults:

Results are msgpack-encoded StoredSuccessOutcome or StoredFailureOutcome objects. The two-phase expiry (initial TTL → crash-safe EXPIRE after fetch) means results are available for up to 4 hours if never retrieved, or for 5 minutes after first retrieval 5.

Per-Tenant Fairness Controls#

Redis stores per-tenant concurrency state that the dispatcher uses to enforce fair round-robin scheduling. The Ray RayTaskDispatcher actor maintains an in-memory deque of tenant IDs for round-robin ordering, but reads/writes the authoritative limits and counters from Redis 6:

• max_concurrent_tasks per tenant (default: 5, DOCLING_SERVE_ENG_RAY_MAX_CONCURRENT_TASKS)
• max_queued_tasks — optional queue depth cap with rejection (DOCLING_SERVE_ENG_RAY_MAX_QUEUED_TASKS, DOCLING_SERVE_ENG_RAY_ENABLE_QUEUE_LIMIT_REJECTION)
• max_documents — optional per-tenant document count limit (DOCLING_SERVE_ENG_RAY_MAX_DOCUMENTS)
• active_tasks and queued_tasks counters checked atomically before dispatch

The tenant ID is extracted from the X-Tenant-Id request header (configurable via DOCLING_SERVE_ENG_RAY_TENANT_ID_HEADER) 7. Child page slices (when PDF fan-out is enabled) do not increment the tenant's active_tasks counter — only the parent task counts 8.

Configuration Reference#

All Ray-backend Redis settings are defined in docling_serve/settings.py and wired into RayOrchestratorConfig in orchestrator_factory.py.

Connection#

Connection Pool Gate (Concurrency Throttle)#

A "gate" limits the number of concurrent callers accessing Redis to prevent connection exhaustion 9:

Per-Tenant Limits#

See .env.example for annotated examples.

Key Source Files#