Plugin Daemon Communication#

The Dify API communicates with the plugin daemon — a separate Go-based service (default port 5002) — for all plugin-related work: model dispatch, tool calls, agent strategy execution, and credential validation. The API never runs plugin code directly; every invocation is proxied over HTTP to the daemon. 1

Dify API ──(HTTP/SSE)──► Plugin Daemon :5002
             X-Api-Key, X-Plugin-ID headers
             URL path: plugin/{tenant_id}/...

Communication Model#

All requests go through BasePluginClient in api/core/plugin/impl/base.py. Two transport modes are used:

• _request() — synchronous, blocking httpx.Client.request() call for non-streaming operations 2
• _stream_request() — synchronous generator using httpx.Client.stream(), consumed line-by-line via SSE for LLM inference, embeddings, and rerank 3

Both modes use a module-level, process-shared httpx.Client created once at import time 4. The client is pooled via get_pooled_http_client() and capped at 50 keep-alive / 100 max connections.

Key architectural constraint: Both _request() and _stream_request() are fully synchronous — they block the calling Gunicorn worker thread for the entire duration of the plugin daemon call, including all LLM inference time.

Configuration#

Defined in PluginConfig (api/configs/feature/__init__.py):

In Docker Compose deployments, set PLUGIN_DAEMON_URL=http://plugin_daemon:5002 (not localhost). 5

Worker Exhaustion Under Concurrent Load#

The Root Cause#

Gunicorn defaults in Docker to 1 worker and 10 connections 6 7:

SERVER_WORKER_AMOUNT=1
SERVER_WORKER_CLASS=gevent
SERVER_WORKER_CONNECTIONS=10

Each Gunicorn worker holds its connection to the plugin daemon open for the full duration of a plugin call — up to PLUGIN_DAEMON_TIMEOUT (default 600 s) for slow LLM inference. When an Agent node executes, it serializes multiple tool calls through the plugin daemon, each blocking the worker. With the default single worker, one in-flight Agent execution can starve all other API requests, causing complete unresponsiveness until the agent finishes or times out. 5

This is a documented failure mode in v1.15.0 — users report that even SERVER_WORKER_AMOUNT=10 may not fully resolve the issue, because a sufficiently concurrent Agent workload can still saturate the enlarged pool before per-request timeouts fire. 5

Why Agent Nodes Are Especially Prone#

Agent nodes are the worst-case consumer because:

1. They make repeated sequential plugin daemon calls per turn (strategy invocation → tool call(s) → LLM call(s)), all while holding the same Gunicorn worker.
2. The PluginAgentClient routes agent strategy execution through the daemon, which in turn calls LLM provider plugins — stacking latency.
3. MCP tool calls via the plugin daemon can individually be long-lived, and each blocks synchronously. 5

Gunicorn Startup#

The entrypoint.sh launches Gunicorn as 7:

gunicorn \
  --workers ${SERVER_WORKER_AMOUNT:-1} \
  --worker-class ${SERVER_WORKER_CLASS:-geventwebsocket.gunicorn.workers.GeventWebSocketWorker} \
  --worker-connections ${SERVER_WORKER_CONNECTIONS:-10} \
  --timeout ${GUNICORN_TIMEOUT:-200} \
  app:socketio_app

The gevent worker class provides cooperative concurrency within each worker process; however, httpx.Client calls are not greenlet-cooperative in the plugin daemon path, so they still block the gevent event loop for that worker. 8 5

Mitigations#

Related Files#