httpx and Gevent Compatibility#

The Problem#

Dify's Gunicorn workers run under the gevent worker class, which provides cooperative concurrency via greenlets. All plugin daemon calls flow through a synchronous httpx.Client — but httpcore's synchronous connection pool does not cooperatively yield to gevent's event loop during I/O. This mismatch causes greenlet starvation and produces httpx.ConnectError: [Errno 111] Connection refused under concurrent browser load, even when the plugin daemon itself is healthy and reachable.

The failure is characteristically only reproducible under real concurrent Gunicorn/gevent traffic — isolated test requests to the daemon from outside the worker process succeed reliably 1.

Root Causes#

1. Non-cooperative httpcore connection pool#

The plugin daemon client is a module-level, process-shared httpx.Client capped at 50 keep-alive / 100 max connections 2. Both _request() and _stream_request() are fully synchronous — they block the calling greenlet for the entire duration of the call, including all LLM inference and SSE streaming time. httpcore does not insert gevent yield points, so each greenlet monopolizes the event loop while waiting for I/O.

When 4–8 XHRs fire in parallel (e.g., loading the Integrations page), greenlets pile up behind the connection pool's internal synchronization, starving each other and producing spurious connection errors 3.

2. threading.Lock import-order sensitivity#

HttpClientPoolFactory uses a threading.Lock() for pool initialization. Gevent monkey-patches threading.Lock to be greenlet-aware — but only if the module is imported after patching. gunicorn.conf.py is loaded before gevent applies monkey-patching (before worker init_process()), so any top-level imports of HTTPS-using libraries there will bind to unpatched stdlib primitives, causing deadlocks or RecursionError in gevent workers 4 5.

This is why gunicorn.conf.py defers gRPC and psycopg2 initialization to a post_patch hook subscribed to GeventDidPatchBuiltinModulesEvent 6. The original bug that introduced this pattern is tracked in issue #26689.

3. No retry logic#

_request_with_plugin_daemon_response() makes a single attempt with no retry. Any transient contention from the pool immediately surfaces as a user-visible error 7.

Why Scaling Workers Doesn't Help#

Adding more workers (SERVER_WORKER_AMOUNT) adds more processes, each with its own pool — but the contention is intra-process among greenlets sharing one httpx.Client. The default SERVER_WORKER_CONNECTIONS=10 means 10 greenlets per worker compete for the same pool simultaneously; more worker processes do not reduce that pressure 8.

Gevent Monkey-Patching Architecture#

Dify's patching entry points:

The ordering constraint: grpc_gevent.init_gevent() must be called after stdlib patching — calling it in post_fork or at module top-level in gunicorn.conf.py causes deadlocks 9.

Workarounds and Mitigations#

There is no code-level fix in place — the plugin daemon client remains a synchronous httpx.Client as of the current codebase.

A proper fix would require migrating to httpx.AsyncClient, adding gevent.sleep(0) yield points within the httpcore transport, or implementing retry-with-backoff in the plugin client 10.

Key Files#