Model Provider Error Handling#

Model provider errors in Dify flow through a typed InvokeError hierarchy. Provider plugins must wrap all failure modes into the correct subclass so the API layer maps them to the right HTTP status codes. Unwrapped raw exceptions surface as a generic 500 Internal Server Error at the API boundary — the exact bug fixed for Agent V2 in PR #38584. 1

InvokeError Subclasses#

These are defined in dify_plugin.errors.model 2:

Error Mapping in Plugins#

Each plugin exposes _invoke_error_mapping, which maps OpenAI SDK exception types to the canonical InvokeError subclasses. For the openai_api_compatible plugin this looks like:

• openai.APIConnectionError / openai.APITimeoutError → InvokeConnectionError
• openai.InternalServerError → InvokeServerUnavailableError
• openai.RateLimitError → InvokeRateLimitError
• openai.AuthenticationError / openai.PermissionDeniedError → InvokeAuthorizationError
• openai.BadRequestError / openai.NotFoundError / openai.APIError → InvokeBadRequestError

3

HTTP / Error Code Mapping at the API Layer#

Two paths apply the final exception-to-HTTP translation 4:

1. Synchronous responses — try/except blocks in controllers such as completion.py
2. Streaming responses — _error_to_stream_response() in base_app_generate_response_converter.py

Exception ordering matters: services.errors.llm.InvokeRateLimitError (service-layer quota) is a subclass of services.errors.llm.InvokeError. If the base InvokeError handler is placed first in the except chain, rate-limit errors are silently downgraded to HTTP 400. Controllers correctly place InvokeRateLimitError before InvokeError. 4

Three distinct InvokeRateLimitError classes exist — services.errors.llm.InvokeRateLimitError (Cloud quota, HTTP 429), graphon.model_runtime.errors.invoke.InvokeRateLimitError (provider throttling, HTTP 400 in sync / status: 429 in streaming), and controllers.web.error.InvokeRateLimitError (HTTP response class). Controllers import all three with aliases to avoid collisions. 4

Agent V2 Error Propagation Path#

Agent V2 runs as a separate dify-agent microservice. Errors from model providers reach the API caller through a two-step translation 1:

1. dify-agent runtime serializes the structured error reason (e.g., "InvokeRateLimitError") into the run_failed SSE event's reason field.
2. AgentAppRunner reads AgentBackendRunFailedInternalEvent and calls _agent_backend_failure_to_exception(), which looks up the reason string in _AGENT_BACKEND_INVOKE_ERROR_BY_REASON and raises the matching InvokeError subclass.

Unmapped reason strings fall through to a generic AgentBackendRunFailedError. 1

Agent V2 / vLLM Compatibility Bugs Fixed in v1.16.x#

Several bugs caused Agent V2 to malfunction with vLLM and OpenAI-compatible providers. All were fixed in the 1.16.x timeframe 5:

6

vLLM Function Calling Requirements#

Agent V2 uses pydantic-ai and requires native OpenAI-style function calling — no CoT/ReAct fallback 5. For vLLM, enable function calling at startup:

--enable-auto-tool-choice
--tool-call-parser <parser> # e.g., "mistral", "llama3_json", "hermes"

If the model doesn't support function calling, fall back to classic agent mode (Agent V1 / ReAct). 5

SSRF Proxy Error Wrapping#

All outbound model provider requests route through api/core/helper/ssrf_proxy.py. Network and SSL failures that occur at the SSRF proxy layer are not automatically wrapped into InvokeError subclasses — the proxy's make_request() raises httpx.RequestError (on first failure when max_retries=0) or MaxRetriesExceededError (after exhausting retries). Providers or plugin daemon wrappers must re-wrap these into InvokeConnectionError or InvokeServerUnavailableError as appropriate. 7

Retried HTTP status codes include 429, 500, 502, 503, 504 with exponential backoff (0.5 × 2^(retry-1)); default max retries = 3, controlled by SSRF_DEFAULT_MAX_RETRIES. 7

Key Source Files#