Model Provider Error Handling#

Model provider errors in Dify flow through the InvokeError hierarchy defined in graphon.model_runtime.errors.invoke. Provider plugins must wrap all failure modes into the correct InvokeError subclass so the API layer can map them to appropriate HTTP status codes and client-visible error codes 1.

InvokeError Subclasses#

If a provider plugin raises a raw exception instead of one of these subclasses, it surfaces as a generic 500 "Internal Server Error" at the API boundary — the exact bug fixed for Agent V2 in PR #38584 2.

Error Propagation Path (Agent V2)#

Agent V2 runs as a separate dify-agent microservice. Errors from model providers reach the caller through a two-step translation 3:

1. dify-agent runtime preserves the structured error reason in the run_failed SSE event field reason (e.g., "InvokeRateLimitError").
2. AgentAppRunner reads AgentBackendRunFailedInternalEvent and calls _agent_backend_failure_to_exception(), which looks up the reason string in _AGENT_BACKEND_INVOKE_ERROR_BY_REASON and raises the matching InvokeError subclass 4.

The lookup table covers: InvokeAuthorizationError, InvokeBadRequestError, CredentialsValidateFailedError (→ InvokeBadRequestError), InvokeConnectionError, InvokeRateLimitError, and InvokeServerUnavailableError. Unmapped reasons fall through to AgentBackendRunFailedError 4.

HTTP Status Code Mapping#

base_app_generate_response_converter.py's _error_to_stream_response() maps InvokeRateLimitError to {code: "rate_limit_error", status: 429}. Before PR #38584, rate limit errors returned a generic completion_request_error with HTTP 400 2.

Agent V2 / vLLM Compatibility Bugs Fixed in 1.16.x#

Several issues caused Agent V2 to malfunction with vLLM and other OpenAI-compatible providers 5:

Error pass-through swallowed (PR #38584, merged 2026-07-09)#

Agent V2 was returning a generic "Internal Server Error" for all provider failures. Structured error reasons are now preserved from the dify-agent runtime through to the API layer with correct HTTP status codes 2.

Text history serialized as part objects (PR #38464, merged 2026-07-06)#

_normalize_prompt_content() collapsed only single-item text content to string form. Multi-part text-only history was sent as [{"type":"text","text":"..."}], which vLLM and similar providers rejected. The fix concatenates all text parts into a plain string 6.

Tool call ID normalization (PR #38592)#

vLLM may omit or emit placeholder IDs ("none", "null", empty). _normalize_tool_call_id() converts these to None; fallback vendor_id values are derived from chunk sequence position, preventing tool calls from collapsing into the same thought record 7.

System message ordering for Qwen/vLLM (PR #39136)#

Qwen served via vLLM requires the system prompt to be the first element in the messages array. _map_messages_to_prompt_messages() now merges all system messages and hoists them to index 0 8.

vLLM Setup Requirements for Agent V2#

Agent V2 requires native OpenAI-style function calling — no CoT/ReAct fallback exists. For vLLM, function calling must be explicitly enabled 9:

--enable-auto-tool-choice
--tool-call-parser <parser> # e.g., "mistral", "llama3_json", "hermes"

If the model doesn't support function calling, fall back to classic agent mode (ReAct/CoT). Agent V2 is enabled by default in Dify ≥ 1.16.0 via ENABLE_AGENT_V2=true in docker/.env 10.

Key Source Files#

Related Issues and PRs#

• PR #38584 — Error pass-through for model invocation failures 2
• PR #38464 — Text history serialization fix 6
• Issue #38882 — Agent V2 incompatibility with vLLM 11
• Issue #15120 — "Non-JSON encountered." with OpenAI-compatible models 11