Agent V2 Architecture#

Overview#

Agent V2 is Dify's next-generation agent execution system, introduced in v1.16.0. It replaces the legacy synchronous, in-process BaseAgentRunner/CotAgentRunner/FunctionCallAgentRunner stack with an async microservice model backed by FastAPI and Pydantic AI 1. Both systems coexist — V2 does not remove the legacy runners.

The architecture is a clean separation of concerns:

• Orchestration stays in the Dify API process via DifyAgentNode, a workflow node that delegates execution to an external service over SSE.
• Execution runs in the dify-agent FastAPI microservice (agent_backend), which owns the pydantic-ai agent loop, tool calling, and LLM interactions via the Plugin Daemon.

Agent V2 nodes are available only in Workflow mode by default; see Feature Flags & Mode Restrictions for gating details. The entry point directory is api/core/workflow/nodes/agent_v2/ 2.

Deployment Configuration#

Agent V2 requires the dify-agent microservice (agent_backend) to run alongside the core API. It is included in the default Docker Compose stack starting in v1.16.0 3.

Two operational modes:

The client factory at api/clients/agent_backend/factory.py selects the client based on these settings. Without AGENT_BACKEND_BASE_URL, the factory raises ValueError: base_url is required when creating a real Agent backend client at runtime 4.

Key environment variables (defined in api/configs/extra/agent_backend_config.py):

Docker Compose service (agent_backend) 5:

• Image: langgenius/dify-agent-backend
• Depends on: redis, plugin_daemon
• Key backend vars: DIFY_AGENT_PLUGIN_DAEMON_URL, DIFY_AGENT_INNER_API_URL, DIFY_AGENT_REDIS_URL, DIFY_AGENT_LOCAL_SANDBOX_ENDPOINT

See docker/docker-compose.yaml and docker/.env.example for the full service definition and defaults.

Feature Flags & Mode Restrictions#

Agent V2 availability is controlled by two independent frontend feature flags 6:

These are checked by isAgentV2Enabled() and isAgentV2InChatflowEnabled(). The node picker hook useAvailableNodesMetaData computes: shouldUseAgentV2 = agentV2Enabled && (!isChatMode || isAgentV2InChatflowEnabled()).

As of v1.16.1, Agent V2 is restricted to Workflow mode only — the node does not appear in Chatflow by default 7 due to lack of multi-turn conversation context support 8. Setting NEXT_PUBLIC_ENABLE_AGENT_V2_IN_CHATFLOW=true re-enables it; when active in Chatflow, sessions are keyed by conversation_id instead of workflow_run_id to support multi-turn memory 1.

Legacy Agent V1 publish regression: When V2 is enabled, the legacy Agent node (agent, version 1) validators were removed from nodesMetaDataMap, causing publish and duplicate operations to fail for existing workflows containing V1 nodes 9. Fixed in PR #41379 by retaining V1 metadata even when the node is hidden from the picker.

LLM Model Configuration#

Agent V2 requires explicit model selection — there is no implicit fallback to a workspace default. This is enforced at two levels:

Schema: AgentSoulModelConfig requires three non-nullable fields:

• plugin_id — identifies the model plugin
• model_provider — provider name (e.g., openai, anthropic)
• model — model name (e.g., gpt-4o)
• credential_ref (optional) — a reference to stored credentials; actual secret values are resolved only at runtime and are never persisted in the config snapshot.

The full agent config is stored as a JSON snapshot in AgentConfigVersion.config_snapshot via AgentSoulConfig, which also holds prompt, tools, knowledge, env, sandbox, and other subsystems.

Publish-time enforcement: AgentComposerService.publish_agent_app_draft() calls agent_soul_has_model() before allowing publish. A missing model raises AgentModelNotConfiguredError (HTTP 400, error_code: "agent_model_not_configured") 10.

Workflow node validation: The runtime request builder validates model config presence before constructing the CreateRunRequest 11. Validation errors surface as workflow node failures with structured error types rather than generic exceptions.

SSE Execution Flow#

DifyAgentNode._run_inner()
  → resolve agent binding (WorkflowAgentBindingResolver)
  → build runtime request (WorkflowAgentRuntimeRequestBuilder)
  → agent_backend_client.create_run() ← HTTP POST to dify-agent
  → _consume_event_stream() ← SSE stream
  → terminal event → output / pause / fail

DifyAgentNode._run_inner() resolves the agent binding, builds a CreateRunRequest (embedding model config, tools, knowledge sets, output schema, and shell/HITL configs), POSTs it to agent_backend, then streams SSE events back.

The AgentBackendRunEventAdapter maps public dify-agent RunEvent variants to API-internal events. _consume_event_stream() processes these:

• RUN_STARTED — filtered, not forwarded downstream
• STREAM_EVENT — raw pydantic-ai stream events; discarded (not emitted as agent_log events — a known parity gap vs. the legacy in-process runner)
• AGENT_MESSAGE_DELTA — captured as node metadata
• Terminal events (returned to caller):
  • RUN_SUCCEEDED — final output + session snapshot
  • RUN_FAILED — error text + error type
  • RUN_CANCELLED — reason + optional session snapshot
  • DEFERRED_TOOL_CALL — triggers a workflow pause for Human-in-the-Loop (ask_human)

Session snapshots from terminal events are persisted to WorkflowAgentWorkspaceStore to support resumption after HITL pauses 12.

Function Calling via Pydantic AI#

Agent V2 uses pydantic-ai-slim as its execution engine. There is no built-in CoT/ReAct fallback — native OpenAI-style function calling (tools in requests, tool_calls in responses) is required 13.

LLM adapter: DifyPluginDaemonProvider implements pydantic-ai's Provider interface, bridging to the Plugin Daemon for LLM calls. It normalizes the tool_calls, function_call, and function_calls finish reason variants to a single internal "tool_call" value 1.

Tool wrapping: Plugin tools are registered as pydantic-ai Tool objects in loose mode (PLUGIN_TOOL_STRICT = False) for compatibility with schema variations across plugins 1.

Provider Compatibility Constraints#

Tool name format: OpenAI and Anthropic require tool names matching ^[a-zA-Z0-9_-]+$. Dotted names (e.g., shell.run) are rejected with HTTP 400. Dify renamed shell tools to underscored equivalents (shell_run, shell_wait) as a fix 1.

vLLM: Function calling is disabled by default. Required startup flags 13 14:

--enable-auto-tool-choice
--tool-call-parser <parser> # e.g., "mistral", "llama3_json", "hermes"

Not all vLLM-served models support function calling; support depends on the model and its chat template.

Version pinning: pydantic-ai must be pinned identically in both the api client and dify-agent backend. The FunctionToolResultEvent wire shape differs across versions; a version mismatch causes empty answers 1.

Fallback for incompatible models: Use the classic Agent V1 (ReAct/CoT) mode, which is available in all Dify versions and requires no tool_calls support 15.

Fixes Shipped in v1.16.x#

Key Source References#