Agent V2 Architecture#

Agent V2 is Dify's next-generation agent execution system, built as a standalone dify-agent subpackage within the monorepo. It shifts from the legacy synchronous, in-process BaseAgentRunner/CotAgentRunner/FunctionCallAgentRunner stack to an async microservice model backed by FastAPI, Redis Streams, and Pydantic AI 1.

The dify-agent package organizes into three namespaces:

Agent V2 does not replace the legacy runner system — both coexist. The dify-agent package was promoted from a dev dependency to a production dependency in PR #36284 1.

The API layer calls the agent backend via create_agent_backend_run_client(), which wraps dify_agent.client.Client for HTTP communication with the dify-agent FastAPI service. Agent runs execute as background asyncio.Tasks; clients poll or stream events via Redis Streams SSE. Model credentials are never persisted to Redis — they are submitted per run request and held only in memory 1.

Deployment Configuration#

Agent V2 is enabled by default in Dify 1.16.0 and runs automatically with docker compose up 2.

Feature flags (set in docker/.env):

ENABLE_AGENT_V2=true # backend (default)
NEXT_PUBLIC_ENABLE_AGENT_V2=true # frontend (default)

The Agent Roster page is available at /roster.

Agent Backend Authentication#

The agent backend /runs API supports optional bearer token authentication. When configured, both sides must share the same token:

• AGENT_BACKEND_API_TOKEN (Dify API side): Bearer token sent to the agent backend /runs API. Must match DIFY_AGENT_API_TOKEN on the server side.
• DIFY_AGENT_API_TOKEN (agent backend side): Inbound bearer token for /runs API authentication. Must match AGENT_BACKEND_API_TOKEN on the Dify API side.

Backward compatibility: Authentication is optional. When DIFY_AGENT_API_TOKEN is None, the agent backend accepts all requests without checking the Authorization header, supporting graceful migration for existing deployments.

Production guidance: Replace the development default token value in production deployments. Generate a secure token using:

python -c 'import secrets; print(secrets.token_urlsafe(32))'

The agent backend validates tokens using constant-time comparison (hmac.compare_digest) to prevent timing attacks.

Two deployment modes#

1. Real microservice (production)

Run the dify-agent FastAPI service and point the API at it:

AGENT_BACKEND_BASE_URL=http://agent_backend:5050

The default in docker/.env.example is http://agent_backend:5050. The agent_backend and local_sandbox services are included in the default Docker Compose deployment starting in 1.16.0. The dify-agent backend service also requires its own environment (Redis, Plugin Daemon URL/key, etc.) — see the full reference in api/configs/extra/agent_backend_config.py and the dify-agent/src/dify_agent/server/settings.py DIFY_AGENT_* vars 1.

Stream management and timeout configuration:

The API client provides three configuration variables that control SSE stream reliability and timeouts when communicating with the Agent backend:

AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS=30
AGENT_BACKEND_STREAM_MAX_RECONNECTS=3
AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200

• AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS sets the read timeout for each individual SSE connection. If the backend stops sending events for this duration, the connection is considered stale and triggers a reconnection attempt.
• AGENT_BACKEND_STREAM_MAX_RECONNECTS limits the number of automatic reconnection attempts. Once exhausted, the run is marked as failed. This prevents infinite retry loops on persistent backend failures.
• AGENT_BACKEND_RUN_TIMEOUT_SECONDS enforces a total deadline for the entire agent workflow execution. If the run does not complete within this window, the stream is terminated and the workflow is cancelled.

These values are defined in agent_backend_config.py and control the client-side behavior. The backend also sends periodic heartbeat events to keep the stream alive during long-running operations.

If AGENT_BACKEND_BASE_URL is not set, the factory raises ValueError: base_url is required when creating a real Agent backend client at runtime 3 4.

2. In-process fake client (single-process workaround)

For deployments without the dify-agent microservice (e.g., non-Docker environments), a deterministic fake client can be used:

AGENT_BACKEND_USE_FAKE=true
AGENT_BACKEND_FAKE_SCENARIO=success # or: failed, paused

This unblocks development/testing without needing the separate service. All three env vars are defined in AgentBackendConfig.

Additional opt-in feature#

AGENT_SHELL_ENABLED (default true) injects the Home, Workspace, Sandbox, and Shell runtime layers into Agent runs. Requires Dify Agent to have a deployment-selected runtime backend 5.

LLM Model Configuration#

Agent V2 requires an explicit model selection on each agent before it can be published. This is enforced at two levels:

Schema: AgentSoulModelConfig requires three non-nullable fields:

An optional credential_ref (AgentSoulModelCredentialRef) holds a reference to stored credentials — the actual secret values are resolved only at runtime and never stored in the config snapshot 1.

Publish-time enforcement: PR #38451 added a guard in AgentComposerService.publish_agent_app_draft() that checks agent_soul_has_model() before allowing publish. If the model is absent, it raises AgentModelNotConfiguredError (HTTP 400, error_code: "agent_model_not_configured"), leaving the draft/snapshot state unchanged 6.

The full agent "soul" configuration is stored as a JSON snapshot in AgentConfigVersion.config_snapshot. The top-level shape is AgentSoulConfig, which contains the model field alongside prompt, tools, knowledge, env, sandbox, and other subsystems.

Function Calling via Pydantic AI#

The dify-agent runtime uses pydantic-ai-slim as its agent execution engine. All tool/function calling goes through this abstraction layer.

How it works#

• LLM adapter: DifyPluginDaemonProvider implements Pydantic AI's Provider interface, bridging to Dify's Plugin Daemon for LLM calls. DifyLLMAdapterModel implements the Model interface, translating Pydantic AI messages to Graphon-compatible request/response format 1.
• Tool wrapping: Plugin tools are wrapped as Pydantic AI Tool objects. Tools run in loose mode (PLUGIN_TOOL_STRICT = False) for compatibility with schema variations across plugins 1.
• Finish reason normalization: The adapter normalizes "tool_calls", "function_call", and "function_calls" to a single "tool_call" finish reason 1.
• Structured output: Invalid structured output triggers Pydantic AI's built-in retry mechanism automatically, without needing business-layer error handling 1.

Provider compatibility constraints#

Function name format: OpenAI and Anthropic require tool names matching ^[a-zA-Z0-9_-]+$. Dotted names (e.g., shell.run) are rejected with HTTP 400. This surfaced when the dify.shell layer was introduced — its tools had to be renamed to shell_run, shell_wait, etc. 7.

Version pinning: The Pydantic AI version must be pinned identically across the api client and the dify-agent backend. The FunctionToolResultEvent wire shape differs across versions, causing empty answers when the versions diverge 7.

Thinking mode + function calling bug: When function calling is used with thinking mode enabled, the terminal output can be null, causing downstream Respond nodes to receive nothing. Fixed in PR #38287 8.

Tool types#

Layers registered in the agent compositor expose different tool categories:

Tool configuration in the persisted soul uses AgentSoulDifyToolConfig, which routes plugin providers through dify.plugin.tools and builtin/api/workflow/mcp providers through dify.core.tools.

Agent Working Environment Architecture#

Agent runtime resources are managed through an immutable home snapshot + persistent workspace architecture introduced in PR #39480. Dify API owns all logical resource lifecycle; Dify Agent remains a stateless execution service.

Core Components#

Home Snapshots are optional immutable base environments managed by AgentHomeSnapshotService (api/services/agent/home_snapshot_service.py). Each snapshot represents a versioned starting point for an agent participant. Home snapshots are created:

• When a Build Draft applies changes via "Apply to Build"

Home snapshots are not automatically created during agent initialization, import, or clone operations. Agent config snapshots, drafts, and workspace bindings can carry a nullable home snapshot reference.

Workspaces are persistent working environments owned by specific scopes (owner_type, owner_id, owner_scope_key) managed by AgentWorkspaceService (api/services/agent/workspace_service.py). Workspaces store files and state across multiple runs within the same product scope (e.g., a conversation or workflow).

Agent Bindings (AgentWorkspaceBinding) are materialized participant identities that connect agents to workspaces. Each binding:

• Connects one agent to one workspace
• Tracks session state and pending interactions
• References an optional base home snapshot version
• Owns the participant identity used by runtime requests

When an Execution Binding has no explicit home snapshot, a backend-default mutable Home is materialized instead. Explicit snapshot references remain fail-fast with no fallback.

Runtime Leases are operation-scoped resource leases for execution. Each run request includes a backend_binding_ref that identifies the participant and its workspace.

Resource Lifecycle#

Resources follow an ACTIVE → RETIRED → collected lifecycle:

1. Allocation: create_binding() allocates a new participant with physical backend resources
2. Retirement: retire_binding() / retire_workspace() synchronously mutates transaction state to RETIRED
3. Collection: Asynchronous physical cleanup via collect_agent_resources_task.py on the retention queue

The separation of retirement (synchronous, marks state) and collection (asynchronous, deletes physical resources) ensures product transactions complete quickly while resource cleanup happens reliably in the background.

Request Building#

Runtime requests now use:

• Required field: backend_binding_ref (replaces session snapshot extraction)
• Runtime layer: DIFY_RUNTIME_LAYER_ID with DifyRuntimeLayerConfig containing the binding ref
• Removed: suspend_on_exit field (all runs now use ExitIntent.SUSPEND)
• Removed: RuntimeLayerSpec extraction and cleanup composition

Sandbox API#

Sandbox API endpoints changed from session-based to caller-based lookup:

• Query parameters changed from conversation_id to caller_type / caller_id
• caller_type values: "conversation", "build_draft", or node execution ID for workflow nodes
• SandboxInfoResponse no longer includes session_id field

Key Source References#