Workflow Iteration Node Execution#

Dify's iteration and loop container nodes execute their sub-graphs by spawning independent child GraphEngine instances via GraphRuntimeState.create_child_engine(). Each child engine runs synchronously per-item (iteration) or per-cycle (loop), sharing the parent's VariablePool and execution_context while maintaining its own GraphRuntimeState. User identity and app context flow through a single run_context object propagated from parent to all child nodes.

Iteration and loop node logic itself lives in the external graphon package; Dify registers those node types via DifyNodeFactory and implements Dify-specific node features (model providers, tool runtime, RAG, human-input callback) in node_factory.py.

Child Engine Creation#

Child engines are created directly by iteration and loop nodes through GraphRuntimeState.create_child_engine(), which lives in the graphon package. The Dify-side integration coordinates runtime state setup and DifyNodeFactory construction:

1. GraphRuntimeState — iteration and loop nodes call create_child_engine() on the parent runtime state. The child engine shares the parent's variable pool by default; the parent's execution_context (Python contextvars snapshot) is copied directly.
2. DifyNodeFactory — DifyNodeFactory.with_runtime_state(child_graph_runtime_state) creates a new factory scoped to the child's runtime state. This pattern ensures each child iteration has its own execution tracking while preserving the Dify-specific node construction logic (model providers, tool runtime, RAG, human-input callbacks).
3. Graph — graphon initializes the child Graph from graph_config, starting at the root_node_id specified by the iteration or loop node.
4. Layers — graphon applies only child-safe layers (e.g., quota enforcement). The parent's ObservabilityLayer and ExecutionLimitsLayer are not re-applied at the child level.
5. Recursion — child engines are fully nested; an iteration or loop node inside a child subgraph can create its own child engines, enabling arbitrarily nested container nodes.

WorkflowEntry no longer holds a separate child-engine builder; child engine construction is delegated to graphon's runtime state directly.

User Context Propagation#

User identity flows through a run_context object (DifyRunContext) containing tenant_id, app_id, user_id, user_from, invoke_from, and trace_session_id. PR #32964 consolidated these fields (previously passed individually) into a single run_context built by build_dify_run_context().

DifyGraphInitContext (in node_factory.py) holds the run_context and bridges it to graphon's GraphInitParams, which are shared across all nodes in both parent and child graphs. This means require_dify_context().user_id is available to every node, including those inside iteration subgraphs.

Bug: empty user_id in single-node iteration/loop runs — PR #34044 fixed a regression where single-iteration and single-loop debug execution paths passed user_id="" into build_dify_run_context. Inside the subgraph, DatasetQuery.created_by was set to '', which PostgreSQL rejects for a UUID column. The fix threads application_generate_entity.user_id through _prepare_single_node_execution → _get_graph_and_variable_pool_for_single_node_run across all app runners 7. A defensive parse_uuid_str_or_none() guard was also added in the RAG query logging path to skip writes when user_id is missing or invalid.

VariablePool Lifecycle#

The VariablePool is bootstrapped once at workflow start via build_bootstrap_variables(), which assembles variables from four namespaces keyed by sentinel node IDs:

SystemVariableKey defines all system variable names as a StrEnum; user identity is available as SystemVariableKey.USER_ID ("user_id").

The pool instance lives in GraphRuntimeState and is shared in-place across the parent engine and all child engines created for each iteration/loop cycle. Child nodes write their outputs back with variable_pool.add((node_id, key), value), making them available to downstream nodes including across iteration boundaries.

Stale variable bug in loop nodes — before PR #30059, variables produced inside a loop subgraph persisted in the parent pool across iterations. A subsequent iteration's child engine could read stale prior-iteration outputs and re-stream them. The fix added _clear_loop_subgraph_variables(loop_node_ids) in the graphon LoopNode._run(), which removes all subgraph-node variables from the shared pool at the start of each cycle before the child engine is recreated 8.

Memory-dependent nodes at construction time — LLM, QUESTION_CLASSIFIER, and PARAMETER_EXTRACTOR nodes that use memory require CONVERSATION_ID before they are instantiated. get_node_creation_preload_selectors() returns those selectors for pre-loading before DifyNodeFactory.create_node() runs.

Human Input Support in Loops and Iteration#

PR #39243 adds Human Input (HUMAN_INPUT node) support within iteration and loop subgraphs. Previously, Human Input nodes paused execution indefinitely and tracked pause state in memory; this was incompatible with container nodes that serialize/deserialize their state across iterations.

The implementation resolves this by:

1. Form persistence — Human Input forms created inside a loop or iteration cycle are persisted in the database with a stable execution_id derived from the node's unique execution context. When the workflow is resumed after user input, the child engine is reconstructed with the same execution_id, allowing the Human Input node to retrieve its previously-created form and complete successfully.
2. Execution ID binding — DifyNodeFactory provides a deferred execution_id_getter callback to the Human Input callback during node construction. The actual execution_id is bound once the node starts executing.
3. Graph state restoration — when resuming a paused workflow that contains iteration or loop nodes, the original workflow graph is restored from WorkflowRun.graph (stored at pause time). This ensures the iteration/loop subgraph configuration is unchanged across the pause boundary.
4. Celery repository cache — CeleryWorkflowNodeExecutionRepository backfills its in-memory cache from the database on the first access to a workflow_execution_id, ensuring resumed workflows can see previously-saved child node executions from prior iterations.

Key Files and References#

Notable PRs and issues: