Workflow Pause-Resume State Management#
Dify workflows can pause mid-execution at Human Input nodes (HITL — human-in-the-loop) and resume after a user submits a form. This cycle spans multiple request contexts, requiring explicit serialization and restoration of execution state.
Two independently-managed state components must survive a pause boundary:
GraphRuntimeState— variable pool, LLM usage counters, execution metadata. Serialized byPauseStatePersistenceLayerand restored on resume.ResponseStreamFilter— a graphon filter that gates when Answer nodes may stream. Serialized byPauseStatePersistenceLayerand restored on resume as of PR #38540.
What Gets Persisted: PauseStatePersistenceLayer#
PauseStatePersistenceLayer is a GraphEngineLayer that fires on every GraphRunPausedEvent. When the workflow pauses, it builds a WorkflowResumptionContext containing:
serialized_graph_runtime_state— the fullGraphRuntimeState.dumps()output (variable pool, LLM usage, execution context)generate_entity— a discriminated-union wrapper around eitherWorkflowAppGenerateEntityorAdvancedChatAppGenerateEntity, needed to reconstruct the correct entity type on resumeserialized_response_stream_filter_state— theResponseStreamFilter.dumps()output, persisted as of PR #38540 (optional for backward compatibility)
The context is serialized via Pydantic's model_dump_json() and stored in the database via APIWorkflowRunRepository.create_workflow_pause(). Graphon session IDs from the GraphRunPausedEvent are translated to Dify form IDs by enrich_graph_pause_reasons() before persistence, so model layers only handle Dify-owned identifiers.
The Streaming Filter: ResponseStreamFilter#
ResponseStreamFilter (from graphon.filters) controls when Answer nodes are allowed to emit stream tokens. On each engine run, iter_dify_graph_engine_events() in workflow_entry.py wraps engine.run() with a ResponseStreamFilter() instance, which may be either a fresh instance (for new runs) or a restored instance (for resumed runs). As of PR #38540, WorkflowEntry accepts an optional response_stream_filter parameter, defaulting to a fresh filter when None:
yield from filter_graph_events(
engine.run(),
context=GraphEventFilterContext.from_engine(engine),
filters=[response_stream_filter or ResponseStreamFilter()],
)
The filter builds a paths_map of blocking edges that must be traversed before each downstream Answer node is allowed to stream. As edges are taken, they are removed from the path. When a path is empty, the Answer node is unlocked.
The filter has dumps() and loads() methods for serialization, carrying state fields including paths_map, active_session, waiting_sessions, pending_sessions, stream_buffers, stream_positions, closed_streams, and node_execution_ids. As of PR #38540, Dify calls these methods at pause/resume boundaries: PauseStatePersistenceLayer invokes dumps() to persist filter state, and on resume WorkflowResumptionContext.get_response_stream_filter() invokes loads() to restore it.
The Event Pipeline: QueueTextChunkEvent → MessageStreamResponse#
When an Answer node streams text, the event pipeline is:
- The graphon engine emits a raw variable stream chunk, which
ResponseStreamFilterallows through as aQueueTextChunkEvent(defined inqueue_entities.py) _handle_text_chunk_event()inAdvancedChatAppGenerateTaskPipelinereceives the event, appendsevent.texttoWorkflowTaskState.answer, then delegates toMessageCycleManager.message_to_stream_response()MessageCycleManagerwraps the token in aMessageStreamResponse(event typeStreamEvent.MESSAGE) and returns it to the SSE stream
The WorkflowTaskState.answer accumulates all streamed tokens in-memory . On pause, _handle_workflow_paused_event() saves the partial answer to the Message row and marks message.status = MessageStatus.PAUSED. On next-invocation resume, _seed_task_state_from_message() re-seeds _task_state.answer from the persisted message answer, so pre-pause streamed text is not double-counted.
Historical Issue: Answer Node Streaming After Resume (Fixed in PR #38540)#
Affected versions: Dify 1.15.0 (introduced with graphon v0.5.x). Reportedly worked in 1.13.0/1.14.x. Fixed in PR #38540.
Symptom: After a Human Input pause/resume, the downstream Answer node executes successfully (node_finished/workflow_finished contain expected text, status is succeeded), but the frontend receives no stream tokens and the final assistant message is empty.
Root cause: Before PR #38540, on resume, iter_dify_graph_engine_events() instantiated a fresh ResponseStreamFilter() with no prior state. If an upstream if-else edge was traversed before the pause, that edge would not be re-emitted on resume — so the new filter never recorded it, and the Answer node's path never became empty. The Answer node was permanently blocked from streaming.
Triggers: The bug surfaced when any of these conditions held upstream of the Human Input node:
- An
if-else/ conditional branch node - A second (or later) Human Input pause in the same run
- An Answer node that already streamed before the pause
- A variable written before the pause that the post-resume Answer node references
Same symptom in issue #38315: "Frontend cannot receive reply after Manual Intervention Node + Conditional Branch" — the branch nodes are the key upstream condition.
What PR #38247 did not fix: The July 2026 graphon v0.6.0 upgrade migrated HITL logic to Dify's DifyHITLCallback/SessionBinding and improved pause-reason enrichment, but made no changes to ResponseStreamFilter state serialization.
The fix (PR #38540): ResponseStreamFilter state is now serialized by PauseStatePersistenceLayer.dumps() on pause and restored via WorkflowResumptionContext.get_response_stream_filter() on resume. The filter instance is threaded through the entire call chain — from WorkflowEntry/iter_dify_graph_engine_events() to both app runners and generators to both resume entry points (UI-driven human-input resume and the trigger/webhook async resume path). Backward compatible: the new persisted field is optional, so a workflow run paused before this fix resumes with the old fresh-filter behavior for that one stale run.
Key Source Files#
| File | Purpose |
|---|---|
api/core/app/layers/pause_state_persist_layer.py | PauseStatePersistenceLayer — serializes GraphRuntimeState, generate entity, and ResponseStreamFilter state on pause |
api/core/workflow/workflow_entry.py | iter_dify_graph_engine_events() — accepts an optional pre-restored ResponseStreamFilter for resumed runs |
api/core/app/apps/advanced_chat/generate_task_pipeline.py | AdvancedChatAppGenerateTaskPipeline — event dispatch loop, pause/resume message persistence |
api/core/app/task_pipeline/message_cycle_manager.py | MessageCycleManager.message_to_stream_response() — converts text chunks to MessageStreamResponse |
api/core/app/entities/queue_entities.py | QueueTextChunkEvent and all other queue event types |
api/core/app/entities/task_entities.py | WorkflowTaskState, MessageStreamResponse, StreamEvent enum |