Documentsdify
Workflow Pause-Resume State Management
Workflow Pause-Resume State Management
Type
Topic
Status
Published
Created
Jul 8, 2026
Updated
Jul 9, 2026
Created by
Dosu Bot
Updated by
Dosu Bot

Workflow Pause-Resume State Management#

Dify workflows can pause mid-execution at Human Input nodes (HITL — human-in-the-loop) and resume after a user submits a form. This cycle spans multiple request contexts, requiring explicit serialization and restoration of execution state.

Two independently-managed state components must survive a pause boundary:

  1. GraphRuntimeState — variable pool, LLM usage counters, execution metadata. Serialized by PauseStatePersistenceLayer and restored on resume.
  2. ResponseStreamFilter — a graphon filter that gates when Answer nodes may stream. Serialized by PauseStatePersistenceLayer and restored on resume as of PR #38540.

What Gets Persisted: PauseStatePersistenceLayer#

PauseStatePersistenceLayer is a GraphEngineLayer that fires on every GraphRunPausedEvent. When the workflow pauses, it builds a WorkflowResumptionContext containing:

  • serialized_graph_runtime_state — the full GraphRuntimeState.dumps() output (variable pool, LLM usage, execution context)
  • generate_entity — a discriminated-union wrapper around either WorkflowAppGenerateEntity or AdvancedChatAppGenerateEntity, needed to reconstruct the correct entity type on resume
  • serialized_response_stream_filter_state — the ResponseStreamFilter.dumps() output, persisted as of PR #38540 (optional for backward compatibility)

The context is serialized via Pydantic's model_dump_json() and stored in the database via APIWorkflowRunRepository.create_workflow_pause(). Graphon session IDs from the GraphRunPausedEvent are translated to Dify form IDs by enrich_graph_pause_reasons() before persistence, so model layers only handle Dify-owned identifiers.

The Streaming Filter: ResponseStreamFilter#

ResponseStreamFilter (from graphon.filters) controls when Answer nodes are allowed to emit stream tokens. On each engine run, iter_dify_graph_engine_events() in workflow_entry.py wraps engine.run() with a ResponseStreamFilter() instance, which may be either a fresh instance (for new runs) or a restored instance (for resumed runs). As of PR #38540, WorkflowEntry accepts an optional response_stream_filter parameter, defaulting to a fresh filter when None:

yield from filter_graph_events(
    engine.run(),
    context=GraphEventFilterContext.from_engine(engine),
    filters=[response_stream_filter or ResponseStreamFilter()],
)

The filter builds a paths_map of blocking edges that must be traversed before each downstream Answer node is allowed to stream. As edges are taken, they are removed from the path. When a path is empty, the Answer node is unlocked.

The filter has dumps() and loads() methods for serialization, carrying state fields including paths_map, active_session, waiting_sessions, pending_sessions, stream_buffers, stream_positions, closed_streams, and node_execution_ids. As of PR #38540, Dify calls these methods at pause/resume boundaries: PauseStatePersistenceLayer invokes dumps() to persist filter state, and on resume WorkflowResumptionContext.get_response_stream_filter() invokes loads() to restore it.

The Event Pipeline: QueueTextChunkEvent → MessageStreamResponse#

When an Answer node streams text, the event pipeline is:

  1. The graphon engine emits a raw variable stream chunk, which ResponseStreamFilter allows through as a QueueTextChunkEvent (defined in queue_entities.py)
  2. _handle_text_chunk_event() in AdvancedChatAppGenerateTaskPipeline receives the event, appends event.text to WorkflowTaskState.answer, then delegates to MessageCycleManager.message_to_stream_response()
  3. MessageCycleManager wraps the token in a MessageStreamResponse (event type StreamEvent.MESSAGE) and returns it to the SSE stream

The WorkflowTaskState.answer accumulates all streamed tokens in-memory . On pause, _handle_workflow_paused_event() saves the partial answer to the Message row and marks message.status = MessageStatus.PAUSED. On next-invocation resume, _seed_task_state_from_message() re-seeds _task_state.answer from the persisted message answer, so pre-pause streamed text is not double-counted.

Historical Issue: Answer Node Streaming After Resume (Fixed in PR #38540)#

Affected versions: Dify 1.15.0 (introduced with graphon v0.5.x). Reportedly worked in 1.13.0/1.14.x. Fixed in PR #38540.

Symptom: After a Human Input pause/resume, the downstream Answer node executes successfully (node_finished/workflow_finished contain expected text, status is succeeded), but the frontend receives no stream tokens and the final assistant message is empty.

Root cause: Before PR #38540, on resume, iter_dify_graph_engine_events() instantiated a fresh ResponseStreamFilter() with no prior state. If an upstream if-else edge was traversed before the pause, that edge would not be re-emitted on resume — so the new filter never recorded it, and the Answer node's path never became empty. The Answer node was permanently blocked from streaming.

Triggers: The bug surfaced when any of these conditions held upstream of the Human Input node:

  • An if-else / conditional branch node
  • A second (or later) Human Input pause in the same run
  • An Answer node that already streamed before the pause
  • A variable written before the pause that the post-resume Answer node references

Same symptom in issue #38315: "Frontend cannot receive reply after Manual Intervention Node + Conditional Branch" — the branch nodes are the key upstream condition.

What PR #38247 did not fix: The July 2026 graphon v0.6.0 upgrade migrated HITL logic to Dify's DifyHITLCallback/SessionBinding and improved pause-reason enrichment, but made no changes to ResponseStreamFilter state serialization.

The fix (PR #38540): ResponseStreamFilter state is now serialized by PauseStatePersistenceLayer.dumps() on pause and restored via WorkflowResumptionContext.get_response_stream_filter() on resume. The filter instance is threaded through the entire call chain — from WorkflowEntry/iter_dify_graph_engine_events() to both app runners and generators to both resume entry points (UI-driven human-input resume and the trigger/webhook async resume path). Backward compatible: the new persisted field is optional, so a workflow run paused before this fix resumes with the old fresh-filter behavior for that one stale run.

Key Source Files#

FilePurpose
api/core/app/layers/pause_state_persist_layer.pyPauseStatePersistenceLayer — serializes GraphRuntimeState, generate entity, and ResponseStreamFilter state on pause
api/core/workflow/workflow_entry.pyiter_dify_graph_engine_events() — accepts an optional pre-restored ResponseStreamFilter for resumed runs
api/core/app/apps/advanced_chat/generate_task_pipeline.pyAdvancedChatAppGenerateTaskPipeline — event dispatch loop, pause/resume message persistence
api/core/app/task_pipeline/message_cycle_manager.pyMessageCycleManager.message_to_stream_response() — converts text chunks to MessageStreamResponse
api/core/app/entities/queue_entities.pyQueueTextChunkEvent and all other queue event types
api/core/app/entities/task_entities.pyWorkflowTaskState, MessageStreamResponse, StreamEvent enum