Workflow Draft Synchronization#

Workflow draft sync is the mechanism by which the canvas editor persists in-progress workflow state. It spans four layers: frontend debouncing, backend HTTP validation, agent binding reconciliation, and (optionally) real-time multi-user collaboration via CRDT. Understanding each layer is critical for debugging save hangs, "stuck in queue" preview failures, and silent publish errors.

Data Flow#

1

Database Model#

The workflows table stores draft and published rows with the same schema — draft rows carry version = "draft" while published rows carry a timestamp string as the version 2. Only one draft row exists per app; published rows are immutable snapshots. Both are fetched by app_id + tenant_id + version, and the draft is selected with .with_for_update() before every write to prevent concurrent overwrites 3.

Key columns:

• graph (LongText) — JSON of all nodes and edges
• features (LongText) — app-level feature config
• _environment_variables (LongText) — secret-encrypted JSON keyed by variable name
• unique_hash — SHA-1 of {"graph": graph_dict} used as an optimistic lock token 4

Frontend: Debounce, Payload, and Guards#

Debounce — workflow-draft-slice.ts creates a 5-second debounced sync function via es-toolkit. Rapid edits coalesce; flushPendingSync() forces immediate dispatch 1.

Payload construction — use-nodes-sync-draft.ts (doSyncWorkflowDraft) 1:

• Strips internal _-prefixed fields and selected flags from nodes and edges.
• Filters out StartPlaceholder and unlinked temporary nodes.
• Includes syncWorkflowDraftHash; a draft_workflow_not_sync server error triggers a full draft refresh.
• In collaboration mode, non-leader sessions delegate to collaborationManager.requestWorkflowSync() instead of calling the API directly.

Page-close guard — syncWorkflowDraftWhenPageClose fires a fetch keepalive on visibilitychange hidden / beforeunload. In collaboration mode this is gated by canFlushGraphOnPageClose() or canUseLocalDraftFallback() to prevent a stale snapshot from overwriting other collaborators' work 1.

App-deletion guard — shouldSkipDraftSync checks isAppDeletingOrDeleted(appId) and skips the HTTP call if the app is mid-deletion, suppressing 404 toasts while still calling the onSettled cleanup callback 1.

Backend: REST Handler and Validation Pipeline#

Entry point — DraftWorkflowApi.post parses a SyncDraftWorkflowPayload 5 and delegates to WorkflowService.sync_draft_workflow 6.

Key SyncDraftWorkflowPayload fields 1:

sync_draft_workflow execution order 7:

1. Row lock — with_for_update() before the hash check.
2. Hash check — raises WorkflowHashNotEqualError → HTTP draft_workflow_not_sync if workflow.unique_hash != unique_hash.
3. Feature/graph validation — validate_features_structure() and validate_graph_structure() (skipped when graph_only=True).
4. Upsert — creates or updates the Workflow row with version="draft".
5. Env var merge — if environment_variable_upserts is provided, variables are merged by ID; otherwise legacy full-replace applies.
6. Agent binding sync — sync_agent_bindings_for_draft() (see below).
7. Draft validation — validate_agent_nodes_for_draft_sync() (lenient: missing bindings silently skipped).
8. Commit + event — session.commit() then app_draft_workflow_was_synced.send(...).

Response: { result: "success", hash, updated_at } — the frontend stores hash as the new syncWorkflowDraftHash 8.

Agent V2 Binding Reconciliation#

Every draft save calls sync_agent_bindings_for_draft which iterates all Agent V2 nodes, deletes bindings for removed nodes, and upserts bindings for present nodes 1.

Validation strictness differs between draft and publish 9:

This asymmetry causes the most common upgrade regression: workflows with Agent nodes created before the V2 binding requirement draft-save successfully but block at publish time 10. The error "Workflow Agent node {id} requires a binding before publishing" is raised at publish validation 9.

Workaround: open the node in the editor, re-select the agent binding, save the draft, then publish. If rebinding fails, delete and recreate the Agent node and reconnect its edges 10.

Collaborative Sync#

Gated by the enable_collaboration_mode system flag. The collaboration layer lives under web/app/components/workflow/collaboration/ 11.

CRDT model — CollaborationManager maintains a LoroDoc with nodesMap and edgesMap. Private _-prefixed keys and selected flags are excluded from CRDT sync to avoid dirtying the document with UI-only state 11.

Leader election — the server assigns one leader per session, preferring visible tabs (graph_active=True). When a tab hides, its requestAnimationFrame loop pauses and CRDT updates queue without applying. The frontend emits graph_view_state on every visibilitychange; the backend atomically demotes a hidden leader under a Redis lock. This was the root cause fixed by PR #38997 ("prevent hidden-tab collaboration leader from saving stale drafts") 12.

Draft persistence — only the elected leader persists to the DB. Follower save requests are routed to the leader via socketio.call() with a 5-second timeout 11.

Local fallback — self-hosted deployments without a WebSocket service activate localDraftFallbackActive when connection to the default socket URL fails, falling back to HTTP-only draft saves 11.

Known Failure Modes#

Celery Worker Backpressure (Editor Stuck in "Queued")#

Since v1.13, workflow execution is offloaded to Celery. When CELERY_WORKER_CONCURRENCY is too low (default: 1), the workflow_based_app_execution queue backs up and preview/test runs never emit SSE events 1 13. The draft sync API call itself returns 200; the hang is in execution, not persistence.

Fix: set CELERY_WORKER_CONCURRENCY=10 (or CELERY_AUTO_SCALE=true) and restart the worker.

Plugin Daemon Version Mismatch#

The plugin daemon must be versioned in lockstep with the Dify API 14. A version mismatch causes all plugin-related calls to fail; the daemon logs "failed to find the version of the plugin sdk". Syncing workflows that reference plugins will silently fail or hang.

Fix: ensure the daemon image tag matches the Dify release (e.g., daemon 0.6.10-local for Dify 1.15.0), then run docker compose exec api flask backfill-plugin-auto-upgrade 14.

Hash Mismatch / Stale Draft (draft_workflow_not_sync)#

If the client's syncWorkflowDraftHash doesn't match the server's stored hash (e.g., after a concurrent save from another tab), the API returns draft_workflow_not_sync. The client automatically fetches a fresh draft copy and resets its local hash 1.

Agent V2 Binding Error at Publish (Upgrade Regression)#

Workflows upgraded from pre-1.15.0 that contain Agent nodes without a WorkflowAgentNodeBinding record will save drafts successfully but fail at publish with WorkflowAgentNodeValidationError 10. See the Agent V2 Binding Reconciliation section above for the workaround.

Key Source Files#