Reasoning Tag Filtering#

Dify extracts or preserves <think>…</think> reasoning tags emitted by chain-of-thought models (e.g., DeepSeek-R1, QwQ) through a two-mode architecture that applies uniformly to LLM nodes and Agent nodes in workflows.

Modes#

tagged is the default in both LLMNodeData and AgentNodeData to preserve backward compatibility with existing workflows.

How Detection and Separation Work#

LLM Node (split_reasoning)#

The actual tag-splitting logic lives in graphon.nodes.llm.reasoning.split_reasoning — an external library imported by Dify 1. It runs a case-insensitive regex (<think[^>]*>(.*?)</think>) over the completed text after streaming finishes 2:

• separated mode: All <think> blocks are removed from the answer; their inner content is returned as reasoning_content on the LLMResult and propagated through ModelInvokeCompletedEvent 2.
• tagged mode: Text is returned unchanged; reasoning_content is empty.

During streaming, both modes forward raw chunks — including <think> tags — to the frontend so the real-time "thinking" panel continues to work. The post-stream cleanup only affects values exposed to downstream nodes 2.

Agent Node (ThinkStreamFilter)#

Agent nodes reuse the ThinkStreamFilter from graphon via message_transformer.py. The filter is stateful across chunks, so it correctly reassembles tags that span multiple streamed segments (e.g., "<thi" + "nk>…</think>") 3. In separated mode:

1. Text chunks are routed through ThinkStreamFilter; clean text is emitted as StreamChunkEvent.
2. Detected reasoning is emitted as StreamReasoningEvent on a separate channel.
3. After stream completion, text_filter.finalize() flushes any buffered partial tags 3.

Event and Pipeline Architecture#

Reasoning content travels through the backend on its own event channel, never mixing with the answer stream:

• QueueReasoningChunkEvent — internal queue event for out-of-band reasoning deltas; carries reasoning: str, from_node_id, and is_final 4.
• ReasoningChunkStreamResponse — SSE response wrapper that sends the reasoning chunk to clients; parallel to text_chunk, only emitted in separated mode 5.
• Pipeline handlers — both advanced_chat and workflow generate_task_pipeline.py implement _handle_reasoning_chunk_event to forward these events and persist terminal reasoning_content per LLM node ID in task_state.metadata.reasoning[node_id] 6.

Special Case: Structured Output Parsing#

The structured output parser handles a common artifact from reasoning models: DeepSeek-R1 and similar models sometimes wrap JSON in <think>\n\n</think>\n. When json_repair.loads() returns a list (the <think> block parsed as an element), the parser extracts the first dict, effectively skipping the reasoning prefix 7.

Frontend Configuration#

• reasoning-format-config.tsx — a Switch toggle on the LLM node panel lets users switch between tagged and separated modes 8.
• panel-output-section.tsx — declares reasoning_content as a named string output variable on the LLM node, available for wiring to downstream nodes regardless of mode.
• types.ts — type: reasoning_format?: 'tagged' | 'separated' on LLMNodeType.
• Agent node UI mirrors LLM node: reasoning_format field added to AgentNodeType and its panel 3.

Key Source Files#