LLM Structured Output (Workflow LLM Node)#

Overview#

The workflow LLM node supports JSON schema-based structured output: users define a schema, Dify enforces it at inference time (either natively or via prompt injection), then parses and repairs the model's response. The feature is toggled by structured_output_enabled on the node data and the schema is stored in structured_output.schema 1.

The core implementation lives in api/core/llm_generator/output_parser/structured_output.py. The single public entry point is invoke_llm_with_structured_output(), which accepts a json_schema mapping and returns either an LLMResultWithStructuredOutput or a generator of LLMResultChunkWithStructuredOutput objects.

Schema Definition (Frontend)#

Users define the output schema in the LLM node panel via a modal with two editing modes: Visual Editor and JSON Schema (raw text) 2. The modal validates schemas client-side (depth limit via JSON_SCHEMA_MAX_DEPTH, Draft-7 compliance, required additionalProperties: false) before saving .

The TypeScript type for a stored schema is SchemaRoot: a top-level object with properties, optional required, and additionalProperties: false. Individual fields (Field) support types including string, number, boolean, object, array, and enum 5.

An AI-assisted schema generator (backed by SYSTEM_STRUCTURED_OUTPUT_GENERATE) lets users describe fields in natural language and receive a valid JSON schema 6.

Native Support vs. Prompt-Based Fallback#

The decision point is model_schema.support_structure_output on the AIModelEntity:

Native path (support_structure_output = True — e.g., GPT-4o, Gemini):

• _handle_native_json_schema() is called.
• The schema is serialized and set as the json_schema model parameter.
• If the model's parameter rules include response_format with a json_schema option, it is also set 7.

Prompt-based fallback (all other models):

• _set_response_format() selects JSON or json_object mode based on model rules.
• _handle_prompt_based_schema() prepends a system message using STRUCTURED_OUTPUT_PROMPT (a template with a {{schema}} placeholder). The serialized schema is injected and the original system prompt (if any) is appended afterward.

Model-Specific Schema Transformations#

Before passing a schema to a native-mode model, _prepare_schema_for_model() applies transformations:

Response Parsing and JSON Repair#

_parse_structured_output() handles all response parsing:

1. Tries TypeAdapter(Mapping).validate_json(result_text) for strict parse.
2. On ValidationError, falls back to json_repair.loads(result_text).
3. If json_repair returns a list (common with reasoning models like DeepSeek-R1 that emit <think>\n\n</think>\n prefixes), the first dict in the list is used 11.
4. Any other result raises OutputParserError.

Streaming vs. Non-Streaming#

Non-streaming: The full response text is parsed once and returned as LLMResultWithStructuredOutput.structured_output 12.

Streaming: Text chunks are accumulated in a generator. Each chunk yields LLMResultChunkWithStructuredOutput without structured output. The final sentinel yield includes the parsed structured_output from the full accumulated text 13.

Key Source Files#