ChatML Message Rendering#

Overview#

Langfuse normalizes LLM inputs/outputs from multiple providers into a unified ChatML message structure before rendering them in the UI. This pipeline lives in two layers:

1. Shared package (packages/shared/src/utils/chatml/): provider-specific adapters + core normalization logic
2. Web package (web/src/components/trace2/components/IOPreview/): React hooks and components that consume normalized messages

The key decision point is canDisplayAsChat: if the normalized data validates as a ChatML array, the UI renders chat-bubble-style messages; otherwise it falls back to a PrettyJsonView (JSON/table view).

Data Model#

The canonical message type is defined by ChatMlMessageSchema in packages/shared/src/utils/IORepresentation/chatML/types.ts. Key fields:

Unknown fields are collected into the json passthrough key by the schema's final transform 1, making them available for PrettyJsonView rendering without polluting the typed message surface.

Two schema variants exist 2:

• BaseChatMlMessageSchema / ChatMlMessageSchema: full frontend schema with transforms (used for rendering)
• SimpleChatMessageSchema: lightweight backend schema used only for compact representation extraction

Adapter Pipeline#

selectAdapter() in adapters/index.ts picks the right provider adapter for each observation using a priority-ordered list:

LangGraph → AI SDK → OpenAI → Gemini → Microsoft Agent → Pydantic AI → Semantic Kernel → Generic (fallback)

3

Each adapter implements a ProviderAdapter interface with two methods:

• detect(ctx): returns true if the adapter recognizes the data — checked against ctx.framework, ctx.observationName, metadata hints, and schema structure
• preprocess(data, kind, ctx): transforms the provider-specific format into a shape that ChatMlArraySchema can parse

Key Adapters#

OpenAI (adapters/openai.ts):

• Handles Chat Completions ({messages, tools}), Responses API ({output: [...]}), and single messages 4
• Flattens nested tool_calls[].function.{name, arguments} → flat {id, name, arguments} 5
• Converts function_call / function_call_output Responses API item types to standard ChatML roles 6
• Extracts reasoning items into the thinking array 7
• Detection explicitly rejects LangGraph, Semantic Kernel, Pydantic AI, and LangChain-typed messages to avoid false positives 8

AI SDK (adapters/aisdk.ts):

• Targets Vercel AI SDK v5; detected via scope.name === "ai" or attributes["operation.name"] starting with "ai." 9
• Normalizes tool-call / tool-result content part arrays into flat tool_calls arrays and tool_call_id fields 10
• Splits multi-result tool messages into separate ChatML messages 11

Generic / Gemini (adapters/generic.ts):

• Handles Google Gemini/VertexAI format: {candidates: [{content: {parts, role}}]} outputs and {contents: [...]} inputs 12
• Maps "model" role → "assistant" and "parts" → "content" 13
• Serves as the universal fallback (detect always returns true) 14

Core Normalization Functions#

packages/shared/src/utils/chatml/core.ts exports the functions that wire adapters to Zod schemas:

React Layer: useChatMLParser Hook#

useChatMLParser (web/src/components/trace2/components/IOPreview/hooks/useChatMLParser.ts) is the single entry point for the UI:

1. Calls deepParseJson on raw Prisma.JsonValue inputs (or accepts pre-parsed data from a Web Worker to avoid ~100ms duplicate work) 15
2. Runs normalizeInput + normalizeOutput with a NormalizerContext containing metadata and observation name
3. Combines messages via combineInputOutputMessages
4. Extracts tool definitions (deduped by name), numbers tool call invocations on output messages only, and sorts tools by call frequency 16
5. Returns canDisplayAsChat — true when either input or output normalized successfully and messages.length > 0 17

IOPreviewPretty: Chat vs. JSON Routing#

IOPreviewPretty is the top-level pretty-view component for trace observations:

• Calls useChatMLParser with optional pre-parsed data 18
• Guards markdown rendering with a size estimate against MARKDOWN_RENDER_CHARACTER_LIMIT to prevent UI freeze on very large payloads 19
• If canDisplayAsChat: renders <ChatMessageList> with chat bubbles 20
• Otherwise: renders <JsonInputOutputView> (two PrettyJsonView panels) 21

ChatMessage Component#

ChatMessage renders a single normalized message. It branches on message shape using helpers from chat-message-utils.ts:

The markdown/JSON toggle is controlled by shouldRenderMarkdown passed from the parent. Passthrough JSON data (message.json) triggers a secondary toggle button to switch between the formatted message and its raw JSON 22.

Key Files Reference#