Bedrock Tool Call Streaming#

Overview#

AWS Bedrock emits tool_call_chunks in a non-standard pattern that differs from most LLM providers: only the first chunk for each tool call carries the id and name fields; all subsequent ("continuation") chunks for the same tool call have id: null and name: null. The @assistant-ui/react-langgraph adapter handles this via a two-tier matching strategy in the chunk-accumulation layer, and the downstream ToolInvocationTracker has an exactly-once contract that governs how mid-stream anomalies (including args regressions) are absorbed.

Chunk Protocol#

A standard streaming provider sends a complete id and name on every chunk. Bedrock sends:

Because id/name on continuation chunks are null, naive string-match merging fails.

Normalization Layer (normalizeLangGraphTupleMessage)#

normalizeToolCallChunk is the entry gate. It:

• Requires only index to be a number — any chunk lacking a numeric index is dropped.
• Coerces null/non-string id and name to "" instead of rejecting the chunk 1.

This coercion is what makes Bedrock's null-identity continuation chunks survive into the accumulator. The LangChainToolCallChunk type enforces index: number; id: string; name: string 2, so callers downstream always receive strings — never null.

Accumulation Layer (appendLangChainChunk)#

appendLangChainChunk merges each incoming AIMessageChunk into the running accumulated message.

First-chunk bootstrap#

When there is no prior AI chunk, the function converts tool_call_chunks directly into tool_calls via the chunkToToolCall helper, which normalizes args/args_json into a partial_json string and a parsed args object 3.

Continuation-chunk matching (two-tier)#

For subsequent chunks, the merge loop applies:

1. Exact id match — finds the existing tool call by non-empty id === chunk.id.
2. Index fallback — if id is empty and chunk.index is present, matches by index where either the accumulated entry or the incoming chunk has no id 4.

If neither tier matches, the chunk is treated as a new tool call. Once matched, the partial_json is concatenated safely (guarding against undefined on existing.partial_json) and the id is promoted from whichever side has it 5.

LangChainToolCall.index field#

LangChainToolCall was extended with an optional index?: number 6 so the accumulated tool call retains the index needed for the fallback lookup on later continuation chunks.

ToolInvocationTracker — Exactly-Once Contract#

After the LangGraph adapter produces a normalized ThreadMessage, the ToolInvocationTracker in @assistant-ui/core processes it 7.

Core guarantee: streamCall (and execute) fires exactly once per logical toolCallId, regardless of how the snapshot mutates afterward.

Args regressions (edge case A.2)#

Bedrock's streaming can produce snapshots where a later argsText is shorter than what was already accumulated — for example, if a stream restart delivers an earlier fragment. The tracker detects this as a regression (new argsText is not a prefix of the already-streamed text), logs a warning in non-production, and does not restart the stream. The controller retains whatever args prefix it already sent; subsequent snapshots that re-converge to a valid prefix can still flow.

Result delivery during regression (known limitation)#

If a snapshot has both a regressed argsText and a backend result on the same tool call, a race occurs: setResponse closes argsText before the result chunk is enqueued, so the args-text-finish path attempts to parse a stale accumulated value, fails, and emits a parse-error result that beats the real backend result to the reader's response promise. The entry.hasResult guard suppresses double-fire at the onResult callback level, but reader.response.get() may already have resolved with the parse error. This is documented as a known limitation and requires a fix in ToolCallStreamControllerImpl.setResponse.

Other relevant state transitions#

See EDGE_CASES.md for the full state-transition catalog.

Key Files#

Fix history: PR #3780 introduced the null id/name coercion and index-based fallback matching (released in @assistant-ui/react-langgraph@0.13.10).