Chat Completion API#

POST /api/v1/chat/completions is RAGFlow's primary endpoint for sending user messages to a chat assistant (Dialog) and receiving RAG-augmented responses. It is implemented in session_completion and drives both the web UI and public API integrations.

The endpoint handles:

• Session creation/resumption and history management
• Per-request LLM and generation-parameter overrides
• Streaming (SSE) and non-streaming response modes
• Dynamic interpolation of prompt template variables ({knowledge}, {date}, custom keys)

Request Parameter Schema#

The handler normalizes the request in _normalize_completion_messages and extracts flags before forwarding.

Generation overrides (temperature, top_p, frequency_penalty, presence_penalty, max_tokens) are extracted from the request body via pop_generation_config and applied to the dialog's llm_setting through merge_generation_config.

No chat_id falls back to an ephemeral dialog with direct-chat defaults (_build_default_completion_dialog), meaning no KB retrieval and an empty system prompt.

Any remaining keys in req after standard parameters are consumed are forwarded as **kwargs to the RAG agent — this is how named template variables (e.g. quote, custom {key} interpolations) reach dialog_service.py 5.

Request Forwarding: From HTTP to rag_agent#

After parameter extraction, the handler calls rag_agent(dia, msg, stream, session_id=session_id, **req) 6. The remaining req dict (stripped of routing/session parameters) is passed as **kwargs to rag_agent in dialog_service.py.

Inside rag_agent, the execution path is:

1. LLM resolution — resolves the dialog's llm_id to a model config (chat or vision), with fallback to tenant default 7.
2. Retrieval — runs vector/keyword search against linked knowledge bases using dialog.top_n, top_k, similarity_threshold, and vector_similarity_weight. Optional sub-paths include SQL (field-map), Tavily web search, knowledge graph, and TOC-enhanced retrieval 8.
3. Prompt assembly — interpolates the system prompt template (see next section), appends citation instructions if quote=True, trims to token budget via message_fit_in 9.
4. LLM inference — streams or collects the model response, yielding intermediate chunks with answer, reference, final keys.
5. Session persistence — after streaming completes, ConversationService.update_by_id saves the updated message list and references 10.

Float sanitization: _sanitize_json_floats recursively replaces NaN/Infinity values in every outgoing SSE payload to avoid RFC 8259 violations from similarity scores 11.

Prompt Template Variable Interpolation#

The system prompt in dialog.prompt_config["system"] is treated as a Python format string. Named variables are declared in prompt_config["parameters"] as objects {key, optional} 12.

Built-in variables:

Custom variables are passed from the caller as extra keys in the request body. In rag_agent:

1. Required parameters missing from kwargs raise KeyError: "Miss parameter: <key>" 15.
2. Optional parameters missing from kwargs are replaced with a space in the system template 16.
3. After all substitutions, the template is rendered via prompt_config["system"].format(**kwargs) 17.

Default prompt config for KB-backed chats includes the {knowledge} and {date} parameters pre-declared 12. For direct (no-KB) chats, the defaults are an empty system, no parameters, and quote: False 18.

When the assistant is created or updated, _apply_prompt_defaults auto-injects the knowledge parameter if {knowledge} appears in the system template and kb_ids are set 19.

Streaming Response Format#

When stream=true (default), the response is Content-Type: text/event-stream. Each SSE event is:

data: {"code": 0, "message": "", "data": { <answer_chunk> }}

The data field carries an accumulated (not delta) answer object: answer, reference, session_id, message_id, chat_id 20. The final event carries "data": true to signal stream completion.

For non-streaming (stream=false), a single JSON object is returned in {"code": 0, "message": "", "data": <answer>}.

Legacy mode (legacy: true) rebuilds the older accumulated-string format by prepending <think>…</think> markers from the start_to_think/end_to_think event flags 4, for clients that haven't migrated to the current streaming protocol.

OpenAI-Compatible Endpoint & Deprecated Routes#

POST /api/v1/openai/<chat_id>/chat/completions (openai_api.py) provides a drop-in OpenAI Chat Completions interface scoped to a specific chat_id.

Key differences from the native endpoint:

• URL: chat_id is in the path, not the request body.
• model field: The literal string "model" means "use the dialog's stored model"; any other value overrides the LLM for the request 21.
• Message content: Handles both string and array-of-parts formats (OpenAI spec).
• References: Controlled via extra_body.reference (boolean) and extra_body.reference_metadata ({include, fields}).
• Response format: Returns OpenAI-shaped objects (id, object, created, model, usage, choices) rather than the {code, message, data} envelope.

backward_compat.py maps old routes to their current equivalents 22: