LLM Configuration and Selection#

RAGFlow's LLM configuration operates at two levels: a static binding stored on each chat assistant (dialog), and an optional per-request runtime override applied during inference. Both paths converge on resolve_model_config, the central resolution function.

Chat Assistant LLM Binding (llm_id)#

Every Dialog record carries two LLM reference fields 1:

Generation parameters are stored alongside the model reference in llm_setting 2, which defaults to {"temperature": 0.1, "top_p": 0.3, "frequency_penalty": 0.7, "presence_penalty": 0.4, "max_tokens": 512}.

On create (POST /chats), if llm_id is not provided it falls back to tenant.tenant_llm_id — the tenant's system-wide default chat model 3.

Model Reference Formats#

llm_id accepts two reference formats, resolved by resolve_model_config:

1. UUID — matches a TenantModel.id; resolved directly via get_model_config_by_id.
2. Composite name — parsed right-to-left by split_model_name as {model_name}@{instance_name}@{provider_name}. The right-anchored split preserves model names that themselves contain @ (e.g. quantization suffixes like text-embedding-v1@q8_0@lmstudio@LM-Studio). If only one @ separator is present, instance_name defaults to "default".

resolve_model_config tries UUID first; on LookupError it falls back to composite-name resolution 4.

Runtime Per-Request LLM Override#

Both the native chat API (POST /chat/completions) and the OpenAI-compatible endpoint (POST /openai/<chat_id>/chat/completions) support overriding the dialog's bound model at request time.

Native API — pass llm_id in the request body 5:

1. If llm_id is present, get_api_key is called to verify the tenant has a valid credential for that model. On failure, the request is rejected with "Cannot use specified model {chat_model_id}." 6.
2. dia.llm_id is replaced in-memory for the duration of the request; the stored dialog record is not mutated.
3. If llm_id is absent and the dialog has no model bound, the tenant's default llm_id is used as a final fallback 7.

OpenAI-compatible API — the model field in the request body drives selection 8:

• If model is the literal string "model" (the placeholder), the dialog's stored llm_id is used unchanged.
• Any other value is validated via _validate_llm_id (which calls resolve_model_config) and then API-key-checked with get_api_key. On success, dia.llm_id is overridden for the request.

Validation Functions#

Key Source Files#

Related Articles#

• Model Provider Architecture — explains the three-tier Provider → Instance → Model hierarchy (TenantModelProvider, TenantModelInstance, TenantModel) and the model_type bit flags that underpin resolve_model_config.
• Model Selection UI — covers ModelTreeSelect, SystemSetting, and tenant default model management.