LLM Driver Integration#

RAGFlow maintains two parallel LLM driver stacks: a Python layer (rag/llm/) for document processing, embeddings, and the main chat API, and a Go layer (internal/entity/models/) for the agent model runner. Both expose the same logical operations — chat, embed, rerank — but are implemented independently: the Go layer uses hand-rolled HTTP clients against provider REST APIs, while Python relies on SDK wrappers and LiteLLM.

Go Layer Architecture#

ModelDriver Interface#

All Go providers implement the ModelDriver interface in internal/entity/models/types.go, which declares 17 methods covering the full LLM capability surface: ChatWithMessages, ChatStreamlyWithSender, Embed, Rerank, TranscribeAudio, AudioSpeech, OCRFile, ParseFile, ListModels, Balance, CheckConnection, and others. Providers that don't implement a capability return a "no such method" error rather than omitting the method.

BaseModel: Shared Infrastructure#

BaseModel in internal/entity/models/base_model.go provides the scaffolding shared by every provider driver:

• HTTP client: NewDriverHTTPClient() clones http.DefaultTransport and sets MaxIdleConns=100, MaxIdleConnsPerHost=10, IdleConnTimeout=90s, ResponseHeaderTimeout=2min. No client-wide timeout is set so SSE streams are never cut short; non-streaming calls use context.WithTimeout individually.
• Request building: buildRequestBody() produces an OpenAI-compatible request map with model, messages, stream, temperature, max_tokens, top_p, stop, tools, and tool_choice.
• SSE parsing: ParseSSEStream[T] is a generic parser that errors on malformed JSON events. ParseSSEStreamTolerant[T] silently skips bad frames and is reserved for providers whose upstreams are documented to send invalid frames.
• Tool call accumulation: accumulateToolCallDeltas, setSortedToolCallsResult, and extractToolCalls reconstruct tool-call payloads across streaming delta events.
• Auth helper: BearerAuth() formats the Authorization: Bearer <key> header value, returning an empty string when no key is configured.

Factory and Registry#

ModelFactory.CreateModelDriver() maps 60+ provider names to their constructor functions via a switch statement, falling back to a DummyModel for unknown providers. The singleton ProviderManager reads JSON provider configuration files at startup, instantiates drivers via the factory, and exposes FindProvider, FindModel, and ListModels for runtime lookup.

Provider-specific wrapper types — ChatModel, EmbeddingModel, RerankModel — pair a ModelDriver with task-specific configuration and token usage tracking. ToolConfig defaults to 5 max rounds and 3 retries.

Baidu ERNIE: Go Driver#

File: internal/entity/models/baidu.go — driver name "BaiduYiyan" 1.

The Go Baidu driver is a fully hand-rolled HTTP client. All requests use Authorization: Bearer <api_key> 2 and a URL constructed from a configurable base URL plus a provider-specific suffix.

Thinking/reasoning parameter mapping 3: The driver inspects the model name to pick the right parameter format:

• ernie/qwen prefix → enable_thinking: true|false
• Other models → thinking: {"type": "enabled"|"disabled"}
• deepseek-v4 additionally sets reasoning_effort: "high"|"max"|…

Streaming uses ParseSSEStream, validates that the stream ends with either [DONE] or a non-empty finish_reason to detect truncated streams 4, and appends a [DONE] marker for OpenAI downstream compatibility 5.

Embeddings post to the embedding URL suffix and enforce input-order preservation via the Index field in the response 6.

OCR accepts either a file URL or raw bytes (base64-encoded), auto-detects PDF vs. image via MIME type, and extracts text from layoutParsingResults[0].markdown.text 7.

Not implemented: TranscribeAudio, AudioSpeech, ParseFile, Balance, ListTasks, ShowTask — these return "BaiduYiyan, no such method" .

Baidu ERNIE: Python Driver#

Class: BaiduYiyanChat in rag/llm/chat_model.py, _FACTORY_NAME = "BaiduYiyan".

Unlike the Go driver, the Python layer uses the Qianfan SDK (qianfan.ChatCompletion) rather than direct HTTP 9. Credentials are a JSON blob with yiyan_ak and yiyan_sk keys — incompatible with the Go driver's raw API key format 10.

Parameter translation 11:

• penalty_score = ((presence_penalty + frequency_penalty) / 2) + 1
• max_tokens is stripped (Baidu SDK uses a different field name)

Streaming calls self.client.do(..., stream=True) and iterates over resp.body["result"] chunks 12. The async path wraps the blocking SDK call in asyncio.to_thread() to avoid blocking the event loop 13.

The Python Baidu driver covers chat only; embedding and reranking for BaiduYiyan use separate classes in embedding_model.py and rerank_model.py backed by the same Qianfan SDK.

Python Layer: Broader Provider Support#

Most providers in the Python layer route through LiteLLMBase in chat_model.py. Three dictionaries in rag/llm/__init__.py define the catalog 14:

At import time, rag/llm/__init__.py auto-discovers all _FACTORY_NAME-annotated subclasses of Base across chat_model, embedding_model, and related modules, and registers them in the ChatModel, EmbeddingModel, and other factory dicts 14.

The ALLOWED_GEN_CONF_KEYS allowlist (temperature, max_completion_tokens, top_p, stream, stop, tools, tool_choice, etc.) filters generation config before forwarding to providers, preventing provider rejection errors from RAGFlow-internal metadata keys.

For provider-specific credential handling and API versioning details, see the Azure OpenAI Integration and LLM Provider Integration knowledge base articles.

Cross-Layer Comparison: Baidu ERNIE#

The two drivers are not interchangeable at runtime: the Python layer is invoked by the document-processing and REST API service, while the Go layer is used exclusively by the agent model runner. Credential formats differ between the two, so users configuring BaiduYiyan must supply the correct format for each runtime.