LLM Model Configuration#

LLM model parameters in Langfuse are configured through two mechanisms: a flat modelParams JSON column shared by the EvalTemplate and DefaultLlmModel Prisma models, and per-connection settings stored in LlmApiKeys. Together these cover everything from basic sampling parameters to provider-specific options.

Core TypeScript Types#

The central type hierarchy lives in packages/shared/src/server/llm/types.ts:

• ZodModelConfig — the Zod schema for the modelParams JSON payload. Fields:

  • max_tokens (number, optional)
  • temperature (number, optional)
  • top_p (number, optional)
  • maxReasoningTokens (number, optional)
  • providerOptions (free-form JSON object, optional)

• ModelParams — extends ModelConfig with provider: string, adapter: LLMAdapter, and model: string. This is the runtime type passed into the LLM completion layer.

• UIModelParams — wraps every field in { value, enabled } for UI toggle support (e.g., enabling/disabling top_p individually).

Prisma Models#

EvalTemplate — model_params column#

Stores per-template model configuration for LLM-as-a-Judge evaluators 1:

model String? // model identifier, e.g. "gpt-4o"
provider String? // provider name, e.g. "openai"
modelParams Json? @map("model_params")

DefaultLlmModel — model_params column#

Stores the project-level default model used when no template-level model is specified 2:

provider String
adapter String
model String
modelParams Json? @map("model_params")

DefaultLlmModel has a required FK to LlmApiKeys, linking stored parameters to a specific API key connection.

LlmApiKeys — connection-level settings#

Beyond the API secret, LlmApiKeys carries fields that affect how LLM calls are constructed 3:

extraHeaders are stored encrypted and decrypted at call time via decryptAndParseExtraHeaders, validated as z.record(z.string(), z.string()).

Supported Adapters#

The LLMAdapter enum defines six adapters:

Built-in model lists (openAIModels, anthropicModels, vertexAIModels, googleAIStudioModels) are maintained in types.ts. Azure and Bedrock have empty built-in lists because model names are deployment-specific. The first entry in each list is the default model used when configuring a new API key 4.

providerOptions — Provider-Specific Parameters#

modelParams.providerOptions is a free-form JSON object 5 for passing parameters that go beyond the standard temperature/top_p/max_tokens trio — for example reasoning_effort or service_tier for OpenAI, or thinkingLevel/thinkingBudget for Google models (docs).

In fetchLLMCompletion, the object is mapped to the adapter's native property name:

Google adapters validate providerOptions through googleProviderOptionsSchema before spreading, narrowing accepted keys to thinkingBudget and thinkingLevel.

Reasoning Model Special Cases#

For OpenAI reasoning models (o1, o3, o4-mini, gpt-5 series), isOpenAIReasoningModel switches the token-limit parameter from maxTokens to maxCompletionTokens. The mapping openAIModelToReasoning 6 tracks which model IDs are reasoning models.

For Anthropic and Google adapters that emit "thinking" blocks, fetchLLMCompletion forces method: "functionCalling" when calling withStructuredOutput to prevent reasoning blocks from corrupting JSON schema parsing 7.

Key Entry Points#