Dosu LogoDosu Logo
Ask
Join our Discord
langfuse/langfusePublic
Langfuse
Documentslangfuse/langfuse
LLM Model Configuration
LLM Model Configuration
Type
Topic
Status
Published
Created
Jul 16, 2026
Updated
Jul 16, 2026
Created by
Dosu Bot
Updated by
Dosu Bot

LLM Model Configuration#

LLM model parameters in Langfuse are configured through two mechanisms: a flat modelParams JSON column shared by the EvalTemplate and DefaultLlmModel Prisma models, and per-connection settings stored in LlmApiKeys. Together these cover everything from basic sampling parameters to provider-specific options.


Core TypeScript Types#

The central type hierarchy lives in packages/shared/src/server/llm/types.ts:

  • ZodModelConfig — the Zod schema for the modelParams JSON payload. Fields:

    • max_tokens (number, optional)
    • temperature (number, optional)
    • top_p (number, optional)
    • maxReasoningTokens (number, optional)
    • providerOptions (free-form JSON object, optional)
  • ModelParams — extends ModelConfig with provider: string, adapter: LLMAdapter, and model: string. This is the runtime type passed into the LLM completion layer.

  • UIModelParams — wraps every field in { value, enabled } for UI toggle support (e.g., enabling/disabling top_p individually).


Prisma Models#

EvalTemplate — model_params column#

Stores per-template model configuration for LLM-as-a-Judge evaluators :

model String? // model identifier, e.g. "gpt-4o"
provider String? // provider name, e.g. "openai"
modelParams Json? @map("model_params")

DefaultLlmModel — model_params column#

Stores the project-level default model used when no template-level model is specified :

provider String
adapter String
model String
modelParams Json? @map("model_params")

DefaultLlmModel has a required FK to LlmApiKeys, linking stored parameters to a specific API key connection.

LlmApiKeys — connection-level settings#

Beyond the API secret, LlmApiKeys carries fields that affect how LLM calls are constructed :

FieldPurpose
provider / adapterProvider name and interface type (e.g. openai, anthropic)
baseURLCustom endpoint override
customModelsExtra model IDs beyond built-in lists
withDefaultModelsWhether to include the built-in model list
extraHeadersEncrypted additional HTTP headers
extraHeaderKeysPlaintext header key names
configBedrock / VertexAI-specific JSON config

extraHeaders are stored encrypted and decrypted at call time via decryptAndParseExtraHeaders, validated as z.record(z.string(), z.string()).


Supported Adapters#

The LLMAdapter enum defines six adapters:

Adapter valueNotes
openaiGPT / o-series, and OpenAI-compatible providers via baseURL
azureAzure OpenAI deployments
anthropicClaude family
bedrockAWS Bedrock Converse API
google-vertex-aiGoogle Vertex AI
google-ai-studioGoogle AI Studio / Gemini API

Built-in model lists (openAIModels, anthropicModels, vertexAIModels, googleAIStudioModels) are maintained in types.ts. Azure and Bedrock have empty built-in lists because model names are deployment-specific. The first entry in each list is the default model used when configuring a new API key .


providerOptions — Provider-Specific Parameters#

modelParams.providerOptions is a free-form JSON object for passing parameters that go beyond the standard temperature/top_p/max_tokens trio — for example reasoning_effort or service_tier for OpenAI, or thinkingLevel/thinkingBudget for Google models (docs).

In fetchLLMCompletion, the object is mapped to the adapter's native property name:

AdapterMapped to
OpenAI / AzuremodelKwargs
AnthropicinvocationKwargs
BedrockadditionalModelRequestFields
VertexAI / GoogleAIStudioSpread directly after schema validation via googleProviderOptionsSchema

Google adapters validate providerOptions through googleProviderOptionsSchema before spreading, narrowing accepted keys to thinkingBudget and thinkingLevel.


Reasoning Model Special Cases#

For OpenAI reasoning models (o1, o3, o4-mini, gpt-5 series), isOpenAIReasoningModel switches the token-limit parameter from maxTokens to maxCompletionTokens. The mapping openAIModelToReasoning tracks which model IDs are reasoning models.

For Anthropic and Google adapters that emit "thinking" blocks, fetchLLMCompletion forces method: "functionCalling" when calling withStructuredOutput to prevent reasoning blocks from corrupting JSON schema parsing .


Key Entry Points#

FilePurpose
packages/shared/prisma/schema.prismaEvalTemplate, DefaultLlmModel, LlmApiKeys model definitions
packages/shared/src/server/llm/types.tsModelParams, ZodModelConfig, LLMAdapter, model lists
packages/shared/src/server/llm/fetchLLMCompletion.tsAdapter instantiation and providerOptions dispatch
packages/shared/src/server/services/DefaultEvaluationModelService/DefaultEvalModelService.tsValidates config via live test call before saving DefaultLlmModel
LLM Connections docsUser-facing configuration guide
Documents
Agent Sandbox Runtime
Annotation Form Components
API Key Management
Authentication Email Handling
Background Migration Timeout Configuration
Blob Storage Export
BullMQ Worker Lifecycle
Chat Prompt Configuration
ChatML Message Rendering
CJK Input and Unicode Handling
ClickHouse Backfill
ClickHouse Full-Text Search
ClickHouse Migrations
ClickHouse Query Design
ClickHouse Query Execution
ClickHouse Version Compatibility
Dashboard Chart Rendering
Dashboard Query Backend Architecture
Dashboard Widget Versioning
Data Masking
Database Upsert and Uniqueness Constraints
Dataset Item Processing Pipeline
Eval Job Execution
Eval Output Schema
Eval Template Versioning
Evaluation Queue Architecture
Evaluator Configuration and Status Management
Events Table Architecture
Events Table Query Routing
Experiments
Feature Flag System
Filter State Management
HTTP Proxy Configuration
Lambda MicroVM Sandbox
LangGraph Integration
LLM Model Configuration
LLM-as-a-Judge Evaluation
Lossless JSON Parsing
MCP Server Integration
MCP Tool Schema Design
Media Token Rendering
Mixpanel Worker Integration
Model Pricing Configuration
NextAuth OAuth Integration
Observation Data Loading
Observation Eval Scheduling
Onboarding State Management
OTel Attribute Serialization
OTel GenAI Message Ingestion
OTel Ingestion and Trace Hierarchy
OTel Token Usage Processing
Redis Client Management
Redis Retry Strategy
Redis Sentinel Integration
Score Configuration Management
Self-Hosted Deployment
Session Score Aggregation
Token Usage Enrichment
Token Usage Storage
Trace Heatmap Visualization
Trace-Level Token Aggregation
Usage Cost Calculation
V4 Data Pipeline Migration
Webhook Reliability