Score Configuration Management#

Score configs are reusable templates that define a scoring dimension's name, data type, and constraints. They standardize the schema that annotators and evaluators score against, ensuring consistent and comparable data across human annotation, annotation queues, and programmatic score ingestion.

Score configs are project-scoped and immutable after creation (data type cannot be changed). They can be archived and restored but not deleted. The dataType field is locked on creation — the UI disables the selector when editing an existing config 1.

Data Types#

Four data types are supported 2:

TEXT configs are intentionally excluded from experiments, LLM-as-a-Judge evaluations, and score analytics because free-form text cannot be meaningfully aggregated.

Numeric Range Constraints#

For NUMERIC configs, minValue and maxValue are both optional floats stored on the ScoreConfig row 3. The key validation rule: if both are provided, maxValue must be strictly greater than minValue — enforced in validateNumericRangeFields.

This check runs in two places:

1. Client-side, via validateScoreConfigUpsertFormInput before form submission, using the shared ScoreConfigValidationSchema.
2. Server-side, inside the tRPC update mutation via validateDbScoreConfigSafe, which merges the incoming patch with the persisted config before re-validating.

Key Source Files#

Creating / Editing Configs#

Via UI: Navigate to Project Settings → Scores / Evaluation and click Create new score config. The UpsertScoreConfigDialog handles both create and update flows — it defaults to NUMERIC with no min/max 4. Changing the data type clears incompatible fields (e.g., switching to TEXT clears minValue/maxValue) 5.

Via API/SDK: Use POST /api/public/score-configs or the api property on the Langfuse client. The tRPC create mutation maps directly to this 6. Updates use the update mutation 7, which only allows changing name, description, isArchived, minValue, maxValue, and categories — not dataType.

Both paths write an audit log entry with before/after snapshots .

Integration with Evaluation Workflows#

• Human Annotation: At least one score config must exist before annotations can be submitted via the UI.
• Annotation Queues: Queues reference specific score configs to define which scoring dimensions annotators will fill in.
• Programmatic ingestion (API/SDK): Scores can optionally reference a config via configId. When provided, Langfuse validates the score's name, data type, and value against the config's constraints at ingestion time.
• Score name is constrained to 1–35 characters 9.

RBAC#

Two RBAC scopes gate access 10:

• scoreConfigs:read — list and fetch configs
• scoreConfigs:CUD — create, update, archive