Eval Template Versioning#

The EvalTemplate model supports two distinct versioning tracks depending on whether a template is project-managed (created by users) or Langfuse-managed (predefined, global templates). Both tracks share the same eval_templates table and version Int field, differentiated by whether projectId is set or null.

Database Schema#

The EvalTemplate model enforces a composite unique constraint:

@@unique([projectId, name, version])

1

This means each (projectId, name, version) triple must be unique. When projectId is null, the row belongs to the global Langfuse-managed pool.

Project Templates: Immutable Version Increment#

When a user creates or updates a project-level eval template, a new immutable version is appended — existing versions are never mutated.

The createTemplate mutation runs inside a transaction:

1. Query all existing versions for the same (projectId, name), ordered by version DESC 2.
2. Pick the highest as latestTemplate 3.
3. Write a new row with version: (latestTemplate?.version ?? 0) + 1 — starting at 1 if none exist 4.

This also covers cloning a Langfuse-managed template into a project: the clone is written with projectId set to the target project and its version counter starts at 1 5.

The allTemplatesForName procedure retrieves the full version history for a template name, returned in descending order, enabling the UI to show version history and allow users to select a specific version.

Managed Evaluators: JSON-Defined Versions#

Langfuse ships 23 built-in evaluators (e.g., Hallucination, Helpfulness, Relevance, RAGAS variants) whose versions are statically declared in worker/src/constants/managed-evaluators.json (imported at line 3 of upsertManagedEvaluators.ts). Each entry in that file includes an explicit version integer field 6.

The upsertManagedEvaluators function syncs this JSON into the database:

• Validates all entries against ManagedEvaluatorSchema (Zod) 7.
• Skips rows whose updatedAt already matches the JSON's updated_at timestamp, unless force=true 8.
• Upserts by stable id — the id and timestamps are stored in the JSON to guarantee deterministic results and stable diffs 9.
• Sets projectId: null for all managed templates 10.

The version field for managed evaluators travels directly from the JSON definition through to the database row 11 — there is no auto-increment. To release a new version of a managed evaluator, a developer increments the version field in the JSON and updates updated_at.

Key Behavioral Differences#

Key Files#