Onboarding State Management#

Langfuse uses a hybrid PostgreSQL + ClickHouse architecture to determine whether a project is in onboarding (no traces yet) or product state (traces present). The core goal is answering "has this project ever received any traces?" cheaply, without continuously scanning ClickHouse.

Architecture Overview#

Frontend (traces.tsx)
  │
  ├─ session.project.hasTraces = true?
  │ └─ skip polling, render product UI immediately
  │
  └─ poll hasTracingConfigured every 10s (until true)
       │
       └─ tRPC: traces.hasTracingConfigured
            │
            ├─ hasAnyTrace(projectId)
            │ ├─ 1. PostgreSQL projects.hasTraces flag (fast path)
            │ ├─ 2. ClickHouse traces table (SELECT 1 ... LIMIT 1, max_threads=1)
            │ └─ 3. [V4 events-only] ClickHouse events tables fallback
            │
            └─ Fallback: retentionDays > 0 (tracing configured, data expired)

Key Components#

1. PostgreSQL hasTraces Flag (Permanent Cache)#

The projects table carries a hasTraces boolean (default false) added in migration 20260209000000_add_project_has_traces. Once set true, it is never reverted. This immutability means no TTL or cache invalidation logic is needed — the flag is a permanent positive signal.

2. hasAnyTrace() — Core Detection Logic#

Defined in packages/shared/src/server/repositories/traces.ts, the function implements a three-step ordered check:

1. PostgreSQL fast path 1: Read project.hasTraces; return true immediately if set.
2. ClickHouse traces table 2: Run SELECT 1 FROM traces WHERE project_id = ? LIMIT 1 with max_threads: 1 to cap scan cost.
3. Persist positive result 3: On positive ClickHouse result, write hasTraces = true to PostgreSQL using updateMany with a hasTraces: false guard to avoid redundant writes.

3. V4 Events-Only Fallback#

After PR #13955, when neither the PostgreSQL flag nor the ClickHouse traces table contains data, hasAnyTrace checks the ClickHouse events tables (events_full / events_core). This ensures that projects ingesting data exclusively via the V4 events pipeline are not incorrectly shown the onboarding UI. A positive events check also persists hasTraces = true to PostgreSQL via persistHasTracesFlag() 4.

4. tRPC hasTracingConfigured Endpoint#

Defined at web/src/server/api/routers/traces.ts, this query wraps hasAnyTrace() and adds one additional fallback: if no traces are found in any store, it returns true if the project has retentionDays > 0 — indicating the user deliberately configured tracing even if data-retention policies have expired all traces 5.

5. Frontend Polling Logic#

In web/src/pages/project/[projectId]/traces.tsx, the hasTracingConfigured query is configured as:

• refetchInterval: false if project?.hasTraces (session flag already true), otherwise 10_000 ms 6
• initialData: true if session flag is set, skipping the first network round-trip 7
• staleTime: Infinity if flag is true (no revalidation needed), else 0 8

The session project.hasTraces value is populated from PostgreSQL at login time 9, so most returning users never issue a single polling request. New projects poll every 10 seconds until the first trace arrives, at which point the flag persists and polling stops permanently.

When !hasTracingConfigured, the page renders the TracesOnboarding component 10.

V4 Migration Context#

PR #14812 promoted the V4 events-table infrastructure (observations_batch_staging, events_full, events_core, events_core_mv) from ad-hoc dev scripts into versioned ClickHouse migrations (0036–0041). Crucially, it flipped the default value of LANGFUSE_MIGRATION_V4_WRITE_MODE from legacy to events_only, meaning new deployments ingest via the events pipeline by default. The events-only onboarding detection in step 3 above is therefore the standard path for new installations 11.

Performance Motivation#

Before PR #11921, four frontend pages polled ClickHouse every 5–10 seconds. On one measured self-hosted instance, this generated ~330 calls/hour scanning 747M rows over 37 hours — all returning true for a project that had long since been set up. The PostgreSQL caching layer with session propagation reduced this to zero redundant ClickHouse calls for established projects 9.

Key Files#