Token Usage Enrichment#

Token usage enrichment is the process by which Langfuse resolves and populates usage_details (token counts) and cost_details (prices) on observation records during ingestion. The pipeline runs inside the worker after events are consumed from the queue and lives in IngestionService.

The two core private methods are:

Enrichment Pipeline#

provided_model_name?
  └─ yes → findModel (L1 cache → Redis → Postgres)
  └─ no → model = null

model + no user-provided usage + level ≠ ERROR?
  └─ yes → tokenCountAsync (worker thread pool)
           └─ fails → tokenCount (synchronous fallback)
  └─ no → use provided values as-is, synthesize total if missing

model found + pricingTiers available?
  └─ yes → matchPricingTier → calculateUsageCosts
  └─ no → cost_details = provided or empty

1. Model Lookup Gate (findModel)#

Tokenization and cost calculation both require a resolved model. getGenerationUsage calls findModel only when provided_model_name is present; otherwise internalModel is null and automatic tokenization is skipped entirely.

findModel resolves models through a three-layer cache:

1. L1 (in-process TTL cache) — default 10 s, no invalidation 1
2. L2 (Redis) — caches model + pricing tiers when Redis is configured 2
3. Postgres — regex match_pattern field on the Model table 3

Negative lookups are cached with a sentinel (NOT_FOUND_TOKEN) to avoid repeated DB hits for unknown models 4.

2. User-Provided Values (getUsageUnits)#

getUsageUnits normalizes all values in provided_usage_details to numbers first (guarding against ClickHouse returning UInt64 as strings). If any user-provided usage key is present, automatic tokenization is skipped entirely 5. When the user supplies individual keys (e.g., input, output) without a total, the total is synthesized by summation 6.

3. Automatic Tokenization Fallback#

Automatic tokenization only runs when all three conditions hold 5:

• A model was resolved (model != null)
• provided_usage_details is empty
• Observation level is not ERROR

The primary path uses tokenCountAsync, which offloads work to a non-blocking worker thread pool. On failure, it falls back to the synchronous tokenCount 7. If both paths fail, ingestion continues with empty usage_details rather than blocking the record 8.

Both tokenizers dispatch on model.tokenizerId to pick the right tokenizer — e.g., OpenAI tiktoken or Claude's tokenizer 9.

4. Pricing Tier Matching and Cost Calculation#

After usage units are resolved, matchPricingTier selects the applicable pricing tier by evaluating non-default tiers in priority order using AND-logic regex conditions on usage detail keys, falling back to the default tier if none match 10.

calculateUsageCosts then computes costs. The same user-preference rule applies: if any provided_cost_details key is set, all automatic cost calculation is skipped and user values are used directly 11. Otherwise, costs are computed as price × units per usage type from the matched tier 12.

Key Source Files#