Generate Policy UpdateRequest Lifecycle#

An UpdateRequest (UR) is a Kyverno CRD that decouples the admission webhook from the background controller for generate (and mutateExisting) policies. When a matching resource is admitted, the webhook creates a UR; the background controller later processes it to create, sync, or delete downstream resources.

UR states: Pending → (background controller processes) → Completed (deleted) or Failed (reset to Pending by the reconciler for retry).

Two creation paths#

The background controller's ProcessUR iterates each RuleContext entry in the UR and calls applyGenerate per trigger. The UR spec is defined in pkg/api/kyverno/v2; newGenerateUR and addRuleContext are the main builders.

Admission Webhook Path and Race Condition#

handleBackgroundApplies spawns two goroutines — one for mutateExisting, one for generate — each with an independent context.Background() and a 30-second timeout. The comment in the code explicitly notes that the HTTP request context is cancelled when the webhook handler returns, necessitating independent contexts to avoid premature cancellation.

The race is structural: in handlers.go, handleBackgroundApplies is called with a dummy wait.Group that is never waited on 1, then ResponseSuccess is returned immediately 2. UR creation goroutines remain in-flight after the trigger resource is already admitted.

For the CEL-based GeneratingPolicy path (gpol/handler.go), UR creation is explicitly fire-and-forget: the handler launches a goroutine at line 55, then returns the admission response at line 145 with no synchronization.

Consequence: if UR creation fails or is slow, the trigger resource exists in the cluster but no UR is queued. Retries rely on the policy controller's forceReconciliation tick rather than a guaranteed per-admission delivery.

generateExisting — Phantom Triggers and Missing CEL Filtering#

When a policy with generateExisting: true is created or updated, the policy controller calls handleGenerateForExisting. It lists trigger candidates via getTriggers, which only uses MatchResources (kind/namespace selectors) and cannot evaluate CEL matchConditions 3.

Bug (issue #16556): For GeneratingPolicy (CEL-based), getGpolTriggers creates UpdateRequests for every resource of the matching GVK before CEL filtering is applied. In large clusters this produces thousands of phantom URs for resources that will ultimately be rejected by the CEL conditions when the background controller processes them.

For Kyverno-native policies, handleGenerateForExisting does call engine.ApplyBackgroundChecks per trigger 4, which filters non-matching resources. But GeneratingPolicy has no equivalent pre-filter.

Suggested fix (from the issue): pass the full GeneratingPolicy into getGpolTriggers and evaluate matchConditions before enqueueing URs.

Duplicate UR Accumulation#

Bug (issue #16152): The generate path has no per-(policy, trigger) deduplication. updaterequests.go Generate() only enforces a global count threshold (updateRequestThreshold configmap key) before calling Create 5. Every forceReconciliation tick and every policy-update event creates a fresh UR, regardless of whether a prior UR for the same policy is still Pending.

Compare to the mutateExisting path, which calls listMutateURs(policyKey, trigger) and skips creation if a UR already exists — no equivalent guard exists in the generate path.

Root cause is label design: GenerateLabelsSet only stores the policy name, so label-based dedup across triggers is impossible without a schema change. MutateLabelsSet encodes both policy and trigger resource, enabling exact-match dedup.

Mitigation in place: splitUR caps each UR at generateBatchSize = 100 RuleContext entries 6, preventing etcd "request too large" errors. However it does not prevent new URs from being created alongside still-pending ones.

Proposed fix: PR #16157 adds per-policy dedup analogous to the mutateExisting path.

Failed UR Retry Loop and Queue Starvation#

ProcessUR iterates all RuleContext entries, collects per-trigger errors, and at the end calls updateStatus with the combined error. If any trigger fails, the entire UR is marked Failed 7.

The reconcileURStatus handler in the background controller immediately flips Failed → Pending with no backoff 8. The UR informer's updateUR event handler enqueues without rate-limiting unless the state is Completed or Skip. The result: a single failing trigger inside a 1000-entry UR causes the controller to reprocess all 1000 triggers in a hot loop, which with 10 workers can fully starve the background-controller queue.

Proposed fix: PR #16176 splits large RuleContext into configurable batches (default 100, configured via updateRequestMaxBatchSize in the Kyverno ConfigMap) and keeps Failed URs in Failed state so they are rate-limited rather than immediately re-queued.

Key files and entry points#