Controller Requeue Semantics#
Overview#
CloudNativePG controllers use two complementary mechanisms to drive state convergence: watch-triggered requeues (an API-server event causes controller-runtime to enqueue the object) and timed requeues (ctrl.Result{RequeueAfter: β¦}). Understanding which mechanism fires under which conditions is essential when tracing why a controller does or does not re-run after a state transition.
Status-Only Patches and the Cluster Watch#
Why a no-op patch must be suppressed#
The main ClusterReconciler watches Cluster objects via For(&apiv1.Cluster{}) with no predicate . This means every write to the Cluster resource β including a status patch β can re-enqueue the reconciler.
If a component rewrites an unchanged status it still issues an API call, and the resulting watch event wakes the reconciler again, creating a self-sustaining loop. The plugin-status hook in internal/controller/plugins.go guards against this by skipping the Status().Patch call when the new plugin status is semantically equal to the stored one. A test comment in internal/controller/plugins_test.go names this pattern explicitly:
"re-writing an unchanged status wakes the reconciler through its own Cluster watch, which has no predicate, so the loop sustains itself."
Even after the no-op guard, the function still returns RequeueAfter: 5s to continue polling the plugin β demonstrating that a timed requeue can stand in for a watch event when the underlying patch would be redundant.
PatchWithOptimisticLock: built-in no-op guard#
The shared PatchWithOptimisticLock function in pkg/resources/status/patch.go compares the current status with the desired status before calling Status().Patch. If they are semantically equal, it skips the patch entirely . This is the standard path for Cluster status updates during normal reconciliation, and it prevents spurious self-requeues across all callers.
Cluster Watch Predicates on Secondary Watches#
When a secondary resource type references the Cluster (e.g. DatabaseRole, EndpointSlice), its Cluster watch is set up with predicate.GenerationChangedPredicate{} so that status-only Cluster changes do not trigger those watches . The comment in cluster_controller.go explains the rationale:
"The cluster only consumes spec.passwordSecret (to maintain the instance RBAC), so status-only changes are irrelevant here."
The same predicate is used on the EndpointSlice watch and on the Database/Publication/Subscription instance-manager controllers' Cluster watches .
Already-Reconciled Gate and Replica Transitions#
The generation-equality gate#
Instance-manager controllers (Database, Publication, Subscription, DatabaseRole) exit early once object.Generation == object.Status.ObservedGeneration. Before PR #10871 this gate was a hard exit; after the fix it delegates to handleReplicaRoleTransition .
Why timed polling is required here#
Cluster status-only updates (e.g. updating CurrentPrimary during a demotion) do not pass GenerationChangedPredicate{} and therefore do not trigger the Cluster watch on the managed-resource controllers. The comment in handleReplicaRoleTransition states this directly:
"Keep polling while the designated primary awaits the promotion, or while the primary recorded in the cluster status is still settling (e.g. a failover concurrent with the demotion): status-only updates don't retrigger the cluster watch."
The function therefore returns an explicit RequeueAfter (default: databaseReconciliationInterval = 30s) instead of relying on a watch event .
Transition logic#
handleReplicaRoleTransition in internal/management/controller/common.go implements three cases :
| Cluster state | Applied | Designated primary? | Outcome |
|---|---|---|---|
| Not a replica | nil or false | β | proceed=true (re-run reconciliation) |
| Not a replica | true | β | proceed=false, no requeue (already applied, wait for spec change) |
| Replica | non-nil + is designated primary | yes | Mark status Unknown, then RequeueAfter |
| Replica | anything | yes OR Applied != nil | RequeueAfter (poll until settled) |
| Replica | nil + not designated primary | β | Return zero result (not this pod's job) |
Crucially, the ownership record (status.applied) is preserved during demotion so a conflicting object cannot take over the managed Postgres entity; re-application is driven off that applied status once the cluster is promoted back .
Cluster-to-resource watch mapper#
mapClusterToManagedResources in common.go is the mapper wired into the Cluster watch for Database/Publication/Subscription controllers. It filters events to the pod's own cluster by name and namespace, then lists all resources of the target type in the cluster's namespace and enqueues only those whose GetClusterRef().Name matches. This means a Cluster spec change (generation bump) re-enqueues all managed resources immediately, while a status-only Cluster change does not.
Key Source Files#
| File | Role |
|---|---|
pkg/resources/status/patch.go | PatchWithOptimisticLock β no-op guard for Cluster status patches |
internal/controller/plugins.go | Plugin status hook β skip patch + unconditional timed requeue pattern |
internal/controller/cluster_controller.go | SetupWithManager β Cluster For() watch (no predicate) and secondary watches with GenerationChangedPredicate |
internal/management/controller/common.go | handleReplicaRoleTransition, mapClusterToManagedResources, markAsUnknown |
internal/management/controller/database_controller.go | Database controller reconcile loop and SetupWithManager with Cluster watch |