CloudNativePG Pod Scheduling and Recreation#

CNPG handles two related scenarios that require destroying and recreating a pod rather than an in-place restart: unschedulable pods (Kubernetes cannot place the pod on any node) and primary restarts where the pod spec must change (requiring pod deletion without a switchover). Both paths live in the cluster reconciler and interact with RWO PVC detach/reattach timing.

Key source files:

processUnschedulableInstances()#

Defined at cluster_controller.go:1003–1064. Called from reconcileResources() after deleteTerminatedPods() completes, before the allInstancesAreActive() gate.

Detection: utils.IsPodUnschedulable(pod) returns true when a pod is in PodPending phase and has a PodScheduled=False condition with reason Unschedulable. Pods with a DeletionTimestamp are skipped.

For each unschedulable pod, the function takes one of two actions:

1. Spec change needed — if isPodNeedingRollout() returns required=true, calls upgradePod() with the reason "recreating unschedulable pod: <reason>" and requeues after 1 second 1.

2. Maintenance window with PVC reuse disabled — if cluster.IsNodeMaintenanceWindowInProgress() && !cluster.IsReusePVCEnabled(): deletes the pod, then calls persistentvolumeclaim.EnsureInstancePVCGroupIsDeleted() to remove the entire PVC group, and requeues after 1 second so the informer cache can register the deletions 2.

If neither condition applies (no spec change, not in a qualifying maintenance window), the function takes no action and the pod remains unschedulable until conditions change.

PhaseInplaceDeletePrimaryRestart#

This phase is set by updatePrimaryPod() when PrimaryUpdateMethod == Restart but the change cannot be applied in-place (i.e., the pod spec must be recreated, not just Postgres reloaded). The sequence 3:

1. RegisterPhase(PhaseInplaceDeletePrimaryRestart, reason) — sets cluster phase immediately.
2. upgradePod() — deletes the primary pod directly, without triggering a switchover first.

Contrast this with PhaseInplacePrimaryRestart (in-place Postgres restart via annotation, no pod deletion) and the normal multi-instance rolling-upgrade path (switchover first, then upgrade the demoted pod).

Wait guard in handleSwitchover(): While the cluster is in PhaseInplaceDeletePrimaryRestart, handleSwitchover() blocks any new switchover attempt until cluster.Status.ReadyInstances == cluster.Spec.Instances. Once all instances are ready, the phase is transitioned to PhaseHealthy 4.

This guard prevents a redundant failover from firing while the primary pod is being recreated — the operator knows the temporary loss of the primary is intentional and expected.

When is in-place impossible? The inPlacePossible flag passed to updatePrimaryPod() is set by the rollout checkers (see isInstanceNeedingRollout() in cluster_upgrade.go). A Postgres config change that only requires pg_reload_conf is marked in-place capable; a pod-spec change (image update, volume config, etc.) is not.

RWO Volume Detach/Reattach During Pod Recreation#

ReadWriteOnce (RWO) PVCs can only be mounted on one Kubernetes node at a time. When a pod is deleted and Kubernetes reschedules it to a different node, the volume must fully detach from the old node before the new pod can start. CNPG does not control this detach timing — that is handled by the Kubernetes storage layer and the CSI driver — but the operator's design accounts for the window:

• After deleting a pod (in either processUnschedulableInstances() or upgradePod()), the reconciler requeues with a 1-second delay rather than immediately creating the replacement, giving the informer cache time to register the deletion 2 5.
• During the detach window, the replacement pod will be in Pending state. The allInstancesAreActive() check 6 sets PhaseWaitingForInstancesToBeActive and requeues at 1-second intervals until the pod becomes active.
• The PhaseInplaceDeletePrimaryRestart wait guard 7 prevents a switchover from firing during this window, which would otherwise race with the primary coming back.

Maintenance window behavior affects whether the volume moves at all:

For PhaseInplaceDeletePrimaryRestart outside a maintenance window (normal rolling update), the recreated primary pod reuses its existing PVC and is scheduled to the same node — so RWO detach is not an issue in practice. A cross-node move only occurs when the original node is no longer available (node failure or drain), in which case Kubernetes CSI detach timeouts determine how long the new pod waits before mounting.