Volume Snapshot Backup#

Overview#

Volume snapshot backup is one of CloudNativePG's backup methods (BackupMethodVolumeSnapshot), managed entirely by the operator — not the instance manager 1. It takes point-in-time CSI VolumeSnapshot objects for each PVC attached to a selected Pod, then tracks their lifecycle to completion. The feature supports both online (PostgreSQL stays up) and offline (Pod fenced) modes.

Entry point: pkg/reconciler/backup/volumesnapshot/reconciler.go — the Reconciler struct and its Reconcile method.

Backup Lifecycle (6 Steps)#

The reconciler drives every snapshot backup through six sequential steps 2:

1. Prepare — puts PostgreSQL into backup mode (online) or fences the Pod (offline). Skipped if snapshots already exist.
2. Create snapshots — creates one VolumeSnapshot per PVC, then requeues after 10 s to let the external-snapshotter controller pick them up.
3. Wait for provisioning — polls until every snapshot has been claimed by a VolumeSnapshotContent; annotates each with SnapshotStartTime/SnapshotEndTime once provisioned.
4. Finalize — exits backup mode / unfences the Pod, sets BackupPhaseFinalizing, and patches backup status with snapshot metadata.
5. Wait for ready — polls until every snapshot's ReadyToUse is true.
6. Complete — sets BackupPhaseCompleted, annotates snapshots with start/end times, WAL position, and backup-label data.

Each step returns a ctrl.Result to requeue when work remains, or nil to advance to the next step in the same reconcile loop.

Configuration Precedence (Backup vs. Cluster)#

GetVolumeSnapshotConfiguration in api/v1/backup_funcs.go applies a simple override: start with cluster.Spec.Backup.VolumeSnapshot, then overwrite with any field set on the Backup spec:

• Backup.Spec.Online overrides cluster.Spec.Backup.VolumeSnapshot.Online
• Backup.Spec.OnlineConfiguration overrides the cluster-level online configuration

GetOnlineOrDefault provides the final effective online flag, defaulting to true (online) when nothing is explicitly set.

Snapshot Error Handling & Retry Logic#

CSI Errors — Deadline-Based Retry (PR #11132)#

Previously, handleSnapshotErrors tried to classify CSI errors as permanent vs. transient via text-pattern matching, but provider-specific messages (OCI 409 "backup in progress", GCP 502, etc.) fell through and caused premature failures 3. The fix in PR #11132 removed all classification logic: every VolumeSnapshot.Status.Error is now retried uniformly until a deadline expires 4.

The deadline is controlled by the annotation backup.cnpg.io/volumeSnapshotDeadline on the Backup resource (default: 10 minutes) 5. The reconciler records the first failure timestamp in backup.Status.PluginMetadata and requeues every 10 s until the deadline is exceeded 6.

Network/Transport Errors — Immediate Requeue (PR #11069)#

Transport-level failures to the instance manager (dial timeout, connection refused, TLS error) are detected by isNetworkErrorRetryable and cause an immediate 5 s requeue 7, not a backup failure. This matters most during the finalize step, where snapshots are already provisioned and a single network blip would otherwise discard an otherwise complete backup. 8

Snapshot Collision Handling (PR #11071)#

The operator lists existing snapshots through a cached client, so the cache can lag behind a snapshot created in a previous reconcile cycle. When Create returns AlreadyExists, handleSnapshotCreateError applies three-tier logic:

1. Same backup owner (label backup.cnpg.io/name matches) → log and return nil; the reconciler requeues and picks it up via label selector.
2. Cache still hides the object (Get returns NotFound after AlreadyExists) → treat as harmless; requeue and re-evaluate when the cache catches up.
3. Foreign snapshot (owned by a different backup) → surface as an error to avoid silently adopting unrelated objects.

9

Key Files & References#