Secret Watch and Reload Mechanism#

Overview#

CloudNativePG's secret watch-and-reload mechanism is a two-layer pipeline:

1. Cluster controller (operator) — watches Kubernetes Secret events, filters them down to only those relevant to a cluster, and enqueues a cluster reconciliation.
2. Instance manager (in-pod) — on every reconciliation, directly reads referenced secrets from the API server, writes updated files to disk, and signals PostgreSQL to reload if anything changed.

Two categories of secrets receive this treatment:

• Cluster-owned secrets — created and managed by the operator (e.g. TLS certificates). These are automatically watched via Kubernetes ownerReference.
• User-managed secrets with the cnpg.io/reload label — external secrets not owned by the cluster that a user wants to propagate automatically (e.g. a custom CA, superuser credentials).

Both paths converge on the same outcome: a cluster reconciliation loop runs, the instance manager re-reads the relevant secrets, overwrites on-disk cert/key files atomically, and calls pg_reload_conf() if any file changed.

Cluster Controller: Predicate Filtering and Routing#

Watch Registration#

The ClusterReconciler registers a Secret watch with a custom predicate in SetupWithManager:

Watches(
    &corev1.Secret{},
    handler.EnqueueRequestsFromMapFunc(r.mapSecretsToClusters()),
    builder.WithPredicates(secretsPredicate),
)

secretsPredicate — Which Secrets Are Admitted#

secretsPredicate passes a Secret event only when isUsefulClusterSecret returns true 1. That function calls isOwnedByClusterOrSatisfiesPredicate, which accepts the secret if either:

• It has a controller ownerReference pointing to a Cluster resource, or
• It carries the label cnpg.io/reload (checked by hasReloadLabelSet)

The cnpg.io/reload label name is exported as utils.WatchedLabelName. Presence of the key is sufficient — the value is ignored.

mapSecretsToClusters() — Routing Events to Clusters#

When a secret passes the predicate, mapSecretsToClusters determines which clusters to reconcile:

1. Lists all clusters in the secret's namespace (or all clusters across namespaces for the operator-wide monitoring secret).
2. Calls cluster.UsesSecret(secret.Name) on each cluster to filter down to those that actually reference the secret.

UsesSecret — Reference Detection#

UsesSecret returns true if the secret name matches any of:

A cluster that does not reference the secret is silently skipped — no spurious reconciliation is triggered.

Instance Manager: Secret Reading and Reload#

The instance manager runs its own reconciliation loop (InstanceReconciler.Reconcile in instance_controller.go). Unlike the operator, it reads secrets directly from the API server (not from an informer cache), ensuring it always sees the current content.

Early Certificate Load#

Before the admission guard runs, EnsureServerCertificateLoaded reads the server TLS secret if the in-memory certificate is absent. This keeps the kubelet's liveness/readiness probes working even when cluster validation fails and the rest of the loop is short-circuited.

RefreshSecrets — File Refresh#

certificateReconciler.RefreshSecrets is called every reconcile loop 2. It performs direct client.Get calls for each certificate secret and writes files atomically. The secrets it handles:

• Server TLS certificate and key (refreshServerCertificateFiles)
• Streaming replication user certificate (refreshReplicationUserCertificate)
• Client CA (refreshClientCA)
• Server CA (refreshServerCA)
• Barman endpoint CA (refreshBarmanEndpointCA)

RefreshSecrets returns a bool (changed) that is true if any file was actually rewritten on disk.

refreshConfigurationFiles — Config Reload Flag#

In parallel, refreshConfigurationFiles checks PostgreSQL configuration files and also returns a boolean. The two flags are OR-ed together into a single reloadNeeded variable 3.

Reload Trigger#

After reconciling the cluster role and checking fencing, the loop reaches 4:

if reloadNeeded && !restarted {
    r.instance.Reload(ctx)
    r.processConfigReloadAndManageRestart(ctx, cluster)
}

restarted is true if a full PostgreSQL restart already happened during this loop iteration (e.g. due to an in-place primary restart). The guard prevents a redundant reload after a restart.

Credential Sync#

After the database is confirmed running, refreshCredentialsFromSecret re-reads the superuser/application password secrets and issues an ALTER ROLE if the password changed.

End-to-End Flow#

Key Files and Entry Points#

Adding a New Secret to Watch#

To make the operator and instance manager react to a new secret:

1. Add the secret name to the UsesSecret switch/checks in api/v1/cluster_funcs.go so the cluster controller enqueues reconciliation when it changes.
2. In the instance manager, add a cli.Get call (inside RefreshSecrets or a new helper) that reads the secret and writes its content to disk. Return true from that helper if the file changed, so it feeds into reloadNeeded.

cnpg.io/reload Label for External Secrets#

Any secret not owned by a cluster can opt into automatic watch-and-reload by setting the label cnpg.io/reload: "" (value is irrelevant) 5. The operator will detect changes and trigger reconciliation, provided the secret is also referenced by UsesSecret. The label alone is not sufficient — the cluster must actually use the secret.