Timeline History File Handling#
PostgreSQL uses .history files (e.g., 00000002.history) to record where each timeline branched from its parent. During WAL restore, CloudNativePG's walrestore component must carefully control which .history files it delivers to avoid two failure classes: crash-looping replicas that download incompatible future-timeline history files, and silent data loss when bootstrap recovery cannot access the history files it needs.
The central validation entry point is validateTimelineHistoryFile in internal/cmd/manager/walrestore/cmd.go. It is called before every WAL restore attempt — both plugin-based and in-tree — so the protection applies uniformly .
Validation Logic#
The function applies rules in this order :
- Not a
.historyfile → allow (pass-through for normal WAL segments). - Pod is
CurrentPrimaryorTargetPrimary→ allow all history files. Primaries need unrestricted access to walk the full timeline genealogy. cluster.Status.TimelineID == 0→ allow all history files. A zeroTimelineIDmeans no primary has promoted yet (bootstrap recovery from an external archive). The split-brain guard this function enforces requires an active primary, so blocking is unnecessary and harmful at this stage.fileTimeline > clusterTimeline→ reject (ErrWALNotFound). A replica must not download history files for timelines ahead of the cluster's established timeline; stale files from a previous cluster life or a split-brain scenario could cause mismatch errors.- Otherwise → allow.
TimelineID is stored in cluster.Status.TimelineID (defined as ClusterStatus.TimelineID in api/v1/cluster_types.go) . Timeline IDs are parsed from history filenames using ParseTimelineFromHistoryFilename in pkg/postgres/wal.go.
Key Fixes and Their Versions#
Future-timeline blocking for replicas — v1.27.3 / v1.28.1 (PR #9650)#
Replicas could crash-loop when stale .history files from a previous cluster life remained in the WAL archive . PR #9650 introduced the fileTimeline > clusterTimeline guard, originally only for in-tree Barman Cloud restores.
Extended to plugin-based restores — v1.27.4 / v1.28.2 (PR #9849)#
The original PR #9650 guard applied only to in-tree restores; plugins could bypass it. PR #9849 moved validateTimelineHistoryFile to execute before any restore attempt, closing the gap for CNPG-I plugin users .
TimelineID reset after major version upgrade — v1.27.3 / v1.28.1 (PR #9830)#
pg_upgrade always resets the database timeline to 1, but the operator was not reflecting this in cluster.Status.TimelineID. If a cluster had been on timeline 2 before the upgrade, replicas would try to restore the 00000002.history file — incompatible with the freshly upgraded system — producing fatal errors like "requested timeline is not a child of this server's history". The fix calls status.SetTimelineID(1) in pkg/reconciler/majorupgrade/reconciler.go immediately after upgrade completion .
Bootstrap recovery with unset timeline — v1.28.4 / v1.29.2 (PR #10818)#
The most impactful fix. During bootstrap recovery from an external archive, the recovering pod is not yet CurrentPrimary or TargetPrimary, and cluster.Status.TimelineID is 0. The pre-PR #10818 guard unconditionally rejected all .history files in this state, preventing PostgreSQL from building the timeline genealogy needed to interpret cross-timeline WAL segments. The consequence was silent data loss: recovery stopped at the base backup's original timeline and dropped all transactions committed on later timelines .
A concrete trigger: after a failover, the new timeline's first WAL segment often contains page headers from both the old and new timelines. If a backup is taken immediately after failover, its redo pointer falls inside this mixed-segment. Recovery must read the .history files to understand the split; without them it emits FATAL: could not locate required checkpoint record .
The fix is a single early-return in validateTimelineHistoryFile: when clusterTimeline == 0, allow all history files . Confirmed fixed in CNPG ≥ 1.29.2 by independent user reproduction . Release note: .
Affected Files#
| File | Role |
|---|---|
internal/cmd/manager/walrestore/cmd.go | Core validation logic (validateTimelineHistoryFile) |
pkg/postgres/wal.go | ParseTimelineFromHistoryFilename — parses TLI from filename |
pkg/reconciler/majorupgrade/reconciler.go | Calls SetTimelineID(1) post-upgrade |
pkg/resources/status/transactions.go | SetTimelineID helper |
internal/cmd/manager/walrestore/cmd_test.go | Unit tests covering all validation branches |
Related Issues#
- Issue #10422 — Bootstrap recovery fails with
FATAL: could not locate required checkpoint recordwhen the redo WAL segment spans a timeline boundary post-failover; root-caused to the blocked.historydelivery during bootstrap . Fixed in v1.29.2. - Issue #11219 —
.historyfile archival failures during the switchover window leave replicas without timeline genealogy; unrelated to the validation logic but causes similar symptoms. - Issue #11114 — Replicas crash-loop with "Refusing to restore future timeline history file" errors when the needed history file is missing from the archive but a future-timeline file is present.
For diagnostic context, the validation function emits Warning-level log entries when it refuses a file ("Refusing to restore future timeline history file") and Trace-level entries when it allows one.