Stebakov · SRE / DevOps
← All posts

Moving CloudNativePG to the Barman Cloud plugin without splitting the archive

· Also in ქართული, Русский

CNPG is removing native barmanObjectStore support. The switch has no valid intermediate state, the database had no base backups at all, and one path decides whether point-in-time recovery still works.

In my home lab a two-instance PostgreSQL 16 cluster, run by CloudNativePG, backs a side-project app and archives to MinIO on a separate VM. CloudNativePG deprecated spec.backup.barmanObjectStore, and the operator says so plainly in an admission warning: native Barman Cloud support will be removed in 1.31.0. The replacement is the Barman Cloud plugin.

Scoping the change turned up two problems before I touched Postgres.

Problem one: cert-manager

The plugin ships a self-signed Issuer and two Certificates for mTLS with the operator, so cert-manager is a hard prerequisite. It was not installed.

Problem two: there were no backups

WAL archiving reported ContinuousArchiving: True and a seven-day retention policy was set. But no ScheduledBackup and no Backup object had ever existed. WAL archiving without a base backup is not a backup: segments replay forward from a base, and there was nothing to replay onto. Both signals were green.

No valid intermediate state

The API makes the shape of the change for you:

spec.plugins: Invalid value: ["barman-cloud.cloudnative-pg.io"]:
Cannot enable a WAL archiver plugin when barmanObjectStore is configured

The old field and a WAL-archiver plugin cannot coexist, so the switch cannot be staged. The ObjectStore, Cluster.spec.plugins, the removal of spec.backup and ScheduledBackup.method: plugin all have to land in one apply.

The order

  1. Make backups exist first. A daily ScheduledBackup with immediate: true. Migrating the backup mechanism of a database that has never produced a restorable backup leaves nothing to fall back on, and nothing to prove the migration preserved.
  2. Install cert-manager, then the plugin, and check each one functionally: a throwaway self-signed Certificate must reach Ready=True.
  3. One atomic commit for the switch.
  4. Prove it with a fresh backup taken by the plugin, not with a condition.

The decision that mattered most

I left destinationPath byte-for-byte identical. Barman lays out an object store as <destinationPath>/<serverName>/, and the server name defaults to the cluster name, so keeping the path means the plugin continues the same archive. The ObjectStore status then showed one continuous recovery window across both mechanisms: the first recoverability point came from a backup taken by the old code, the last successful backup from the plugin.

Pointing the plugin at a new prefix would have left the base backups in one folder and every new WAL segment in another. Nothing reports that. Both halves look healthy, and the split surfaces only when you try a point-in-time restore.

How I verified it

  • Cluster phase went through “Primary instance is being restarted without a switchover” and back to healthy in four to five minutes. Application traffic was unaffected, but this is a maintenance window, not a config edit.
  • pg_stat_archiver from Postgres itself, after pg_switch_wal() so the check is about now: archived count rising, failed count frozen.
  • A Backup with METHOD plugin and PHASE completed, listed by its fully qualified name, backups.postgresql.cnpg.io. With Longhorn installed, the short name backup resolves to Longhorn’s CRD.

Traps on the way

  • kubectl apply --dry-run=server failed on the correct manifest with the same “cannot enable” error. The object had been created by Argo CD with server-side apply, so client-side merge did not know the old field should be removed. Server-side dry-run with Argo’s field manager passed.
  • The plugin sidecar is a native init container with restartPolicy: Always. A check that only reads spec.containers concludes it was never injected.
  • ScheduledBackup.method defaults to barmanObjectStore. Forget to set it and the schedule points at a mechanism the cluster no longer has, while the object still looks fine.
  • Retention moves to the ObjectStore. Leave it on the Cluster and the policy is silently lost.

Takeaways

  • WAL archiving without base backups is not a backup, however green it looks.
  • When two configurations cannot coexist, the change must be a single atomic apply.
  • Keep the destination path identical to continue the same archive.
  • Prove a backup mechanism with a backup it produced, not with a status condition.
  • A backup nobody has restored is still a hypothesis. A restore drill is the next step.