From 5a9c1a4416f289f39b4f57f44f94e2573be2c7a7 Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 16:34:17 +0300 Subject: [PATCH 01/12] test(mongodb): assert PSMDB CR has delete-psmdb-pvc finalizer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add helm-unittest assertion verifying that the rendered PerconaServerMongoDB CR carries percona.com/delete-psmdb-pvc finalizer. Currently fails — establishes red phase before adding the fix. Signed-off-by: Arsolitt --- packages/apps/mongodb/tests/mongodb_test.yaml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/packages/apps/mongodb/tests/mongodb_test.yaml b/packages/apps/mongodb/tests/mongodb_test.yaml index 4154ca25..085030a8 100644 --- a/packages/apps/mongodb/tests/mongodb_test.yaml +++ b/packages/apps/mongodb/tests/mongodb_test.yaml @@ -38,6 +38,23 @@ tests: value: my-mongodb documentIndex: 0 + ################## + # Finalizers # + ################## + + - it: sets delete-psmdb-pvc finalizer for automatic PVC cleanup + release: + name: test-mongodb + namespace: tenant-test + set: + _cluster: + cluster-domain: cozy.local + asserts: + - contains: + path: metadata.finalizers + content: percona.com/delete-psmdb-pvc + documentIndex: 0 + ################## # CR Version # ################## From 53636d616a2049ca1efebf571e0b287275196d69 Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 16:47:53 +0300 Subject: [PATCH 02/12] fix(mongodb): add delete-psmdb-pvc finalizer to PSMDB CR PerconaServerMongoDB CR now carries percona.com/delete-psmdb-pvc finalizer. Percona Operator deletes associated PVCs automatically when the CR is removed, eliminating storage quota leaks. Signed-off-by: Arsolitt --- packages/apps/mongodb/templates/mongodb.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/apps/mongodb/templates/mongodb.yaml b/packages/apps/mongodb/templates/mongodb.yaml index 9273e506..7b04dae6 100644 --- a/packages/apps/mongodb/templates/mongodb.yaml +++ b/packages/apps/mongodb/templates/mongodb.yaml @@ -4,6 +4,8 @@ apiVersion: psmdb.percona.com/v1 kind: PerconaServerMongoDB metadata: name: {{ .Release.Name }} + finalizers: + - percona.com/delete-psmdb-pvc spec: crVersion: 1.21.1 clusterServiceDNSSuffix: svc.{{ $clusterDomain }} From 8eb764dd40fb969e54c72700989a4080610bae6c Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 16:50:13 +0300 Subject: [PATCH 03/12] test(mongodb): tighten finalizer assertion Rename test description to accurately reflect what's verified (template renders the finalizer, not runtime cleanup behavior). Add equal assertion locking the exact finalizer list to catch regressions where a stray finalizer is appended. Signed-off-by: Arsolitt --- packages/apps/mongodb/tests/mongodb_test.yaml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/packages/apps/mongodb/tests/mongodb_test.yaml b/packages/apps/mongodb/tests/mongodb_test.yaml index 085030a8..f1e449e8 100644 --- a/packages/apps/mongodb/tests/mongodb_test.yaml +++ b/packages/apps/mongodb/tests/mongodb_test.yaml @@ -42,7 +42,7 @@ tests: # Finalizers # ################## - - it: sets delete-psmdb-pvc finalizer for automatic PVC cleanup + - it: renders percona.com/delete-psmdb-pvc finalizer on PSMDB CR release: name: test-mongodb namespace: tenant-test @@ -54,6 +54,11 @@ tests: path: metadata.finalizers content: percona.com/delete-psmdb-pvc documentIndex: 0 + - equal: + path: metadata.finalizers + value: + - percona.com/delete-psmdb-pvc + documentIndex: 0 ################## # CR Version # From b9c4c8b4976efa89fe6bff4fb0c3cd676b6cc88a Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 16:52:41 +0300 Subject: [PATCH 04/12] docs(mongodb): document PVC and secret cleanup on uninstall Document that uninstalling the release reclaims PVCs and operator-managed user secrets via the percona.com/delete-psmdb-pvc finalizer. Pre-existing users who relied on orphaned PVCs to recover data should take backups before uninstalling. Signed-off-by: Arsolitt --- packages/apps/mongodb/README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/packages/apps/mongodb/README.md b/packages/apps/mongodb/README.md index 82127c59..4b67f494 100644 --- a/packages/apps/mongodb/README.md +++ b/packages/apps/mongodb/README.md @@ -35,6 +35,15 @@ When `external: true` is enabled: On first install, the credentials secret will be empty until the Percona operator initializes the cluster. Run `helm upgrade` after MongoDB is ready to populate the credentials secret with the actual password. +### Data lifecycle + +When the MongoDB release is uninstalled, the operator reclaims: + +- All PVCs backing the replica set storage (via the `percona.com/delete-psmdb-pvc` finalizer on the `PerconaServerMongoDB` CR). +- Operator-managed user secrets associated with the cluster. + +If you need to retain data, take a backup before deletion. Refer to the [Percona Operator for MongoDB documentation](https://docs.percona.com/percona-operator-for-mongodb/) for backup/restore workflows. + ## Parameters ### Common parameters From 733226ded0959abac315f4aad180cc5c69ee9cdb Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 16:55:07 +0300 Subject: [PATCH 05/12] test(mongodb): cover sharded mode and drop strict equal Replace the equal assertion (which would block adding more finalizers) with a sharded-mode test case ensuring the finalizer is rendered regardless of replicaset/sharded configuration. Signed-off-by: Arsolitt --- packages/apps/mongodb/tests/mongodb_test.yaml | 23 ++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/packages/apps/mongodb/tests/mongodb_test.yaml b/packages/apps/mongodb/tests/mongodb_test.yaml index f1e449e8..2e4c1e80 100644 --- a/packages/apps/mongodb/tests/mongodb_test.yaml +++ b/packages/apps/mongodb/tests/mongodb_test.yaml @@ -54,10 +54,27 @@ tests: path: metadata.finalizers content: percona.com/delete-psmdb-pvc documentIndex: 0 - - equal: + + - it: renders percona.com/delete-psmdb-pvc finalizer in sharded mode + release: + name: test-mongodb + namespace: tenant-test + set: + _cluster: + cluster-domain: cozy.local + sharding: true + shardingConfig: + configServers: 3 + configServerSize: 3Gi + mongos: 2 + shards: + - name: rs0 + replicas: 3 + size: 10Gi + asserts: + - contains: path: metadata.finalizers - value: - - percona.com/delete-psmdb-pvc + content: percona.com/delete-psmdb-pvc documentIndex: 0 ################## From 1a2cb2537671999118e01bd40c4f0dc6ddf58dcd Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 16:57:28 +0300 Subject: [PATCH 06/12] test(mongodb): assert delete-psmdb-pods-in-order finalizer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Upstream Percona Operator v1.21.1 ships both delete-psmdb-pvc and delete-psmdb-pods-in-order in deploy/cr.yaml by default. The pods-in-order finalizer ensures primary-last graceful shutdown, preventing data loss on cluster deletion under load. Currently fails — establishes red phase before adding the finalizer. Signed-off-by: Arsolitt --- packages/apps/mongodb/tests/mongodb_test.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/packages/apps/mongodb/tests/mongodb_test.yaml b/packages/apps/mongodb/tests/mongodb_test.yaml index 2e4c1e80..66efdd35 100644 --- a/packages/apps/mongodb/tests/mongodb_test.yaml +++ b/packages/apps/mongodb/tests/mongodb_test.yaml @@ -54,6 +54,10 @@ tests: path: metadata.finalizers content: percona.com/delete-psmdb-pvc documentIndex: 0 + - contains: + path: metadata.finalizers + content: percona.com/delete-psmdb-pods-in-order + documentIndex: 0 - it: renders percona.com/delete-psmdb-pvc finalizer in sharded mode release: @@ -76,6 +80,10 @@ tests: path: metadata.finalizers content: percona.com/delete-psmdb-pvc documentIndex: 0 + - contains: + path: metadata.finalizers + content: percona.com/delete-psmdb-pods-in-order + documentIndex: 0 ################## # CR Version # From 63239c1617924a538af45ba064b947cf4aea2398 Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 16:59:52 +0300 Subject: [PATCH 07/12] fix(mongodb): add delete-psmdb-pods-in-order finalizer PerconaServerMongoDB CR now carries the upstream-default finalizer pair: percona.com/delete-psmdb-pods-in-order ensures primary-last graceful shutdown before percona.com/delete-psmdb-pvc reclaims storage. This matches deploy/cr.yaml defaults in operator v1.21.1 and prevents potential data loss on deletion under load. Signed-off-by: Arsolitt --- packages/apps/mongodb/templates/mongodb.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/apps/mongodb/templates/mongodb.yaml b/packages/apps/mongodb/templates/mongodb.yaml index 7b04dae6..ece403f6 100644 --- a/packages/apps/mongodb/templates/mongodb.yaml +++ b/packages/apps/mongodb/templates/mongodb.yaml @@ -5,6 +5,7 @@ kind: PerconaServerMongoDB metadata: name: {{ .Release.Name }} finalizers: + - percona.com/delete-psmdb-pods-in-order - percona.com/delete-psmdb-pvc spec: crVersion: 1.21.1 From 80aa1e0dc512efcd3ed875fb43552e5244b62edd Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 17:02:14 +0300 Subject: [PATCH 08/12] docs(mongodb): clarify secret cleanup on uninstall Distinguish between operator-managed secrets reclaimed via the finalizer flow (e.g. internal-*-users) and Helm-managed secrets (credentials, user secrets, s3 backup creds) removed by helm uninstall. The previous wording was ambiguous and could mislead users about which cleanup path applies to which secret. Signed-off-by: Arsolitt --- packages/apps/mongodb/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/packages/apps/mongodb/README.md b/packages/apps/mongodb/README.md index 4b67f494..f746edaa 100644 --- a/packages/apps/mongodb/README.md +++ b/packages/apps/mongodb/README.md @@ -37,10 +37,11 @@ Run `helm upgrade` after MongoDB is ready to populate the credentials secret wit ### Data lifecycle -When the MongoDB release is uninstalled, the operator reclaims: +When the MongoDB release is uninstalled, all storage and credentials are reclaimed: - All PVCs backing the replica set storage (via the `percona.com/delete-psmdb-pvc` finalizer on the `PerconaServerMongoDB` CR). -- Operator-managed user secrets associated with the cluster. +- Internal operator-managed secrets (e.g. `internal--users`) are deleted as part of the same finalizer flow. +- Helm-managed secrets (`-credentials`, `-user-*`, `-s3-creds`) are removed by the standard `helm uninstall`. If you need to retain data, take a backup before deletion. Refer to the [Percona Operator for MongoDB documentation](https://docs.percona.com/percona-operator-for-mongodb/) for backup/restore workflows. From e1b6793a8a62872080970507745df8413964c586 Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 17:04:38 +0300 Subject: [PATCH 09/12] docs(mongodb): enumerate all secrets affected by uninstall The previous wording listed only one operator-managed secret with 'e.g.' prefix. The Percona Operator finalizer flow actually deletes four specific secrets (internal-, internal--users, -mongodb-encryption-key, and the global default percona-server-mongodb-users). Document each by name so operators understand the full deletion blast radius, especially the global default which can collide with externally-created secrets sharing the same name. Signed-off-by: Arsolitt --- packages/apps/mongodb/README.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/packages/apps/mongodb/README.md b/packages/apps/mongodb/README.md index f746edaa..849d3cd8 100644 --- a/packages/apps/mongodb/README.md +++ b/packages/apps/mongodb/README.md @@ -40,8 +40,15 @@ Run `helm upgrade` after MongoDB is ready to populate the credentials secret wit When the MongoDB release is uninstalled, all storage and credentials are reclaimed: - All PVCs backing the replica set storage (via the `percona.com/delete-psmdb-pvc` finalizer on the `PerconaServerMongoDB` CR). -- Internal operator-managed secrets (e.g. `internal--users`) are deleted as part of the same finalizer flow. -- Helm-managed secrets (`-credentials`, `-user-*`, `-s3-creds`) are removed by the standard `helm uninstall`. +- Operator-managed secrets deleted as part of the same finalizer flow: + - `internal-` — internal operator state + - `internal--users` — operator-managed user credentials + - `-mongodb-encryption-key` — at-rest encryption key + - `percona-server-mongodb-users` — global operator users secret (shared default name; if a tenant has a Secret with this exact name in the namespace, it will also be deleted) +- Helm-managed secrets removed by the standard `helm uninstall`: + - `-credentials` — connection string for application code + - `-user-` — per-user passwords + - `-s3-creds` — backup destination credentials (if backups are configured) If you need to retain data, take a backup before deletion. Refer to the [Percona Operator for MongoDB documentation](https://docs.percona.com/percona-operator-for-mongodb/) for backup/restore workflows. From 8e777da24ce537148420a19cdb8d060019a0d57a Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 19:01:25 +0300 Subject: [PATCH 10/12] fix(mongodb): scope users secret to release name The operator deletes spec.secrets.users (defaulted to the global percona-server-mongodb-users) as part of the delete-psmdb-pvc finalizer flow. With the default value, deleting one MongoDB release in a namespace would remove the secret shared with any other MongoDB release there. Override spec.secrets.users with a release-scoped name so each release owns its own secret and uninstalls do not collide. Signed-off-by: Arsolitt --- packages/apps/mongodb/templates/mongodb.yaml | 2 ++ packages/apps/mongodb/tests/mongodb_test.yaml | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/packages/apps/mongodb/templates/mongodb.yaml b/packages/apps/mongodb/templates/mongodb.yaml index ece403f6..c15b3341 100644 --- a/packages/apps/mongodb/templates/mongodb.yaml +++ b/packages/apps/mongodb/templates/mongodb.yaml @@ -10,6 +10,8 @@ metadata: spec: crVersion: 1.21.1 clusterServiceDNSSuffix: svc.{{ $clusterDomain }} + secrets: + users: {{ .Release.Name }}-percona-server-mongodb-users pause: false unmanaged: false image: percona/percona-server-mongodb:{{ include "mongodb.versionMap" $ }} diff --git a/packages/apps/mongodb/tests/mongodb_test.yaml b/packages/apps/mongodb/tests/mongodb_test.yaml index 66efdd35..6bbf27bc 100644 --- a/packages/apps/mongodb/tests/mongodb_test.yaml +++ b/packages/apps/mongodb/tests/mongodb_test.yaml @@ -85,6 +85,23 @@ tests: content: percona.com/delete-psmdb-pods-in-order documentIndex: 0 + ################## + # Secrets # + ################## + + - it: scopes users secret name to the release for namespace isolation + release: + name: test-mongodb + namespace: tenant-test + set: + _cluster: + cluster-domain: cozy.local + asserts: + - equal: + path: spec.secrets.users + value: test-mongodb-percona-server-mongodb-users + documentIndex: 0 + ################## # CR Version # ################## From ff69555dbb2f9f1e020d4a84a42498e0dfc9c832 Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 19:02:40 +0300 Subject: [PATCH 11/12] test(mongodb): rename finalizer assertions to reflect both finalizers Test descriptions said 'percona.com/delete-psmdb-pvc finalizer' while the assertions verified both delete-psmdb-pvc and delete-psmdb-pods-in-order. Use 'Percona finalizers' to match what the tests actually check. Signed-off-by: Arsolitt --- packages/apps/mongodb/tests/mongodb_test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/apps/mongodb/tests/mongodb_test.yaml b/packages/apps/mongodb/tests/mongodb_test.yaml index 6bbf27bc..595940a7 100644 --- a/packages/apps/mongodb/tests/mongodb_test.yaml +++ b/packages/apps/mongodb/tests/mongodb_test.yaml @@ -42,7 +42,7 @@ tests: # Finalizers # ################## - - it: renders percona.com/delete-psmdb-pvc finalizer on PSMDB CR + - it: renders Percona finalizers on PSMDB CR release: name: test-mongodb namespace: tenant-test @@ -59,7 +59,7 @@ tests: content: percona.com/delete-psmdb-pods-in-order documentIndex: 0 - - it: renders percona.com/delete-psmdb-pvc finalizer in sharded mode + - it: renders Percona finalizers in sharded mode release: name: test-mongodb namespace: tenant-test From 660620689669a62764615cc218558899b08c3ce1 Mon Sep 17 00:00:00 2001 From: Arsolitt Date: Tue, 28 Apr 2026 19:05:16 +0300 Subject: [PATCH 12/12] docs(mongodb): refine uninstall behavior and add operator recovery note Update Data lifecycle section to address review feedback: - Reflect the release-scoped users secret naming. - Distinguish operator-driven PVC release from actual PV/data deletion (depends on StorageClass reclaimPolicy). - Document that TLS secrets (-ssl, -ssl-internal) are not reclaimed automatically. - Add a recovery note for the case where the operator is removed before the CR, leaving finalizers unable to run. Signed-off-by: Arsolitt --- packages/apps/mongodb/README.md | 36 ++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-) diff --git a/packages/apps/mongodb/README.md b/packages/apps/mongodb/README.md index 849d3cd8..8b4f6b4e 100644 --- a/packages/apps/mongodb/README.md +++ b/packages/apps/mongodb/README.md @@ -37,18 +37,36 @@ Run `helm upgrade` after MongoDB is ready to populate the credentials secret wit ### Data lifecycle -When the MongoDB release is uninstalled, all storage and credentials are reclaimed: +When the MongoDB release is uninstalled, the operator finalizers reclaim release-scoped resources: -- All PVCs backing the replica set storage (via the `percona.com/delete-psmdb-pvc` finalizer on the `PerconaServerMongoDB` CR). -- Operator-managed secrets deleted as part of the same finalizer flow: +**Reclaimed by the `percona.com/delete-psmdb-pvc` finalizer:** + +- All PVCs backing the replica set storage. Whether the underlying PersistentVolume and on-disk data are actually deleted depends on the StorageClass `reclaimPolicy` (`Delete` removes them, `Retain` leaves them for manual cleanup). +- Operator-managed secrets: + - `-percona-server-mongodb-users` — operator users credentials - `internal-` — internal operator state - - `internal--users` — operator-managed user credentials + - `internal--users` — operator-internal users data - `-mongodb-encryption-key` — at-rest encryption key - - `percona-server-mongodb-users` — global operator users secret (shared default name; if a tenant has a Secret with this exact name in the namespace, it will also be deleted) -- Helm-managed secrets removed by the standard `helm uninstall`: - - `-credentials` — connection string for application code - - `-user-` — per-user passwords - - `-s3-creds` — backup destination credentials (if backups are configured) + +**Reclaimed by `helm uninstall`:** + +- `-credentials` — connection string for application code +- `-user-` — per-user passwords +- `-s3-creds` — backup destination credentials (if backups are configured) + +**Not reclaimed automatically:** + +- TLS secrets `-ssl` and `-ssl-internal` (issued by cert-manager) remain in the namespace after uninstall. Delete them manually if no longer needed. + +**Recovery from a stuck deletion:** + +If the `psmdb-operator` is uninstalled before MongoDB CRs are deleted, the finalizers cannot run and the `PerconaServerMongoDB` CR hangs in `Terminating`. To recover, clear the finalizers manually: + +```bash +kubectl --namespace patch psmdb --type merge --patch '{"metadata":{"finalizers":[]}}' +``` + +Note that this skips the operator-driven cleanup — PVCs and operator-managed secrets will remain orphaned and must be removed manually. If you need to retain data, take a backup before deletion. Refer to the [Percona Operator for MongoDB documentation](https://docs.percona.com/percona-operator-for-mongodb/) for backup/restore workflows.