Find a file
Aleksei Sviridkin 6498c61f68
fix(postgres-operator): block HelmRelease Ready until the cnpg webhook actually serves (#2482)
## What this PR does

Closes a bootstrap race where the `cnpg-webhook-service` Service gets
its EndpointSlice populated and the data plane (kube-proxy / Cilium)
programmed a second or two after `helm install --wait` on
`cozy-postgres-operator` declares the HelmRelease Ready. Any HelmRelease
that `dependsOn: postgres-operator` and creates a
`postgresql.cnpg.io/v1` resource in that window (cozy-keycloak, tenant
Postgres apps) has kube-apiserver's call to `mcluster.cnpg.io` fail with

```
Internal error occurred: failed calling webhook "mcluster.cnpg.io":
Post "https://cnpg-webhook-service.cozy-postgres-operator.svc:443/...":
dial tcp <svc-ip>:443: connect: connection refused
```

which fails the downstream release's install.

The fix is a post-install / post-upgrade Helm hook (ServiceAccount +
ClusterRole + ClusterRoleBinding + Job) that probes `/readyz` on the
webhook through the apiserver service proxy. Apiserver's service proxy
uses the same endpoint-resolution and apiserver-initiated pod dial as
the admission webhook path, so once `/readyz` answers through the proxy,
the subsequent admission call will also succeed. Helm `--wait` blocks
the install from completing until the Job exits 0, so the HelmRelease
Ready condition does not lie anymore.

Hardened per review:

- RBAC scoped to a single `services/proxy` resourceName
(`https:cnpg-webhook-service:webhook-server`) — the exact string the
apiserver URL path parser expects for the Service proxy subresource.
- A drift-guard helm-unittest test renders the vendored cnpg Service
template and fails if its `metadata.name` / `ports[0].name` diverge from
the literals in the hook, so a future `make update` that renames the
service forces this template to be updated in the same change.
- Image digest-pinned (`clastix/kubectl:v1.32@sha256:…`) with a
`renovate:` annotation; `imagePullPolicy: IfNotPresent` when
digest-pinned, `Always` when tag-only.
- Job runs under PSA-restricted-compatible securityContext (non-root,
seccomp RuntimeDefault, readOnlyRootFilesystem, drop ALL caps).
- `activeDeadlineSeconds` is derived from `maxAttempts × sleepSeconds +
60s` so a values override raising retries does not get silently cut by a
fixed deadline.
- `backoffLimit: 2` (configurable) so a transient pod-level failure
(image pull rate limit, OOM, CNI hiccup) does not fail the whole
HelmRelease.
- On timeout the Job prints the last `kubectl get --raw` stderr so the
operator can distinguish DNS / refused / 401 / TLS from the Job logs.

21 helm-unittest assertions cover ordering, RBAC/URL parity, subchart
drift, image policy, retry-loop bounds, securityContext, and the
deadline-scaling invariant. Wired via `make test` in
`packages/system/postgres-operator/Makefile`.

First surfaced on #2470 E2E run 24862782568 where
`cozy-keycloak/keycloak` failed with the exact signature above. Not tied
to that PR's branch — this is independently applicable to `main`.

### Release note

```release-note
fix(postgres-operator): add a post-install readiness gate that blocks the HelmRelease from reporting Ready until the cnpg admission webhook actually serves through the cluster Service, preventing "dial tcp <svc-ip>:443: connect: connection refused" on the first HelmRelease that depends on postgres-operator (keycloak, tenant Postgres apps).
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added webhook readiness validation during install/upgrade to block
completion until the admission webhook is reachable.
* New configuration options for the readiness probe image and
retry/timeouts.

* **Tests**
* Introduced a comprehensive test suite validating rendered manifests,
probe behavior, RBAC, image rendering, and retry/timeout logic.

* **Chores**
* Added a test entry point to the project Makefile to run chart/unit
tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-28 15:14:36 +03:00
.gemini docs: adopt Conventional Commits across contributing docs 2026-04-13 22:50:08 +03:00
.github ci(api): add codegen drift check (#2463) 2026-04-28 12:41:48 +05:00
api [vm-instance] Fix PortList not filtering ingress ports (#2501) 2026-04-28 11:17:59 +02:00
cmd fix(kube-ovn): scope kubeovn-plunger cache and RBAC to its namespace 2026-04-21 17:13:27 +02:00
dashboards feat(monitoring): add MongoDB Grafana dashboards 2026-03-05 22:47:53 +03:00
docs [vm-instance] Add externalAllowICMP knob, drop in-PR changelog 2026-04-28 08:37:13 +02:00
examples/backups/vmi feat(backups): restore vmi to copy in another namespace 2026-04-09 14:06:33 +04:00
hack test(hack): rename remediation-guard bats test to match what it pins 2026-04-16 22:32:24 +03:00
img Cozystack logo for dark GitHub theme (#9) 2024-02-09 11:02:41 +01:00
internal feat(operator): add per-package upgradeCRDs policy for HelmRelease 2026-04-27 11:19:29 +03:00
packages fix(postgres-operator): block HelmRelease Ready until the cnpg webhook actually serves (#2482) 2026-04-28 15:14:36 +03:00
pkg fix(kubernetes): history guard non-empty check + nits from review 2026-04-16 21:50:10 +03:00
tools/openapi-gen [docs] Added openapi generation tool 2026-03-25 15:57:25 +05:00
.coderabbit.yaml [ci] Enable CodeRabbit incremental reviews 2026-04-13 13:32:15 +03:00
.gitignore feat(application): add WorkloadsReady condition and Events tab 2026-04-15 17:20:45 +03:00
.pre-commit-config.yaml ci(api): broaden codegen drift trigger paths and detect untracked files 2026-04-23 22:19:35 +05:00
ADOPTERS.md Update ADOPTERS.md by adding new adopter 2025-12-18 16:45:06 +03:00
AGENTS.md docs(agents): use full path .github/labels.yml in AGENTS.md 2026-04-27 03:30:43 +03:00
CODE_OF_CONDUCT.md Update CODE_OF_CONDUCT.md 2025-10-08 09:43:34 +05:00
CONTRIBUTING.md [docs] Proofread the readme and contributing 2025-04-09 11:09:19 +03:00
CONTRIBUTOR_LADDER.md Update CONTRIBUTOR_LADDER.md 2025-10-08 09:28:27 +05:00
go.mod feat(controller): add BucketClaim support to WorkloadMonitorReconciler 2026-04-17 10:58:47 +03:00
go.sum feat(controller): add BucketClaim support to WorkloadMonitorReconciler 2026-04-17 10:58:47 +03:00
GOVERNANCE.md Create GOVERNANCE.md (#733) 2025-04-01 18:48:14 +02:00
LICENSE Preapare release v0.0.1 2024-02-08 12:04:32 +01:00
MAINTAINERS.md Add Mattia Eleuteri (@mattia-eleuteri) as Maintainer 2026-04-15 23:46:43 +05:00
Makefile build(linstor): include linstor-gui in root image build target (#2498) 2026-04-28 13:55:25 +03:00
README.md Update README.md 2026-04-16 14:30:51 +05:00
SECURITY.md docs: add SECURITY.md 2026-03-17 02:57:52 +05:00

Cozystack Cozystack

Open Source Apache-2.0 License Support Active GitHub Release GitHub Commit OpenSSF Best Practices

Cozystack

Cozystack is a free platform and framework for building clouds.

Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.

With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.

Use Cozystack to build your own cloud or provide a cost-effective development environment.

Cozystack user interface

Use-Cases

Documentation

The documentation is located on the cozystack.io website.

Read the Getting Started section for a quick start.

If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.

Versioning

Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.

Contributions

Contributions are highly appreciated and very welcomed!

In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.

You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.

If you have Feature Requests please use the Discussion's Feature Request section.

Community

You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.

License

Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.

Commercial Support

A list of companies providing commercial support for this project can be found on official site.