fix(kubernetes): gate CP-side Deployments on tenant etcd DataStore

The soft-skip wrap in cluster.yaml only silenced Cluster and
KamajiControlPlane rendering when _namespace.etcd is empty. The three
CP-side Deployments (cluster-autoscaler, kccm, kcsi-controller) still
rendered, their wait-for-kubeconfig init containers CrashLoopBackOff'd
forever (no KamajiControlPlane = no admin-kubeconfig Secret),
HelmRelease hit its 15m wait timeout and triggered the very install
remediation cycle the rest of this PR prevents. Self-contradiction.

Wrap each of the three Deployment templates in
{{- if .Values._namespace.etcd }}...{{- end }} so they render only
when there is a DataStore to back them. Add an invariant bats test
that renders the whole chart with etcd empty and asserts zero
Deployments reference *-admin-kubeconfig.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
This commit is contained in:
Aleksei Sviridkin 2026-04-16 21:25:50 +03:00
parent 03426fbd71
commit 12632c60c7
No known key found for this signature in database
GPG key ID: 7988329FDF395282
4 changed files with 61 additions and 0 deletions

View file

@ -70,3 +70,48 @@
echo "Invariant holds for $matched Deployment(s)"
}
@test "chart emits zero admin-kubeconfig Deployments when tenant has no etcd DataStore" {
# Without a DataStore (parent Tenant has not populated _namespace.etcd yet)
# the control-plane-side Deployments must NOT render at all. If they did,
# the wait-for-kubeconfig init would CrashLoopBackOff indefinitely - there
# would be no KamajiControlPlane to provision the Secret - consuming the
# HelmRelease wait budget and triggering exactly the remediation cycle the
# rest of this chart tries to avoid. This test renders the whole chart
# with etcd empty and asserts no Deployment references the admin-kubeconfig
# Secret.
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
helm template invariant packages/apps/kubernetes \
--namespace tenant-root \
--set _namespace.etcd="" \
--set _namespace.monitoring="" \
--set _namespace.ingress="" \
--set _namespace.seaweedfs="" \
--set _namespace.host="" \
--set _cluster.cluster-domain=cozy.local \
--set 'nodeGroups=null' \
2>/dev/null > "$tmp/rendered.yaml"
matched=$(
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
| jq -s '
map(select(.kind == "Deployment")) |
map(select(
(.spec.template.spec.volumes // [])
| any(.secret.secretName | test("-admin-kubeconfig$")?)
)) |
length
'
)
if [ "$matched" -ne 0 ]; then
echo "Expected zero Deployments mounting *-admin-kubeconfig when etcd is empty, got $matched:" >&2
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
| jq -s 'map(select(.kind == "Deployment") | .metadata.name)' >&2
exit 1
fi
echo "No admin-kubeconfig Deployments rendered for empty etcd (as expected)"
}

View file

@ -1,3 +1,14 @@
{{- /*
Gate the control-plane-side workloads on the parent tenant having an etcd
DataStore. Without it no KamajiControlPlane is ever created, Kamaji never
provisions -admin-kubeconfig, and rendering these Deployments would cause
the wait-for-kubeconfig init to CrashLoopBackOff indefinitely, consuming
the parent HelmRelease install timeout and triggering the very uninstall
remediation cycle this chart is supposed to avoid. Rendering them only
when $etcd is set keeps the HelmRelease Ready while flux retries on its
interval and picks up the DataStore as soon as the Tenant chart finishes.
*/}}
{{- if .Values._namespace.etcd }}
---
apiVersion: apps/v1
kind: Deployment
@ -108,3 +119,4 @@ rules:
- list
- update
- watch
{{- end }}

View file

@ -1,3 +1,4 @@
{{- if .Values._namespace.etcd }}
kind: Deployment
apiVersion: apps/v1
metadata:
@ -238,3 +239,4 @@ spec:
secretName: {{ .Release.Name }}-admin-kubeconfig
optional: true
name: kubeconfig
{{- end }}

View file

@ -1,3 +1,4 @@
{{- if .Values._namespace.etcd }}
apiVersion: apps/v1
kind: Deployment
metadata:
@ -60,3 +61,4 @@ spec:
optional: true
name: kubeconfig
serviceAccountName: {{ .Release.Name }}-kccm
{{- end }}