Compare commits

..

54 commits

Author SHA1 Message Date
Aleksei Sviridkin
1f771fec88
ci: retry CI
Empty commit to retrigger E2E that hit two known flakes:
- kubernetes-previous (CSINode registration race per
  flake-kubernetes-previous-csi memory): nfs-test-pvc timed out waiting
  for condition.
- vminstance: VMI did not get IP in 20s timeout (KubeVirt flake).

Neither is related to the admission-chain or VAP changes in this PR.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-26 07:38:38 +03:00
Aleksei Sviridkin
8fb47c72fa
ci: retry CI
Empty commit to retrigger E2E that flaked on KubeVirt vminstance test
(Create a VM Instance — VMI did not get IP in 20s timeout). Same flake
that hit #2489 earlier in this session; not related to the
admission-chain or VAP changes in this PR.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-26 05:58:55 +03:00
Aleksei Sviridkin
1146abce94
fix(cozystack-api): invoke admission chain on Application Create and Delete
The custom REST handler at pkg/registry/apps/application/rest.go is the
storage backend for every kind in apps.cozystack.io/v1alpha1 (Tenant,
MariaDB, Postgres, Kubernetes, ...). genericapiserver passes the
admission chain — mutating webhooks, validating webhooks, and
ValidatingAdmissionPolicies — to each storage method as a callback:
createValidation for Create, updateValidation for Update,
deleteValidation for Delete. Update was correctly invoking its callback
since 23e399bd, but Create and Delete accepted the parameter and never
called it. Admission was silently bypassed for both verbs on every
apps.cozystack.io resource.

The ValidatingAdmissionPolicy cozystack-tenant-host-policy added in
this PR is the first in-tree policy that depends on Create-side
enforcement. The accompanying e2e probe
'cozystack-tenant-host-policy blocks non-trusted callers from setting
tenant.spec.host' caught the gap.

Mirrors the existing updateValidation pattern: invoke createValidation
in Create after the cheap format/length checks but before any state
change, and invoke deleteValidation in Delete after the HelmRelease has
been resolved (so admission sees the same shape it would on UPDATE) but
before the actual r.c.Delete call.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-26 04:35:19 +03:00
Aleksei Sviridkin
75c06ce6e8
fix(tests): use dashless tenant name in tenant-host-policy probe
The gateway.bats e2e test creates a Tenant named 'vap-host-probe' to
verify that a non-trusted ServiceAccount cannot set tenant.spec.host.
Tenant name validation rejects dashes (lowercase letters and digits
only), so the apply was rejected by the name validator before reaching
the cozystack-tenant-host-policy ValidatingAdmissionPolicy. Under the
test's set -e, the subsequent grep for 'ValidatingAdmissionPolicy' in
the error output then exits non-zero, failing the test.

Renaming the probe to 'vaphostprobe' lets the request pass the name
validator and reach the VAP, which is what the test is meant to
exercise.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-26 02:51:35 +03:00
Aleksei Sviridkin
0457e98ee7
ci: retry CI
Empty commit to retrigger Build job that failed on Docker Hub anonymous
pull rate limit (429 Too Many Requests) for alpine:latest.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-26 01:24:14 +03:00
Aleksei Sviridkin
206a31f89c
refactor(gateway): delegate CiliumLoadBalancerIPPool to tenant chart
#2468 introduced packages/apps/tenant/templates/cilium-lb-pool.yaml as the per-tenant owner of the LoadBalancer IP pool. Extend that template with the .Values.gateway branch: the pool now renders when either _cluster.expose-mode=loadBalancer (with .Values.ingress=true on the publishing tenant) OR .Values.gateway=true. Cilium LB IPAM forbids overlapping CIDRs across pools regardless of serviceSelector, so a single pool per tenant is required — the same CIDRs cannot back both ingress and gateway from two separate pools without one of them being marked Conflicting.

Delete packages/extra/gateway/templates/cilium-lb-pool.yaml — the gateway chart no longer renders its own pool. The empty-externalIPs render-time fail moves to the tenant template, guarded on the gateway branch so the legacy ingress path remains unchanged.

Gateway tests lose the pool-rendering asserts that previously lived in packages/extra/gateway/tests/gateway_test.yaml. The tenant chart tests in packages/apps/tenant/tests/exposure_test.yaml gain five gateway-path cases covering pool rendering, non-publishing-tenant rendering, the simultaneous ingress+gateway single-pool case, gateway-only with ingress=false, and the empty-externalIPs fail.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:07:08 +03:00
Aleksei Sviridkin
604b84f78f
test(e2e): invert VAP rejection tests to use if ! so set -e is respected
Previous pattern relied on a "__SUCCEEDED__" marker captured via
`output=$(kubectl apply … && echo "__SUCCEEDED__")`. On dash
(hack/cozytest.sh is #!/bin/sh with set -e) a failing command
substitution propagates its non-zero exit through the assignment and
kills the test before the follow-up grep can run — so admission
correctly rejecting the probe read as a test failure. The marker
approach also hides what actually happened behind a string comparison.

Rewrite each VAP negative-path test as `if ! output=$(…); then …
else …; fi`. The if-condition disables set -e per POSIX, the exit
code is consumed directly (admission reject = happy path), and on the
unexpected-success branch we print the captured admission response
and clean up the resource that slipped through. Happy-path cleanup
only removes the scaffolding RBAC because the probe resource was
never created.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:04 +03:00
Aleksei Sviridkin
ac70320544
fix(cozystack-basics): Flux lives in cozy-fluxcd, not flux-system
Root cause of the install regression on run 24885268333: the Flux
ServiceAccount in cozystack is `system:serviceaccount:cozy-fluxcd:flux`,
not `flux-system:flux`. My trustedCaller list had
`system:serviceaccounts:flux-system`, so once the namespace-host-label
VAP was broadened to CREATE, Flux's first-time write of
`namespace.cozystack.io/host` on tenant-test got rejected:

  namespaces "tenant-test" is forbidden: ValidatingAdmissionPolicy
  'cozystack-namespace-host-label-policy' denied request: namespace
  label namespace.cozystack.io/host is immutable once set (was ,
  requested test.example.org); only cluster-admins and cozystack/Flux
  service accounts may change it. Request user:
  system:serviceaccount:cozy-fluxcd:flux

Replace `flux-system` with `cozy-fluxcd` in the trustedCaller group
list of both cozystack-tenant-host-policy and
cozystack-namespace-host-label-policy, plus the README.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:04 +03:00
Aleksei Sviridkin
3a698e76d3
fix(cozystack-basics): close namespace-host-label-policy CREATE gap
The policy previously only fired on UPDATE and only when oldObject already
carried namespace.cozystack.io/host, which left two paths unguarded:

  - CREATE of a namespace with the label pre-set.
  - UPDATE that adds the label for the first time (oldObject has no label,
    so the matchCondition was false and the VAP was skipped).

Both paths required namespace create/update on labels, which is normally
cluster-admin-only, but the VAP is meant to be the source of truth for
this label's integrity. Close the gap:

  - operations: [CREATE, UPDATE]
  - matchCondition now fires when either object or oldObject carries the
    label (renamed had-host-label -> touches-host-label).
  - oldHost is computed with a null-safe ternary so CREATE (where
    oldObject is null) evaluates cleanly.

The existing validation (newHost == oldHost || trustedCaller) then
naturally denies first-time label writes from non-trusted callers while
still allowing cozy-system / cozy-cert-manager / flux-system / kube-system
SAs to stamp the label during the tenant chart apply.

Update packages/extra/gateway/README.md layer 5 description to match and
add an e2e test that asserts a namespace CREATE with the label from an
untrusted SA is rejected.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:04 +03:00
Aleksei Sviridkin
ff52afc02a
test(e2e): fix shell logic — set -e killed VAP tests on happy path
Observed on run 24880685728: the hostname-policy VAP correctly rejected
the foreign-hostname Gateway (apiserver returned the exact "Gateway
listener hostname must equal test.example.org" message), but the bats
test still failed. Root cause: the guard

  echo "$output" | grep -q "__SUCCEEDED__" && { return 1; }

is an AND list at statement level. When the marker is ABSENT (the
intended happy path — admission rejected, no marker captured), grep
exits 1, && short-circuits, the overall compound command returns 1,
and cozytest.sh's set -e kills the test.

Rewrite each of the four VAP negative-path guards as an if-statement:

  if echo "$output" | grep -q "__SUCCEEDED__"; then
    echo "BUG: …" >&2
    return 1
  fi

Bash's set -e explicitly ignores non-zero exits from commands inside
if-conditions, so the happy path falls through to the follow-up greps
that verify the expected rejection message.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:04 +03:00
Aleksei Sviridkin
e036a644e7
test(e2e): grant RBAC to impersonated SA in gateway VAP negative tests
Address review feedback from coderabbitai on hack/e2e-apps/gateway.bats:170 —
the VAP negative-path tests impersonate
system:serviceaccount:tenant-test:default, but that SA has no RBAC to
create apps.cozystack.io/tenants or to update/patch namespaces.
Authorization runs BEFORE admission, so without those grants the
apiserver returns a plain RBAC Forbidden, the test's grep for
"ValidatingAdmissionPolicy" fails, and the test appears to be a VAP
regression when the VAP is actually fine.

Create an ad-hoc Role + RoleBinding in tenant-test granting create/update
on apps.cozystack.io/tenants to default SA (scoped to tenant-test) for
the tenant-host-policy test, and a ClusterRole + ClusterRoleBinding
granting update/patch on core namespaces for the namespace-host-label
policy test. Both are deleted after the test runs so the sandbox stays
clean for subsequent tests.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:03 +03:00
Aleksei Sviridkin
2925795a5f
fix(cilium): set defensive envoy DaemonSet resource baseline
Address observation from kitsunoff on packages/system/cilium/values.yaml:21 —
envoy.enabled: true turns on a per-node envoy DaemonSet but the upstream
default leaves envoy.resources at {} (no requests, no limits). PR
description cites ~100 MB RAM/node at idle; under L7 traffic that grows
without a cap, and the lack of requests also lets the DaemonSet schedule
onto any node with zero reserved headroom.

Pin the baseline matching the commented-out example in the upstream
Helm values: 100m CPU + 512Mi memory requests, 1Gi memory limit. That
gives the scheduler something to plan around and prevents a runaway
listener from OOM-ing the node without drawing attention.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:03 +03:00
Aleksei Sviridkin
a622d346c6
fix(cozystack-basics): close tenant-host-policy bypass via clearing spec.host
Address review feedback from coderabbitai on
packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:133 —
the matchCondition required object.spec.host != "", so an UPDATE that
unset or emptied the host field skipped the VAP entirely. A tenant with
empty spec.host inherits the apex from its parent, so clearing a
previously-set host is itself an effective apex change that the policy's
intent already covers. A non-trusted caller with UPDATE permission on
the Tenant could flip spec.host: foo.example.org → "" without ever
hitting the trustedCaller gate.

Rewrite the gate so the VAP fires whenever EITHER side of the update
has a non-empty host, and compute hostChanged over both oldHost and
newHost (each with its own CEL variable and ternary fallback to ""
when the field is absent). Leaves the validation unchanged: the change
is allowed only when hostChanged=false or variables.trustedCaller=true.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:03 +03:00
Aleksei Sviridkin
9c96dc8e78
fix(gateway): require dns01 solver — Let's Encrypt refuses HTTP-01 for wildcard certs
Address review feedback from kitsunoff on
packages/extra/gateway/templates/issuer.yaml:21 — the per-tenant Issuer was
emitting an http01.gatewayHTTPRoute solver while the accompanying
Certificate in gateway.yaml requests both the apex and the *.<apex>
wildcard. Let's Encrypt refuses to issue wildcard certificates through
HTTP-01 (see LE challenge-types docs and the cert-manager CRD schema
note at crd-cert-manager.io_clusterissuers.yaml:728-729). Under the
default publishing.certificates.solver=http01, the Order would loop
forever, the Secret would never be created, and the Gateway HTTPS
listeners would sit without a cert. cert-manager-issuers ClusterIssuers
already branch on _cluster.solver (http01 vs dns01 Cloudflare); the
per-tenant Issuer was ignoring that signal entirely and the E2E flag
that would have exercised the full issuance flow was reverted in
a4bef14, so CI missed the regression.

Fail the render when publishing.certificates.solver=http01 with an
explicit message pointing at the root cause and the dns01/selfsigned
alternatives, and switch the emitted solver to dns01.cloudflare (same
shape cluster-issuers.yaml uses). Add a unit test for the http01-fail
path and update the existing Issuer test asserts to match the dns01
solver shape.

Follow-up: add an E2E assertion that actually issues a Certificate
through the per-tenant Issuer and waits for .status.conditions[Ready]=True
once the gateway.enabled=true install path is re-enabled.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:03 +03:00
Aleksei Sviridkin
3ff3d68132
fix(cozystack-basics): matchCondition by namespace prefix, move label lookup to variables with ternary fallback
Previous matchConditions on namespaceObject.metadata.labels were evaluating
to false or skipping silently despite the label being present
(verified via diagnostic test on run 24875535058: VAP exists, binding
has validationActions: [Deny], tenant-test namespace has
namespace.cozystack.io/host=test.example.org, yet admission still
accepted a Gateway with listener hostname dashboard.example.org).

Restructure the VAP so matchCondition does NOT touch namespaceObject —
it just checks the Gateway's declared namespace starts with tenant-
via object.metadata.namespace. The namespaceObject-based label lookup
moves into a CEL variable with a ternary that returns "" when the label
is missing, and the validation accepts if tenantHost is empty (skip) or
if every listener hostname matches. This way the gate fires predictably
for every tenant-* Gateway and the namespaceObject access happens in a
context where its shape is easier to reason about.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:03 +03:00
Aleksei Sviridkin
4ef14c4147
fix(cozystack-basics,e2e): diagnose and simplify hostname-policy matchCondition
Run 24870368740 showed the VAP still not rejecting cross-tenant hostname
after the CEL chain fix. Diagnostic test on run 24873181209 confirmed:

1. tenant-test namespace DOES have namespace.cozystack.io/host=test.example.org.
2. admission still CREATES a Gateway with listener hostname dashboard.example.org
   (__SUCCEEDED__ marker appeared — kubectl apply returned 0).

So the label is present but the VAP's matchCondition evaluates to false
for some reason, skipping the policy entirely. The prime suspect is
has(namespaceObject.metadata) on a Kubernetes unstructured representation
where the has() macro semantics don't match the typed Namespace resource
my chain fix assumed.

Simplify the matchCondition to use only the in-operator for map-key
presence ("k" in map) — this is the canonical CEL pattern for
K8s-style unstructured labels and does not rely on has() working
consistently across nested selection:

  namespaceObject != null &&
  "namespace.cozystack.io/host" in namespaceObject.metadata.labels &&
  namespaceObject.metadata.labels["..."] != ""

Also add a diagnostic test to hack/e2e-apps/gateway.bats that dumps the
VAP + binding yaml and asserts validationActions includes Deny — so if
anything structural breaks with the VAP installation, the failure is
visible in the first few lines of output.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:03 +03:00
Aleksei Sviridkin
992c67a433
test(e2e): diagnostic test — verify tenant-test has host label before VAP tests
The VAP 'ValidatingAdmissionPolicy rejects Gateway with foreign hostname'
silently fails to reject when tenant-test namespace lacks the
namespace.cozystack.io/host label — matchCondition returns false, VAP
skips, Gateway is admitted. That's what happened on run 24870368740.

Add a dedicated test BEFORE the VAP tests that asserts tenant-test
actually has the expected host label. If it doesn't, the test fails with
a clear SETUP FAILURE message and dumps the full namespace yaml to CI
logs, so the real root-cause (label missing — tenant chart bug, helm
timing, or namespaceObject representation issue) is visible instead of
being obscured by the downstream VAP test failure.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:02 +03:00
Aleksei Sviridkin
74c406e331
test(e2e): rewrite VAP tests without bats run helper (cozytest.sh is pure shell)
hack/cozytest.sh is a pure-shell bats-compat runner, not real bats. The
'run' builtin that bats ships (captures cmd output/status into $output
and $status) is unavailable, so every test using 'run kubectl …' fails
with 'run: not found' as soon as it's exercised. Previous runs didn't
surface this because install-cozystack was failing before the gateway
bats file could execute — now that the install path is green, the VAP
tests actually run and hit this trap.

Rewrite the four VAP negative-path tests to capture kubectl output and
exit status manually. Uses '&& echo __SUCCEEDED__' inside the command
substitution so the happy-path (admission accepted something it
shouldn't) is detectable, and captures stderr via 2>&1 so admission
rejection messages land in $output for grep assertions.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:02 +03:00
Aleksei Sviridkin
cec5f8d1d2
fix(cozystack-basics): chain has() through each level of namespaceObject
Second CEL bug in the same VAP surfaced in E2E run 24862782568 (gateway
bats test on tenant-test namespace): matchCondition
has(namespaceObject.metadata.labels) errored at admission time with
"no such key: metadata" — apiserver rejected the Gateway with
"ValidatingAdmissionPolicy denied request: expression '...' resulted in
error: no such key: metadata".

In Kubernetes CEL the has() macro checks field presence at the last
selection only — has(a.b.c) requires a.b to already be resolvable.
When the intermediate step is missing or the object is represented as
an unstructured map with a missing key, CEL raises 'no such key' at
evaluation rather than letting has() short-circuit to false.

Chain the checks per Kubernetes CEL docs recommendation (Check if a
field is set → has(a.b) && has(a.b.c) && ...):

  namespaceObject != null &&
  has(namespaceObject.metadata) &&
  has(namespaceObject.metadata.labels) &&
  "namespace.cozystack.io/host" in namespaceObject.metadata.labels &&
  namespaceObject.metadata.labels["namespace.cozystack.io/host"] != ""

The null-check handles the cluster-scoped case that K8s documents as
'namespaceObject is null when the incoming object is cluster-scoped',
the has() chain handles the unstructured-map case that surfaced in the
gateway-e2e-probe on tenant-test.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:02 +03:00
Aleksei Sviridkin
e4963d47f4
fix(cozystack-basics): has(oldObject) is not valid CEL — use oldObject != null
The cozystack-tenant-host-policy VAP failed to install because its CEL
variable used has(oldObject), which is not a valid CEL macro: has()
requires a field selection expression like has(x.y), not a bare
identifier check. The apiserver rejected the policy with
"invalid argument to has() macro", Helm install of cozystack-basics
failed, Flux triggered uninstall remediation which deleted the namespaces
created by the chart (cozy-public, tenant-root), and every downstream
HelmRelease that had to create resources in cozy-public or tenant-root
(kubevirt-cdi, the tenant chart, etc.) then failed with
"namespace ... because it is being terminated". Root cause for the
whole install-cozystack E2E regression on this branch.

Replace has(oldObject) with oldObject != null. On CREATE oldObject is
null and the ternary returns true (hostChanged); on UPDATE oldObject is
set and the rest of the condition runs as before.

Found via the cozyreport artifact from run 24858285228 rerun — the
kubectl describe hr cozystack-basics output exposed the VAP compilation
error that the top-level HelmRelease status message truncated with
"1 error occurred: ...".

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:02 +03:00
Aleksei Sviridkin
f5066e977a
fix(gateway): fail render when publishing.externalIPs is empty and guard pre-CIDR input
Address review feedback from kitsunoff on
packages/extra/gateway/templates/cilium-lb-pool.yaml:9: the previous
behaviour rendered the tenant Gateway without a matching
CiliumLoadBalancerIPPool when publishing.externalIPs was empty. Cilium LB
IPAM then had no pool to serve the tenant's cilium-gateway-cozystack
Service, so the Service sat in <pending> forever with no address and the
operator got a silently non-working Gateway.

Replace the conditional skip with an explicit helm fail that names the
root cause. Also guard the CIDR suffix append on the absence of "/" to
accept pre-CIDR input (consistent with the same guard already shipped on
packages/extra/ingress/templates/cilium-lb-pool.yaml).

Update the unit test that asserted hasDocuments: count: 0 to assert the
new failed-template error message, and add expose-external-ips to every
other test so they don't accidentally cross the empty-IPs guard.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:02 +03:00
Aleksei Sviridkin
824be9354c
test(e2e): cover the three remaining VAPs in gateway.bats
Address review feedback from kitsunoff on
hack/e2e-apps/gateway.bats:79: only cozystack-gateway-hostname-policy was
exercised end-to-end. Commit 8770b87 specifically retrofitted
matchConditions onto the other three VAPs to stop failurePolicy: Fail from
turning missing fields into denials, which is a high-regression-risk area,
so each VAP needs its own test.

Add:
- cozystack-gateway-attached-namespaces-policy rejects a Package with a
  tenant-* entry in spec.components.platform.values.gateway.attachedNamespaces.
- cozystack-tenant-host-policy rejects a Tenant.spec.host set by an
  impersonated tenant-namespace ServiceAccount (non-trusted group list).
- cozystack-namespace-host-label-policy rejects a kubectl label overwrite
  on namespace.cozystack.io/host by the same non-trusted SA.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:01 +03:00
Aleksei Sviridkin
afa69148cb
docs(gateway): rewrite security model to cover all four VAPs plus the render-time check
Address review feedback from kitsunoff on
packages/extra/gateway/README.md:17: the security model section described
only two layers, but packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml
ships four ValidatingAdmissionPolicies plus a render-time fail in
cozystack-basics. Rewrite the section to cover each VAP's scope and what
it guards against, plus the listener allowedRoutes whitelist and the
render-time fail — six layers, each one explained.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:01 +03:00
Aleksei Sviridkin
ccde31cc8b
fix(platform): add default namespace to gateway.attachedNamespaces
Address review feedback from kitsunoff on
packages/system/cozystack-api/templates/api-tlsroute.yaml:10: the
kubernetes-api TLSRoute lives in the default namespace (it has to sit
next to the kubernetes Service it points at), but default was missing
from the platform's gateway.attachedNamespaces default list. The tenant
Gateway's listener allowedRoutes selector then rejected the TLSRoute,
so Kubernetes API through Gateway API silently failed to work.

Add default to the default whitelist, plus a comment explaining why it
has to be there.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:01 +03:00
Aleksei Sviridkin
e60549e7da
fix(tenant): stop inheriting parent gateway reference
Address review feedback from kitsunoff on
packages/apps/tenant/templates/namespace.yaml:32: the previous
$gateway := $parentNamespace.gateway | default "" pattern made a child
tenant without gateway: true inherit the parent's gateway namespace into
_namespace.gateway. Harbor and Bucket in that child would then render
HTTPRoutes with parentRefs.namespace pointing at the parent, which fails
in two ways:

1. The parent Gateway's listener allowedRoutes selector whitelists only
   the parent namespace plus publishing.gateway.attachedNamespaces (a static
   cozy-* list). The child tenant namespace is not there, so the HTTPRoute
   ends up NotAllowedByListeners.
2. The parent wildcard Certificate covers <apex> and *.<apex> only. A
   child tenant apex is <name>.<parent-apex>, so a harbor hostname would
   be harbor.<name>.<parent-apex> — two subdomain levels deep — and no
   valid cert would be presented.

Drop the inheritance and require explicit tenant.spec.gateway=true on
each tenant that should host Gateway-attached apps. Tenants without
their own gateway fall back to the inherited ingress reference, which is
the same behaviour the ingress path already has.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:01 +03:00
Aleksei Sviridkin
8aeeab0ae2
fix(bucket): bind HTTPRoute to the https listener only
Address review feedback from kitsunoff on
packages/system/bucket/templates/httproute.yaml:9: same HTTP->HTTPS
redirect regression as dashboard's HTTPRoute. Pin sectionName: https so
the bucket UI HTTPRoute stays on the TLS listener and the generic
http-to-https-redirect on port 80 actually fires for cleartext requests.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:01 +03:00
Aleksei Sviridkin
966659903f
fix(harbor): bind HTTPRoute to the https listener only
Address review feedback from kitsunoff on
packages/apps/harbor/templates/httproute.yaml:10: same HTTP->HTTPS redirect
regression as dashboard's HTTPRoute. Pin sectionName: https so port 80
stays owned by the generic redirect HTTPRoute on the tenant Gateway.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:01 +03:00
Aleksei Sviridkin
4a5e856f7b
fix(keycloak): bind HTTPRoute to the https listener only
Address review feedback from kitsunoff on
packages/system/keycloak/templates/httproute.yaml:11: same HTTP->HTTPS
redirect regression as dashboard's HTTPRoute — without sectionName, the
route attached to both listeners and a specific-hostname route on port 80
won the precedence contest against the generic redirect HTTPRoute. Pin
sectionName: https so the keycloak route stays on the TLS-terminating
listener exclusively.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:00 +03:00
Aleksei Sviridkin
49bc8c487a
fix(dashboard): bind HTTPRoute to the https listener only
Address review feedback from kitsunoff on
packages/system/dashboard/templates/httproute.yaml:11: without sectionName,
the HTTPRoute attached to both http:80 and https:443. Under Gateway API
hostname-precedence rules, a route with a specific hostname wins over the
generic http-to-https-redirect HTTPRoute on the http listener, so HTTP
traffic to http://dashboard.example.org/ was proxied directly to the
backend with no TLS — a regression against ingress-nginx behaviour.

Pin sectionName: https so port 80 stays fully owned by the redirect route.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:00 +03:00
Aleksei Sviridkin
57f1c82aea
fix(cozystack-basics): narrow trustedCaller to explicit cozystack group list
Address review feedback from kitsunoff on
packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:120:
startsWith("system:serviceaccount:cozy-") without a trailing colon trusted
any ServiceAccount in any namespace whose name begins with cozy- — including
a hypothetical attacker-controlled cozy-evil namespace. Creating such a
namespace requires cluster-admin today, so this is not an active escalation
path, but it is an unnecessarily wide trust boundary for a defense-in-depth
check.

Switch both tenant-host-policy and namespace-host-label-policy to a
group-based check that matches the exact system:serviceaccounts:<ns> group
for cozy-system, cozy-cert-manager, flux-system and kube-system. Every SA
in those namespaces is a member of that group automatically, so there is no
behaviour regression for cozystack's own controllers.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:00 +03:00
Aleksei Sviridkin
21a3f21777
fix(cozystack-basics): gate gateway-hostname-policy with matchConditions
Address review feedback from kitsunoff on
packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:22:
the first VAP had no matchConditions, so with failurePolicy: Fail and a
validation expression of variables.tenantHost != "", any Gateway in a
namespace without the namespace.cozystack.io/host label (kube-system,
default, any non-cozystack namespace) was rejected cluster-wide, including
clusters running with gateway.enabled: false.

Hoist the namespace-label precondition into matchConditions so the VAP only
fires for cozystack-managed tenant namespaces, then simplify the validation
body — the explicit tenantHost-nonempty check becomes redundant.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:00 +03:00
Aleksei Sviridkin
9ea31dbcfb
fix(platform): move gateway-api-crds include out of the variant prologue to avoid YAML comment+separator collision
The bundle file begins with a variant-switching '# Networking'
comment followed by '{{- if eq .Values.bundles.system.variant ...}}'.
The left-strip on that if eats the newline after '# Networking' and
the next emitted '---' from the first package helper ends up
appended to the comment as '# Networking---', which YAML treats as
pure comment (no document separator). That hides the separator
between whatever was emitted before the variant block and the first
Package inside it, so the two documents merge and duplicate-key
errors fire at Flux post-render time.

Pre-existing behaviour was safe only because the variant block
itself produced the first cozystack.io Package, so no previous
Package existed to duplicate keys with. The previous commit put a
brand-new 'cozystack.gateway-api-crds' Package include right before
the '# Networking' comment — that Package's apiVersion then
collided with the networking Package's apiVersion because the
separator between them was consumed by the comment.

Move the gateway-api-crds include down to the 'Common Packages'
section, which has no strip-dash interaction with surrounding
comments. Package ordering in the bundle does not determine install
ordering — Flux honours the dependsOn chain on cozystack.networking
and cozystack.cert-manager, both of which declare a dependsOn on
cozystack.gateway-api-crds.

Verified locally that the rendered platform chart now emits every
Package preceded by '# Source:' comment on its own line, with the
helper-emitted '---' on a dedicated line. 'apiVersion' keys no
longer duplicate within any single YAML document.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:00 +03:00
Aleksei Sviridkin
74046751cb
fix(cozystack-basics): gate VAP validations through matchConditions for safe CEL evaluation
Three of the four ValidatingAdmissionPolicies walked deeply into
object fields inside validation CEL expressions. With failurePolicy:
Fail and non-structural CRD schemas (Package, Tenant) the evaluator
could reject the request instead of skipping — which would cascade
into HelmReleases never being produced and the e2e install hanging
on 'waiting for more than 10 HelmReleases'.

Move path existence checks into matchConditions (Kubernetes 1.30+).
matchConditions are evaluated before validations; a false matchCondition
cleanly skips the validation rather than forcing failurePolicy. Switch
to the 'key in map' operator where possible — it is the supported safe
form for Kubernetes JSON without structural schema guarantees.

Policies fixed:

- cozystack-gateway-attached-namespaces-policy: matchCondition
  'has-gateway-attached-namespaces' checks every level of the path
  spec.components.platform.values.gateway.attachedNamespaces using
  'key in map'. Validation runs only on Packages that actually carry
  the field.
- cozystack-tenant-host-policy: matchCondition 'has-host-field' skips
  Tenants without a non-empty spec.host. The hostChanged variable no
  longer needs defensive has() checks.
- cozystack-namespace-host-label-policy: matchCondition 'had-host-label'
  skips namespaces that never had the namespace.cozystack.io/host label.
  The immutability rule now only fires on true UPDATE-of-set-label.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:04:00 +03:00
Aleksei Sviridkin
f9041fa49a
feat(cozystack-basics): two more runtime VAPs close the tenant.spec.host and namespace label hijack vectors
Security review left two residual trust-assumption gaps:

1. tenant.spec.host is user-controlled. A tenant admin with RBAC
   permission to create Tenant CRs could claim an arbitrary host
   (dashboard.example.org), have the tenant chart write that host
   into their namespace label, and thereby satisfy the per-namespace
   host VAP with a hijacked value.

2. namespace.cozystack.io/host is written by cozystack charts but
   there is no runtime check that prevents a later update to the
   label from an untrusted principal. A tenant with Update on their
   own namespace could rewrite the label to whatever they want and
   bypass the per-namespace host VAP.

Both close with parallel VAPs. 'Make it impossible — forbid it twice.'

cozystack-tenant-host-policy

  matchConstraints: apps.cozystack.io/v1alpha1 tenants, CREATE/UPDATE.
  Rejects setting or changing spec.host unless the caller is
  cluster-admin (system:masters group) or a cozystack/Flux/kube-system
  service account. Tenants are still free to leave spec.host empty
  (the normal default) — the tenant chart will compute the
  inherited host from the parent namespace. Admins who need to
  override do so explicitly via kubectl / cozystack-api.

cozystack-namespace-host-label-policy

  matchConstraints: core/v1 namespaces, UPDATE only.
  Rejects any change to the namespace.cozystack.io/host label once
  it has been set, unless the caller is in the same trusted caller
  whitelist. CREATE is unrestricted (initial label write happens
  there, by the chart). Result: the label is effectively
  write-once-by-cozystack — no tenant can rewrite their own host
  label after it is provisioned.

Both policies use failurePolicy: Fail and validationActions: [Deny],
so a missing paramRef or CEL error fails closed.

Combined with the two earlier VAPs (Gateway listener hostname +
Package attachedNamespaces) and the listener allowedRoutes namespace
whitelist, the hijack surface for multi-tenant Gateway API is now
guarded at four independent admission points and one runtime-proxy
allow-list. Bypass requires compromising a trusted service account
or obtaining cluster-admin credentials — at which point Gateway API
isolation is not the weakest link anyway.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:59 +03:00
Aleksei Sviridkin
3869a83983
feat(cozystack-basics): runtime VAP pair — per-namespace host + Package attachedNamespaces
Replaces the earlier ConfigMap-parameterised hostname VAP with two
independent ValidatingAdmissionPolicies that both enforce at runtime
and do not depend on the publisher's assumption that 'tenant host is
always <name>.<platform apex>'.

1. cozystack-gateway-hostname-policy

Now reads the tenant's real apex from the namespace label
namespace.cozystack.io/host, which the tenant chart already writes
onto every child tenant namespace and which this commit adds to
tenant-root itself. No CRD lookup, no paramRef, no assumption that
the tenant host is a subdomain of the cluster-wide publishing.host.

CEL:
  tenantHost := namespaceObject.metadata.labels['namespace.cozystack.io/host']
  every listener.hostname must equal tenantHost or endsWith '.' + tenantHost

Consequence:
- tenant-alice with spec.host = alice.example.org gets label
  alice.example.org → may publish alice.example.org and
  *.alice.example.org, nothing else.
- tenant-corp with spec.host = customer1.io (fully independent
  domain, NOT a subdomain of the platform apex) gets label
  customer1.io → may publish customer1.io and *.customer1.io.
  Previous VAP would have rejected this legitimate case.
- A namespace without the label (no cozystack-managed tenant) falls
  through tenantHost == '' → all listener hostnames rejected.

2. cozystack-gateway-attached-namespaces-policy (new)

Runtime counterpart to the render-time 'fail' guard on
_cluster.gateway-attached-namespaces. Watches Package CRs
(cozystack.io/v1alpha1) and rejects any UPDATE that introduces a
tenant-* entry under spec.components.platform.values.gateway.attachedNamespaces.
'kubectl edit packages.cozystack.io cozystack.cozystack-platform'
bypasses the Helm render path; this policy catches it.

CEL walks the array via .all(ns, !ns.startsWith('tenant-')) with
full path presence guards so it ignores packages that do not touch
the gateway block.

3. tenant-root namespace label

cozystack-basics/templates/tenant-root.yaml now writes
  labels.namespace.cozystack.io/host = {{ _cluster.root-host }}
so root tenant's Gateway resources get validated through the same
label-based VAP as child tenants. No ConfigMap-driven param, no
split path between root and child.

Both policies use failurePolicy: Fail and validationActions: [Deny].
They install unconditionally (no gateway.enabled gate) — Gateway CRDs
are installed on every bundle variant, so the policies must guard
every path.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:59 +03:00
Aleksei Sviridkin
31f630f763
fix(gateway): read per-tenant apex from _namespace.host, not _cluster.host
Two correctness fixes that make extra/gateway actually work in a real
cluster, not just in the tests:

1. _cluster.host does not exist

The Secret projected from platform's apps.yaml only exports
_cluster.root-host — the cluster-wide publishing apex (e.g.
example.org). There is no _cluster.host key. The gateway template was
reading _cluster.host, which always evaluated to empty, which always
tripped the 'publishing.host must be set' fail guard. extra/gateway
therefore never rendered in a real cluster; the helm-unittest cases
passed only because they explicitly put a host field under _cluster,
which never happens in production. Switch to _namespace.host — the
per-tenant apex that cozystack-basics (for tenant-root) and the
tenant chart's namespace.yaml (for child tenants) both populate.

2. Guard against 'non-publishing namespace' blocked child tenants

The previous 'if not (eq $exposeIngress .Release.Namespace)' guard
forced extra/gateway to render only in the publishing tenant
namespace. Child tenants with spec.gateway=true deploy extra/gateway
into their own namespace, so the guard always fired and no child
Gateway was ever materialised — despite earlier commits claiming
child-tenant ACME works. The guard is removed; extra/gateway now
renders in whichever namespace it is released into, which matches
the per-tenant HelmRelease template.

Side-effects:

- CiliumLoadBalancerIPPool no longer carries the '== publishing
  namespace' condition — each tenant-scoped release produces its
  own pool, selecting its own cilium-gateway-cozystack Service by
  io.kubernetes.service.namespace. Different pools do not conflict
  as Kubernetes resources; IP allocation across pools with
  overlapping blocks is an operator concern (document).
- issuer.yaml unchanged — the Issuer's parentRef already had no
  namespace, so the solver attaches to the local Gateway regardless
  of which tenant namespace we render in.
- A new helm-unittest case exercises the child-tenant path
  (tenant-alice with host=alice.example.org): Certificate dnsNames,
  Gateway listener hostnames, allowedRoutes whitelist, and
  CiliumLoadBalancerIPPool selector all resolve to the child's
  apex and namespace.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:59 +03:00
Aleksei Sviridkin
0f44e2af57
docs(gateway): child-tenant ACME is no longer a limitation
Remove the child-tenant ACME HTTP-01 entry from Known limitations in
the gateway README. With the per-tenant Issuer added in the previous
commit, each tenant that opts into gateway: true gets its own ACME
account, its own HTTPRoute solver targeting its own Gateway, and no
dependency on the publishing tenant's Gateway.

In its place, document the actual remaining constraint: the
per-tenant Issuer template currently supports two issuer names
(letsencrypt-prod, letsencrypt-stage) — any other value fails the
render with an actionable error. Supporting more ACME providers is
a one-case extension of templates/issuer.yaml, so operators know
exactly where to hook in.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:59 +03:00
Aleksei Sviridkin
49d896caca
feat(gateway): per-tenant Issuer enables child-tenant ACME, VAP unconditional
Two correctness fixes for multi-tenant Gateway ACME, plus a small
default-tightening.

1. Per-tenant Issuer (child-tenant ACME closes)

packages/extra/gateway/templates/issuer.yaml (new): render a
cert-manager.io/v1 Issuer named 'gateway' in the tenant's own
namespace. It carries its own ACME account (privateKeySecretRef
'gateway-acme-account') and an http01.gatewayHTTPRoute solver whose
parentRef points to the local Gateway 'cozystack' at sectionName http
— no namespace on parentRef, so it attaches in-namespace only.

packages/extra/gateway/templates/gateway.yaml: the wildcard
Certificate now references issuerRef kind: Issuer, name: gateway
(namespace-scoped) instead of the previous cluster-scoped
letsencrypt-prod. The result: every tenant that enables
tenant.spec.gateway=true gets a fully self-contained ACME flow —
its own account, its own HTTPRoute, its own Gateway, no cross-
namespace dependency on the publishing tenant. Child tenants with
gateway: true now work end-to-end for ACME HTTP-01.

The template maps 'publishing.certificates.issuerName' to a
concrete ACME server URL: letsencrypt-prod →
acme-v02.api.letsencrypt.org/directory, letsencrypt-stage →
acme-staging-v02.api.letsencrypt.org/directory. Any other value
fails the render with an explicit error pointing at the file to
extend.

Three new helm-unittest cases cover Certificate→Issuer linkage,
staging-server selection, and the unknown-issuer fail path.

2. Hostname VAP installs unconditionally

packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:
drop the 'eq gateway-enabled true' guard. The ValidatingAdmissionPolicy
now installs whenever publishing.host is set, regardless of whether
gateway.enabled is flipped. Rationale: Gateway CRDs ship
unconditionally (gateway-api-crds is loaded in every bundle), so a
tenant could create a Gateway manually even with gateway.enabled=false.
The VAP must guard that path too.

3. Restore opt-in default in e2e

hack/e2e-install-cozystack.bats: revert the gateway.enabled: true
line so the default e2e platform configuration matches production
defaults. Gateway API remains strictly opt-in. The
'exposed services render HTTPRoute/TLSRoute but not Ingress' test
(which required platform-wide gateway.enabled=true) is removed; the
VAP cross-tenant reject test stays and now works against the
unconditionally-installed policy.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:59 +03:00
Aleksei Sviridkin
8442f4e44e
test(e2e): flip gateway.enabled=true by default and extend gateway.bats
hack/e2e-install-cozystack.bats: add 'gateway: {enabled: true}' to the
values used for the cozystack.cozystack-platform Package. The e2e
pipeline therefore exercises the Gateway API path as its primary
integration scenario — dashboard, keycloak render as HTTPRoutes;
cozystack-api, vm-exportproxy, cdi-uploadproxy render as TLSRoutes;
the legacy Ingresses for those services should not exist.

hack/e2e-apps/gateway.bats: two new test cases.

1. 'exposed services render HTTPRoute/TLSRoute but not Ingress when
   gateway.enabled=true' — waits for the HTTPRoutes for dashboard and
   keycloak to reach Accepted, confirms the three TLSRoutes exist in
   their respective namespaces, and asserts (negative tests) that the
   old Ingress objects are gone. Catches regressions where a developer
   forgets to wrap an Ingress in the gateway-enabled conditional.
2. 'ValidatingAdmissionPolicy rejects Gateway with foreign hostname' —
   tries to apply a Gateway in tenant-test that claims
   dashboard.example.org (a domain outside the tenant's allowed suffix
   test.example.org), asserts kubectl fails with the expected VAP error
   and the expected message.

Together these cover the two most important regressions this PR is
shipping against: Ingress leaking through when it should not, and
the hostname-hijacking defence silently disappearing.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:58 +03:00
Aleksei Sviridkin
72b2c80cf1
docs(gateway): recommend per-tenant certificate count quota, refresh known limitations
README now shows the exact Tenant manifest snippet for capping the
number of Certificate objects a tenant may create
(count/certificates.cert-manager.io: 10), and calls out that the
default is unlimited so operators on shared-apex multi-tenant clusters
must set this before handing gateway=true to untrusted tenants.

Also rewrites the 'Known limitations' section to reflect what this
PR actually ships: TLS passthrough (cozystack-api, vm-exportproxy,
cdi-uploadproxy) and per-tenant apps (harbor, bucket) are no longer
limitations — they all migrate to Gateway API when gateway.enabled=true.
What's left:

- Child-tenant ACME HTTP-01 still relies on the publishing tenant's
  Gateway; a follow-up PR will add namespace-scoped Issuers.
- Upstream application-level gaps (harbor ACL / bucket upstream
  features) — carried as upstream PRs, not a cozystack blocker.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:58 +03:00
Aleksei Sviridkin
1cddf71288
feat(cozystack-basics): reject tenant-* namespaces in gateway.attachedNamespaces
Render-time guard at the top of gateway-hostname-policy.yaml walks
_cluster.gateway-attached-namespaces and fails the chart render with
an explicit message if any entry begins with 'tenant-'.

Adding a tenant namespace to the attached whitelist would defeat the
whole point of the whitelist: that tenant could then attach arbitrary
HTTPRoutes to the publishing tenant's Gateway and pick any hostname
inside the apex domain. Catching this at 'helm template' time beats
catching it after the Gateway listener is already programmed and
traffic can flow.

The message spells out the correct recourse — child tenants that
need their own Gateway must flip tenant.spec.gateway=true, which
materialises their own Gateway in their own namespace rather than
borrowing the publishing tenant's.

Applies whether gateway.enabled is true or false — validation on the
attachedNamespaces list is orthogonal to whether the policy itself
is installed.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:58 +03:00
Aleksei Sviridkin
b719cf67e3
feat(tenant,harbor,bucket): wire per-tenant apps through the tenant Gateway
Tenants can now opt their own apps into Gateway API without touching
the publishing tenant. The mechanism mirrors the existing
_namespace.ingress contract exactly.

packages/apps/tenant/templates/namespace.yaml:

- Compute a new $gateway value: if the tenant sets .Values.gateway
  it becomes the tenant's own namespace, otherwise it inherits from
  the parent namespace (empty means no Gateway anywhere in the chain).
- Ship $gateway out through the cozystack-values Secret under
  _namespace.gateway and through a namespace label
  namespace.cozystack.io/gateway.

packages/apps/harbor and packages/system/bucket:

- Existing Ingress wrapped in '{{ if not $gateway }}' so it only
  renders when no Gateway is attached anywhere up the tenant chain.
- New templates/httproute.yaml render when $gateway is non-empty:
  apiVersion gateway.networking.k8s.io/v1 HTTPRoute, parentRef to
  the 'cozystack' Gateway in whichever namespace $gateway points
  at, hostname <release>.<host> (harbor) or <bucketName>.<host>
  (bucket), backendRef to the existing Service on its current port.

Per-tenant resolution: a tenant with .Values.gateway=true gets its
own Gateway in its own namespace, and harbor/bucket deployed inside
that tenant attach to that namespace's Gateway — no cross-namespace
references, no coupling to the publishing tenant.

Default remains ingress-nginx: tenants that do not opt in keep
rendering Ingress verbatim.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:58 +03:00
Aleksei Sviridkin
2c5bf50f43
feat(gateway): TLSRoute for TLS-passthrough services (cozystack-api, vm-exportproxy, cdi-uploadproxy)
Wire the three HTTPS-passthrough services through Gateway API so
ingress-nginx is no longer required for them when gateway.enabled=true.

extra/gateway chart: new values.tlsPassthroughServices (defaults to
[api, vm-exportproxy, cdi-uploadproxy]). For each service in that list
AND in _cluster.expose-services, the Gateway emits an extra
port-443 listener with protocol: TLS, tls.mode: Passthrough, hostname
<svc>.<host>, and allowedRoutes.kinds restricted to TLSRoute. The
specific hostname wins over the wildcard HTTPS Terminate listener at
SNI-matching time per Gateway API spec.

Per-service packages:

- packages/system/cozystack-api: existing Ingress wrapped in
  '!= gateway-enabled=true'. New api-tlsroute.yaml renders
  apiVersion gateway.networking.k8s.io/v1alpha2 TLSRoute in the
  'default' namespace (where the kubernetes Service lives) targeting
  the tls-api listener, hostname api.<root-host>, backendRef kubernetes:443.
- packages/system/kubevirt: same treatment for
  vm-exportproxy-ingress.yaml → vm-exportproxy-tlsroute.yaml,
  hostname vm-exportproxy.<root-host>, backendRef vm-exportproxy:443
  in cozy-kubevirt.
- packages/system/kubevirt-cdi: same for cdi-uploadproxy —
  hostname cdi-uploadproxy.<root-host>, backendRef cdi-uploadproxy:443
  in cozy-kubevirt-cdi.

TLSRoute apiVersion v1alpha2 is intentional: Cilium 1.19 vendors
Gateway API CRDs v1.4 (TLSRoute is v1alpha2 there) and v1.5.1 ships
both v1 and v1alpha2 in the experimental channel, so v1alpha2 is
compatible with both. A follow-up can bump to v1 once Cilium v1.20
(Gateway API v1.5) lands in the platform.

Backward compatibility: default gateway.enabled=false keeps the
existing Ingress path verbatim for all three services.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:58 +03:00
Aleksei Sviridkin
6914b8dbc0
docs(gateway): document security model, Let's Encrypt rate limits, known gaps
Add operator-facing documentation for the Gateway API feature:

packages/extra/gateway/README.md grows three sections:

- Security model — explains the two layers that protect cross-tenant
  isolation (Gateway listener allowedRoutes namespace whitelist +
  ValidatingAdmissionPolicy for tenant hostname ownership). Makes it
  explicit which namespaces are on the default whitelist so operators
  who add a new system component know where to register it.
- Rate limits — spells out the Let's Encrypt quotas (50 certs /
  registered domain / week, 5 duplicate certs / week, 300 new orders /
  account / 3h), lists mitigations (letsencrypt-stage, resourceQuotas
  with count/certificates.cert-manager.io, self-signed ClusterIssuer,
  internal ACME).
- Known limitations — TLS passthrough services still use ingress-nginx,
  tenant-scoped apps (harbor, bucket) not yet wired to per-tenant
  Gateway, child-tenant ACME HTTP-01 needs a namespace-scoped Issuer.
  All three are tracked as follow-up work rather than hidden failure
  modes.

packages/core/platform/values.yaml adds a comment block under
publishing.certificates.issuerName reminding operators of the
rate-limit consequences before they flip gateway.enabled=true on a
production cluster.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:58 +03:00
Aleksei Sviridkin
9505b08dd2
test(e2e): gateway.bats smoke-tests Cilium Gateway API wiring
Three bats test cases verify the end-to-end infrastructure brought up
by this PR:

1. Gateway API CRDs are installed (gatewayclasses, gateways, httproutes)
   and the 'cilium' GatewayClass is Accepted by the controller —
   proves gateway-api-crds installed and cilium.gatewayAPI.enabled
   propagated.
2. A minimal Gateway in tenant-test reconciles to Programmed and the
   controller materialises its cilium-gateway-<name> LoadBalancer
   Service — proves envoy.enabled kicked in and the data-plane wiring
   is live.
3. An HTTPRoute with a matching parentRef reaches Accepted status —
   proves Cilium's HTTPRoute attachment logic works.

Not covered (because the test harness has no reachable Let's Encrypt
endpoint and no way to flip cluster-wide gateway.enabled mid-pipeline
without trampling other tests):

- cert-manager HTTP-01 solver via gatewayHTTPRoute.
- The ValidatingAdmissionPolicy that enforces tenant hostname
  ownership (installed only when gateway.enabled=true at platform
  level).
- Full tenant.spec.gateway=true flow with extra/gateway chart and
  CiliumLoadBalancerIPPool.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:57 +03:00
Aleksei Sviridkin
bb3fc7cf26
feat(cozystack-basics): ValidatingAdmissionPolicy blocks cross-tenant hostname hijacking on Gateways
When gateway.enabled is true and publishing.host is set, cozystack-basics
now installs a ValidatingAdmissionPolicy (plus its Binding and a
ConfigMap that carries the apex host and publishing namespace as
parameters) that rejects any Gateway whose listeners reference a
hostname outside the tenant's own domain suffix.

Scheme:

- Publishing tenant namespace ($publishing.ingressName, default
  tenant-root): allowed suffix is the platform-wide $publishing.host.
- Any namespace prefixed with 'tenant-': allowed suffix is
  <name-stripped-of-tenant-prefix>.$publishing.host.
- Any other namespace: the CEL variable resolves to "" and every
  listener with a hostname is rejected — only cluster-admin-managed
  namespaces should be creating Gateways anyway.

The VAP is scoped to CREATE and UPDATE on gateway.networking.k8s.io/v1
(and v1beta1) Gateways cluster-wide; the Binding uses
validationActions: [Deny] with parameterNotFoundAction: Deny so that
if the ConfigMap is missing the policy fails safe instead of
open-allowing. messageExpression spells out the tenant-scoped allowed
suffix in the rejection, so operators can see at a glance why the
request was denied.

This closes the hostname-hijacking half of the multi-tenant threat
model on top of the namespace whitelist on Gateway listeners. The two
work in layers: the whitelist stops a non-authorised namespace from
attaching any HTTPRoute; the VAP stops an authorised namespace from
claiming someone else's hostname on its own Gateway.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:57 +03:00
Aleksei Sviridkin
142108e507
feat(dashboard,keycloak): attach to tenant Gateway via HTTPRoute when gateway.enabled
Each package now ships a conditional templates/httproute.yaml which is
rendered only when the cluster-wide gateway.enabled flag is true
(propagated via _cluster.gateway-enabled). The existing Ingress
template is wrapped in the inverse condition, so exactly one of the
two is rendered at a time and traffic does not split.

Dashboard: HTTPRoute hostname dashboard.<root-host>, backendRef
incloud-web-gatekeeper:8000 (same as the existing Ingress).

Keycloak: HTTPRoute hostname <ingress.host> (default
keycloak.<root-host>), backendRef keycloak-http:80.

Both routes parentRef the tenant Gateway named 'cozystack' in the
publishing namespace and rely on the namespace whitelist on Gateway
listeners (see 'fix(gateway): restrict tenant Gateway listener
allowedRoutes') to ensure only authorised namespaces can attach.

Known gap (follow-up): services that rely on TLS passthrough
(cozystack-api, vm-exportproxy, cdi-uploadproxy) still render their
existing Ingress regardless of gateway.enabled, because Gateway API
passthrough requires a separate TLSRoute + a dedicated
mode: Passthrough listener on the Gateway. Harbor and bucket are
tenant-scoped apps that need _namespace.gateway plumbing — also
follow-up.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:57 +03:00
Aleksei Sviridkin
6c2872af33
fix(gateway): restrict tenant Gateway listener allowedRoutes to an explicit namespace whitelist
Security fix: before this change the Gateway listeners used
'allowedRoutes.namespaces.from: Selector' with the tenant namespace
label, but when per-service HTTPRoutes had to live outside the
tenant namespace that label model broke — so the natural fix was to
relax to 'from: All'. That would have let any namespace in the
cluster attach an HTTPRoute with an arbitrary hostname to the
publishing tenant's Gateway, which is hostname hijacking across
tenants.

The correct solution is to pin each listener to an explicit namespace
whitelist using the built-in 'kubernetes.io/metadata.name' label that
kube-apiserver assigns to every namespace by default. Namespaces not
on the whitelist literally cannot attach HTTPRoutes to this Gateway,
so a non-publishing tenant cannot create a hijacking HTTPRoute in its
own namespace.

The whitelist is the union of:
- The publishing tenant namespace (always, implicitly added).
- 'publishing.gateway.attachedNamespaces' in the platform chart —
  a list of system namespaces that host exposed services and need to
  attach HTTPRoutes (cozy-cert-manager for ACME, cozy-dashboard,
  cozy-keycloak, cozy-system, cozy-harbor, cozy-bucket,
  cozy-kubevirt, cozy-kubevirt-cdi, cozy-monitoring,
  cozy-linstor-gui).

Plumbed through cozystack-values as '_cluster.gateway-attached-namespaces'
(CSV of namespaces). The gateway chart concats this CSV with the
release namespace, de-dupes via 'uniq', and emits the result as the
matchExpressions 'values' list on every listener.

Tests: 3 new helm-unittest cases in packages/extra/gateway/tests/
covering the default (publishing namespace only), CSV parsing with
empty-entry filtering, and the invariant that HTTPS listeners share
the whitelist with the HTTP listener.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:57 +03:00
Aleksei Sviridkin
77b57e8075
feat(cert-manager): switch HTTP-01 solver to gatewayHTTPRoute when gateway.enabled
Add a cluster-wide 'gateway.enabled' flag in packages/core/platform/values.yaml
(default false) and plumb it into cozystack-values as _cluster.gateway-enabled.
When set to true, the cert-manager ClusterIssuers in cert-manager-issuers
switch their http01 solver from the ingress-nginx backend (ingressClassName)
to the Gateway API backend (gatewayHTTPRoute). Both letsencrypt-prod and
letsencrypt-stage are updated in lockstep.

The new solver attaches to the tenant Gateway named 'cozystack' in the
publishing namespace via parentRefs with sectionName: http, so cert-manager
places its ephemeral HTTPRoute on the Gateway's HTTP (:80) listener. Path
matching on /.well-known/acme-challenge/ is more specific than the sibling
HTTPRoute that redirects HTTP to HTTPS, so ACME challenges arrive correctly.

Default behaviour (gateway.enabled=false) is unchanged — ingress-nginx
path remains the only solver on existing clusters.

Tests: packages/system/cert-manager-issuers/tests/solver_test.yaml adds
4 helm-unittest cases covering the ingress fallback, the gateway path for
both issuers, dns01 unaffected by the flag, and the empty-string default.
Picked up automatically by hack/helm-unit-tests.sh via the new Makefile
test target.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:57 +03:00
Aleksei Sviridkin
8afa6601c9
fix(tenant,gateway): regenerate values.schema.json and ApplicationDefinition CRD with cozyvalues-gen v1.3.0
The previous commit in this PR regenerated schema and CRD artefacts
using a local 'dev' build of cozyvalues-gen, which sorts
properties alphabetically. CI installs the pinned upstream release
(v1.3.0), which preserves values.yaml source order, so the committed
outputs diverged from what CI regenerates and pre-commit failed with
exit code 123.

Re-ran 'make generate' locally with the release binary from
https://github.com/cozystack/cozyvalues-gen/releases/tag/v1.3.0 to
produce the same output the CI pipeline expects.

Files refreshed:

- api/apps/v1alpha1/tenant/types.go — field order reverted to
  values.yaml source order (host, etcd, monitoring, ingress, gateway,
  seaweedfs, schedulingClass, resourceQuotas).
- packages/apps/tenant/values.schema.json — same reordering.
- packages/system/tenant-rd/cozyrds/tenant.yaml — regenerated from
  the refreshed schema, gateway bool field now properly sorted.
- packages/extra/gateway/values.schema.json — regenerated for the
  same reason; gateway package itself has only one field so the diff
  is cosmetic, but keeps the pipeline clean.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:40 +03:00
Aleksei Sviridkin
5dc416293f
feat(gateway): add per-tenant Gateway API package and tenant toggle
Introduces a full per-tenant Gateway API setup backed by Cilium. The
publishing tenant (the one whose namespace matches
publishing.ingressName) can opt into a Gateway via tenant.spec.gateway
instead of or alongside the existing Service.spec.externalIPs-based
ingress-nginx deployment.

Package layout:

- packages/extra/gateway/ — new Helm chart that renders, per release:
  * cert-manager Certificate for the apex and wildcard hostnames.
  * One Gateway with three listeners — HTTP:80 (ACME challenges and
    redirect), HTTPS:443 wildcard *.<host>, HTTPS:443 apex <host>.
    Each listener uses allowedRoutes.namespaces.from=Selector keyed on
    label cozystack.io/gateway=<tenant namespace> so system services
    can attach HTTPRoutes cross-namespace without ReferenceGrants.
  * HTTPRoute http-to-https-redirect (301) on the HTTP listener.
    cert-manager's own ACME solver HTTPRoute for
    /.well-known/acme-challenge/ will take precedence via Gateway
    API's more-specific-path matching rules.
  * CiliumLoadBalancerIPPool scoped to the tenant namespace and the
    Gateway's cilium-gateway-cozystack Service, so the IPs from
    publishing.externalIPs get announced via LB IPAM.
  * Render-time guards: fails loudly if released outside the
    publishing namespace or if publishing.host is unset.

- packages/apps/tenant/ — new values key 'gateway: false' and a
  conditional HelmRelease template that references the gateway
  ExternalArtifact when the tenant opts in. Mirrors the existing
  ingress toggle verbatim, including labels and Flux reconcile config.

- packages/core/platform/sources/gateway-application.yaml — new
  PackageSource cozystack.gateway-application with a single 'default'
  variant that wires extra/gateway as a tenant module and dependsOn
  cozystack.gateway-api-crds + cozystack.cert-manager.

- packages/core/platform/templates/bundles/system.yaml — loads
  cozystack.gateway-application in every system variant so the
  ExternalArtifact is materialised before any tenant tries to use it.

- api/apps/v1alpha1/tenant/types.go, values.schema.json, README.md,
  packages/system/tenant-rd/cozyrds/tenant.yaml — regenerated to
  include the new Gateway bool field on TenantSpec.

- packages/extra/gateway/tests/gateway_test.yaml — 6 helm-unittest
  cases covering the IPv4/IPv6/mixed IP paths, empty-entry filtering,
  the no-IPs-still-renders-Gateway path, the non-publishing-namespace
  guard, and the empty-host guard.

Default behaviour is unchanged: tenants default to gateway=false and
the platform only materialises the ExternalArtifact (no resources
inside any tenant namespace unless it opts in).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:39 +03:00
Aleksei Sviridkin
7f2354170e
feat(platform): install gateway-api-crds on the host cluster
Register a dedicated cozystack.gateway-api-crds PackageSource and wire
it into the system bundle before networking/cert-manager load. Without
this the host cluster has no gateway.networking.k8s.io CRDs, Cilium's
Gateway API controller cannot bootstrap (no GatewayClass registered),
and cert-manager's gateway-shim controller silently disables Gateway
API certificate automation.

- New PackageSource cozystack.gateway-api-crds → installs
  packages/system/gateway-api-crds into namespace cozy-gateway-api.
- cozystack.networking (variants cilium, cilium-kilo, cilium-generic,
  kubeovn-cilium, kubeovn-cilium-generic) dependsOn
  cozystack.gateway-api-crds so Cilium only starts once the CRDs are
  present.
- cozystack.cert-manager dependsOn cozystack.gateway-api-crds, closing
  the startup race that the #2208 discussion flagged — cert-manager's
  gateway-shim discovers CRDs at process start, so racing with CRD
  install left Gateway API solvers permanently disabled until the pod
  was restarted.
- system bundle loads cozystack.gateway-api-crds unconditionally for
  every variant (isp-full, isp-hosted, isp-full-generic).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:39 +03:00
Aleksei Sviridkin
2eb7bccee4
feat(cilium): enable embedded envoy and Gateway API controller
Flip envoy.enabled and gatewayAPI.enabled on the vendored Cilium chart.
Cilium becomes the Gateway API implementation for Cozystack:
cilium-envoy DaemonSet starts on each node (approximately 100MB RAM
per node in steady state), and the Cilium operator begins reconciling
gateway.networking.k8s.io resources and publishing the built-in
'cilium' GatewayClass.

This value flip alone does not create any Gateway objects. The
platform installs only an idle Envoy DaemonSet and a GatewayClass
until per-tenant gateway manifests are actually created by the
platform chart.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:39 +03:00
Aleksei Sviridkin
47aca33c5a
chore(gateway-api-crds): bump to v1.5.1
Align vendored Gateway API CRDs with the latest stable upstream
release (v1.5.1, Oct 2026). TLSRoute graduates to GA in v1.5, still
shipped from the experimental channel to keep the superset of
optional fields. Cilium 1.19 consumes Gateway API CRDs through
standard backward-compatible versioning, so a v1.5 schema is
compatible with the Cilium 1.19 controller.

Notable additions over the previous v1.2.0 vendoring:

- ListenerSet (was XListenerSet / GEP-1713) — shared Gateway with
  listeners delegated from separate LS objects. Will enable a
  one-Gateway-per-cluster topology for the upcoming Gateway API work.
- BackendTLSPolicy for upstream-to-upstream TLS termination.
- XMesh (experimental) for mesh-specific routing hooks.
- ValidatingAdmissionPolicy 'safe-upgrades.gateway.networking.k8s.io'
  plus its binding, shipped by upstream to prevent mixing channels
  and downgrading CRD versions.

Stepping stone for the upcoming Gateway API via Cilium work; isolated
from that change so it can be rolled back on its own if any downstream
consumer breaks on the newer schema.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:39 +03:00
209 changed files with 10434 additions and 13858 deletions

View file

@ -1,7 +1,7 @@
---
name: Bug report
about: Create a report to help us improve
labels: 'kind/bug'
labels: 'bug'
assignees: ''
---

View file

@ -1,10 +1,9 @@
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Scopes for system components: dashboard, platform, cilium, kube-ovn, linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft.

371
.github/labels.yml vendored
View file

@ -1,371 +0,0 @@
# Cozystack repository labels
#
# Label conventions follow the Kubernetes scheme:
# https://github.com/kubernetes/test-infra/blob/master/label_sync/labels.md
#
# Synced into the repository by .github/workflows/labels.yaml
# (EndBug/label-sync@v2). Edit this file via pull request — UI changes
# will be overwritten on the next sync.
#
# Constraints (enforced by the validate job in labels.yaml):
# - description ≤ 100 characters (GitHub REST API limit)
# - color is a 6-character hex string (no leading #)
# - label names are unique
# - aliases do not collide with top-level names
#
# Categories:
# kind/ issue or PR type
# priority/ urgency
# triage/ review state
# lifecycle/ issue or PR lifecycle
# area/ subsystem; extensible — add when 3+ open issues exist
# do-not-merge/ PR merge blockers
# security/ security-finding severity and status (Cozystack-specific)
# size: PR size (auto-applied)
#
# `aliases:` lets EndBug/label-sync rename existing labels without losing
# references on already-tagged issues and PRs.
#
# GitHub-default labels not migrated here (`wontfix`, `invalid`) currently
# carry zero issues/PRs in this repo and will be removed in a follow-up
# cleanup PR rather than aliased to a different namespace.
# ──────────────────────────────────────────────
# kind/ — issue or PR type
# ──────────────────────────────────────────────
- name: kind/bug
color: 'd73a4a'
description: Categorizes issue or PR as related to a bug
aliases: ['bug']
- name: kind/feature
color: 'a2eeef'
description: Categorizes issue or PR as related to a new feature
aliases: ['enhancement']
- name: kind/documentation
color: '0075ca'
description: Categorizes issue or PR as related to documentation
aliases: ['documentation']
- name: kind/support
color: 'd876e3'
description: Categorizes issue as a support question
aliases: ['question']
- name: kind/cleanup
color: 'c7def8'
description: Categorizes issue or PR as related to cleanup of code, process, or technical debt
- name: kind/regression
color: 'e11d21'
description: Categorizes issue or PR as related to a regression from a prior release
- name: kind/flake
color: 'f7c6c7'
description: Categorizes issue or PR as related to a flaky test
- name: kind/failing-test
color: 'e11d21'
description: Categorizes issue or PR as related to a consistently or frequently failing test
- name: kind/api-change
color: 'c7def8'
description: Categorizes issue or PR as related to adding, removing, or otherwise changing an API
- name: kind/breaking-change
color: 'e11d21'
description: Indicates the change introduces a breaking API or behaviour change
# ──────────────────────────────────────────────
# priority/ — urgency
# ──────────────────────────────────────────────
- name: priority/critical-urgent
color: 'e11d21'
description: Highest priority. Must be actively worked on as someone's top priority right now
- name: priority/important-soon
color: 'eb6420'
description: Must be staffed and worked on either currently, or very soon, ideally in time for the next release
- name: priority/important-longterm
color: 'fbca04'
description: Important over the long term, but may not be staffed and/or may need multiple releases to complete
- name: priority/backlog
color: 'fef2c0'
description: General backlog priority. Lower than priority/important-longterm
# ──────────────────────────────────────────────
# triage/ — review state
# ──────────────────────────────────────────────
- name: triage/needs-triage
color: 'ededed'
description: Indicates an issue needs triage by a maintainer
- name: triage/accepted
color: '0e8a16'
description: Indicates an issue is ready to be actively worked on
- name: triage/needs-information
color: 'fbca04'
description: Indicates an issue needs more information in order to work on it
- name: triage/not-reproducible
color: 'fbca04'
description: Indicates an issue can not be reproduced as described
- name: triage/duplicate
color: 'cfd3d7'
description: Indicates an issue is a duplicate of another issue
aliases: ['duplicate']
- name: triage/unresolved
color: 'cfd3d7'
description: Indicates an issue that can not or will not be resolved
# ──────────────────────────────────────────────
# lifecycle/ — issue or PR lifecycle
# ──────────────────────────────────────────────
- name: lifecycle/active
color: '1d76db'
description: Indicates that an issue or PR is actively being worked on by a contributor
- name: lifecycle/frozen
color: 'db5dd6'
description: Indicates that an issue or PR should not be auto-closed due to staleness
aliases: ['frozen']
- name: lifecycle/stale
color: 'dadada'
description: Denotes an issue or PR has remained open with no activity and has become stale
aliases: ['stale']
- name: lifecycle/rotten
color: '795548'
description: Denotes an issue or PR that has aged beyond stale and will be auto-closed
# ──────────────────────────────────────────────
# area/ — subsystem (extensible)
# Add a new area/* when there are 3+ open issues on the topic.
# ──────────────────────────────────────────────
- name: area/api
color: 'bfd4f2'
description: Issues or PRs related to the cozystack-api aggregated API server
- name: area/ai
color: 'bfd4f2'
description: Issues or PRs related to AI agent guides, AGENTS.md, docs/agents/
- name: area/build
color: 'bfd4f2'
description: Issues or PRs related to image build infrastructure, multi-arch support
- name: area/ci
color: 'bfd4f2'
description: Issues or PRs related to CI workflows, GitHub Actions, automation
- name: area/dashboard
color: 'bfd4f2'
description: Issues or PRs related to the dashboard / UI
- name: area/extra
color: 'bfd4f2'
description: Issues or PRs related to tenant-specific modules (packages/extra/)
- name: area/database
color: 'bfd4f2'
description: Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse)
- name: area/kubernetes
color: 'bfd4f2'
description: Issues or PRs related to the tenant Kubernetes app
- name: area/monitoring
color: 'bfd4f2'
description: Issues or PRs related to the monitoring stack (vlogs, vmstack, grafana, workloadmonitor)
- name: area/networking
color: 'bfd4f2'
description: Issues or PRs related to networking (ingress, gateway, vpn, metallb, cilium, kube-ovn)
- name: area/platform
color: 'bfd4f2'
description: Issues or PRs related to platform infrastructure (bundle, flux, talos, installer)
- name: area/release
color: 'bfd4f2'
description: Issues or PRs related to release tooling (changelog, backport, release pipeline)
- name: area/storage
color: 'bfd4f2'
description: Issues or PRs related to storage (linstor, seaweedfs, bucket, velero, harbor)
- name: area/testing
color: 'bfd4f2'
description: Issues or PRs related to testing (e2e, bats, unit tests)
- name: area/virtualization
color: 'bfd4f2'
description: Issues or PRs related to virtualization (kubevirt, cdi, vmi, vm-import)
- name: area/uncategorized
color: 'fbca04'
description: PR auto-labeler could not map title scope to a known area/*; please review
# ──────────────────────────────────────────────
# do-not-merge/ — PR merge blockers (Prow convention)
# ──────────────────────────────────────────────
- name: do-not-merge/work-in-progress
color: 'e11d21'
description: Indicates that a PR should not merge because it is a work in progress
# Both legacy spellings collapse here. EndBug processes aliases sequentially;
# the second rename hits a name collision and logs a warning — the legacy
# label survives and gets cleaned up in the follow-up dedup PR.
aliases: ['do-not-merge', 'do not merge']
- name: do-not-merge/hold
color: 'e11d21'
description: Indicates that a PR should not merge because someone has issued /hold
# ──────────────────────────────────────────────
# Cozystack-specific (preserved)
# ──────────────────────────────────────────────
- name: epic
color: 'a335ee'
description: A large development increment that brings definite value to Cozystack users
- name: community
color: '97458a'
description: Community contributions are welcome in this issue
- name: help wanted
color: '008672'
description: Extra attention is needed
- name: good first issue
color: '7057ff'
description: Good for newcomers
- name: quality-of-life
color: 'aaaaaa'
description: QoL improvements
- name: upstream-issue
color: 'aaaaaa'
description: Requires resolving an issue in an upstream project
- name: backport
color: 'fbca04'
description: Should change be backported on previous release
- name: backport-previous
color: 'fbd876'
description: Backport target — previous release line
- name: release
color: 'aaaaaa'
description: Releasing a new Cozystack version
- name: automated
color: 'ededed'
description: Created by automation
- name: debug
color: '704479'
description: Debugging in progress
- name: sponsored
color: '00ff00'
description: Sponsored work
- name: lgtm
color: '238636'
description: This PR has been approved by a maintainer
- name: ok-to-test
color: '00ff00'
description: Indicates a non-member PR is safe to run CI on
# ──────────────────────────────────────────────
# size: — PR size (auto-applied by sizing bot)
# ──────────────────────────────────────────────
- name: 'size:XS'
color: '00ff00'
description: This PR changes 0-9 lines, ignoring generated files
- name: 'size:S'
color: '77b800'
description: This PR changes 10-29 lines, ignoring generated files
- name: 'size:M'
color: 'ebb800'
description: This PR changes 30-99 lines, ignoring generated files
- name: 'size:L'
color: 'eb9500'
description: This PR changes 100-499 lines, ignoring generated files
- name: 'size:XL'
color: 'ff823f'
description: This PR changes 500-999 lines, ignoring generated files
- name: 'size:XXL'
color: 'ffb8b8'
description: This PR changes 1000+ lines, ignoring generated files
# ──────────────────────────────────────────────
# security/ — security-finding severity and status
# ──────────────────────────────────────────────
- name: security
color: 'aaaaaa'
description: Security-related issues and features
- name: security/critical
color: 'd73a4a'
description: Critical security vulnerability
- name: security/high
color: 'e99695'
description: High severity security finding
- name: security/medium
color: 'f9c513'
description: Medium severity security finding
- name: security/low
color: '0e8a16'
description: Low severity security finding
- name: security/triage-needed
color: 'fbca04'
description: Needs security triage
- name: security/confirmed
color: '1d76db'
description: Confirmed vulnerability
- name: security/false-positive
color: 'c5def5'
description: Triaged as false positive
- name: security/accepted-risk
color: 'bfd4f2'
description: Risk accepted with justification
- name: security/in-progress
color: '0075ca'
description: Fix in progress
- name: security/fixed
color: '0e8a16'
description: Fix released

View file

@ -1,61 +0,0 @@
name: Codegen Drift Check
on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- 'api/**'
- 'pkg/apis/**'
- 'pkg/generated/**'
- 'internal/crdinstall/manifests/**'
- 'packages/system/cozystack-controller/definitions/**'
- 'packages/system/application-definition-crd/definition/**'
- 'packages/system/backup-controller/definitions/**'
- 'packages/system/backupstrategy-controller/definitions/**'
- 'hack/update-codegen.sh'
- 'hack/boilerplate.go.txt'
- 'Makefile'
- 'go.mod'
- 'go.sum'
- '.github/workflows/codegen-drift.yml'
concurrency:
group: codegen-drift-${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
codegen-drift:
name: Verify generated code is up to date
runs-on: ubuntu-22.04
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Pre-fetch k8s.io/code-generator module
# hack/update-codegen.sh sources kube_codegen.sh from the Go module cache.
# The module is not declared in go.mod, so fetch it explicitly at the
# version pinned in the script.
run: |
version=$(grep -oP 'code-generator@\Kv[0-9.]+' hack/update-codegen.sh)
tmpdir=$(mktemp -d)
cd "$tmpdir"
go mod init codegen-fetch
go get "k8s.io/code-generator@${version}"
- name: Run make generate
run: make generate
- name: Fail on drift
run: |
if [ -n "$(git status --porcelain)" ]; then
echo "::error::'make generate' produced changes. Run 'make generate' locally and commit the result."
git status --short
git diff --color=always
exit 1
fi

View file

@ -1,84 +0,0 @@
name: Labels
on:
pull_request:
paths:
- .github/labels.yml
- .github/workflows/labels.yaml
push:
branches: [main]
paths:
- .github/labels.yml
- .github/workflows/labels.yaml
workflow_dispatch:
schedule:
- cron: '17 4 * * 1' # Mondays at 04:17 UTC
permissions:
contents: read
concurrency:
group: labels-sync
cancel-in-progress: false
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Validate labels.yml schema
run: |
python3 - <<'PY'
import re, sys, yaml
path = '.github/labels.yml'
data = yaml.safe_load(open(path))
errors = []
# 1. description ≤ 100 chars (GitHub REST API limit)
for label in data:
desc = label.get('description', '') or ''
if len(desc) > 100:
errors.append(f"{label['name']}: description {len(desc)} chars (max 100)")
# 2. color is 6-char hex without leading #
for label in data:
color = label.get('color', '') or ''
if not re.match(r'^[0-9A-Fa-f]{6}$', color):
errors.append(f"{label['name']}: bad color {color!r} (must be 6-char hex without #)")
# 3. unique top-level names
names = [label['name'] for label in data]
dups = sorted({n for n in names if names.count(n) > 1})
for n in dups:
errors.append(f"duplicate name: {n}")
# 4. aliases do not collide with any top-level name
name_set = set(names)
for label in data:
for alias in (label.get('aliases') or []):
if alias in name_set:
errors.append(f"alias {alias!r} (under {label['name']!r}) collides with a top-level name")
if errors:
for err in errors:
print(f"::error::{err}")
sys.exit(1)
print(f"labels.yml schema OK ({len(data)} labels)")
PY
sync:
needs: validate
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/checkout@v4
- uses: EndBug/label-sync@v2
with:
config-file: .github/labels.yml
delete-other-labels: false

View file

@ -1,206 +0,0 @@
name: PR Auto-Label
on:
pull_request_target:
types: [opened, edited, reopened, synchronize]
permissions:
contents: read
pull-requests: write
jobs:
label:
runs-on: ubuntu-latest
steps:
- name: Apply labels from PR title
uses: actions/github-script@v7
with:
script: |
// Conventional Commits types accepted by Cozystack (per docs/agents/contributing.md):
// feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert
// Mapping below maps a subset to kind/* — types not listed do not produce a kind/*.
const typeToKind = {
feat: 'kind/feature',
fix: 'kind/bug',
docs: 'kind/documentation',
chore: 'kind/cleanup',
refactor: 'kind/cleanup',
// style, perf, test, build, ci, revert — no kind mapping
};
// scope -> area/* mapping. Keys are the scopes observed in cozystack issues
// and PRs. Add new entries when a scope recurs (3+ times).
const scopeToArea = {
// area/api
'api': 'area/api',
'cozystack-api': 'area/api',
// area/ai
'agents': 'area/ai',
'ai': 'area/ai',
// area/build
'build': 'area/build',
// area/ci
'ci': 'area/ci',
// area/dashboard
'dashboard': 'area/dashboard',
// area/database
'postgres': 'area/database',
'postgres-operator': 'area/database',
'mariadb': 'area/database',
'mariadb-operator': 'area/database',
'redis': 'area/database',
'etcd': 'area/database',
'kafka': 'area/database',
'clickhouse': 'area/database',
// area/extra
'extra': 'area/extra',
// area/kubernetes
'kubernetes': 'area/kubernetes',
'apps/kubernetes': 'area/kubernetes',
// area/monitoring
'monitoring': 'area/monitoring',
'vlogs': 'area/monitoring',
'vmstack': 'area/monitoring',
'grafana': 'area/monitoring',
'workloadmonitor': 'area/monitoring',
// area/networking
'ingress': 'area/networking',
'ingress-nginx': 'area/networking',
'gateway': 'area/networking',
'vpn': 'area/networking',
'metallb': 'area/networking',
'cilium': 'area/networking',
'kube-ovn': 'area/networking',
'tcp-balancer': 'area/networking',
'securitygroups': 'area/networking',
'cozy-proxy': 'area/networking',
// area/platform
'platform': 'area/platform',
'bundle': 'area/platform',
'flux': 'area/platform',
'fluxcd': 'area/platform',
'cluster-api': 'area/platform',
'talos': 'area/platform',
'installer': 'area/platform',
'cozyctl': 'area/platform',
'cozystack-engine': 'area/platform',
'cozy-lib': 'area/platform',
// area/release
'backport': 'area/release',
'release': 'area/release',
// area/storage
'seaweedfs': 'area/storage',
'seaweedfs-cosi-driver': 'area/storage',
'bucket': 'area/storage',
'linstor': 'area/storage',
'velero': 'area/storage',
'harbor': 'area/storage',
'backups': 'area/storage',
// area/testing
'tests': 'area/testing',
'e2e': 'area/testing',
// area/virtualization
'kubevirt': 'area/virtualization',
'cdi': 'area/virtualization',
'vmi': 'area/virtualization',
'vm-import': 'area/virtualization',
'virtual-machine': 'area/virtualization',
'hami': 'area/virtualization',
'gpu-operator': 'area/virtualization',
};
const pr = context.payload.pull_request;
const title = pr.title || '';
const body = pr.body || '';
const existing = new Set((pr.labels || []).map(l => l.name));
const toAdd = new Set();
// 1. Strip "[Backport release-1.x]" prefix if present.
const backportMatch = title.match(/^\[Backport ([^\]]+)\]\s+(.+)$/);
const cleanTitle = backportMatch ? backportMatch[2] : title;
if (backportMatch) {
toAdd.add('area/release');
toAdd.add('backport');
}
// 2. Try Conventional Commits form: type(scope)?(!)?: description
const conv = cleanTitle.match(/^([a-z]+)(?:\(([^)]+)\))?(!)?:\s*.+$/);
// 3. Fall back to bracket form: [scope] description
const bracket = !conv && cleanTitle.match(/^\[([^\]]+)\]\s+.+$/);
let type = null, scopeStr = null, breaking = false;
if (conv) {
type = conv[1];
scopeStr = conv[2] || null;
breaking = !!conv[3];
} else if (bracket) {
scopeStr = bracket[1];
}
// 4. Detect BREAKING CHANGE: or BREAKING-CHANGE: footer in body.
// Conventional Commits 1.0 spec item 16 treats them as synonymous.
if (/^BREAKING[ -]CHANGE:/m.test(body)) {
breaking = true;
}
// 5. Apply kind/* from type.
if (type) {
if (typeToKind[type]) {
toAdd.add(typeToKind[type]);
} else {
core.warning(`type "${type}" has no kind/* mapping — typo or new type? See .github/workflows/pr-labeler.yaml typeToKind`);
}
}
// 6. Apply area/* from scope. Composite scopes split on comma.
const scopes = (scopeStr || '')
.split(/,\s*/)
.map(s => s.trim())
.filter(Boolean);
for (const s of scopes) {
if (scopeToArea[s]) {
toAdd.add(scopeToArea[s]);
} else {
core.warning(`scope "${s}" has no area/* mapping — consider extending scopeToArea in .github/workflows/pr-labeler.yaml if it recurs`);
}
}
// 7. kind/breaking-change.
if (breaking) {
toAdd.add('kind/breaking-change');
}
// 8. Fallback: no area/* applied -> area/uncategorized.
const hasArea = [...toAdd].some(l => l.startsWith('area/'));
if (!hasArea) {
toAdd.add('area/uncategorized');
}
// 9. Additive only — never remove existing labels.
const newLabels = [...toAdd].filter(l => !existing.has(l));
if (newLabels.length === 0) {
core.info('No new labels to apply');
return;
}
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
labels: newLabels,
});
core.info(`Applied labels: ${newLabels.join(', ')}`);

View file

@ -223,7 +223,7 @@ jobs:
repo: context.repo.repo,
head,
base,
title: `chore(release): cut v${version}`,
title: `Release v${version}`,
body: `This PR prepares the release \`v${version}\`.`,
draft: false
});
@ -411,7 +411,7 @@ jobs:
repo: context.repo.repo,
head: changelogBranch,
base: baseBranch,
title: `docs(release): add changelog for v${version}`,
title: `docs: add changelog for v${version}`,
body: `This PR adds the changelog for release \`v${version}\`.\n\n✅ Changelog has been automatically generated in \`docs/changelogs/v${version}.md\`.`,
draft: false
});
@ -421,7 +421,7 @@ jobs:
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.data.number,
labels: ['kind/documentation', 'automated']
labels: ['documentation', 'automated']
});
console.log(`Created PR #${pr.data.number} for changelog`);

View file

@ -1,17 +1,6 @@
repos:
- repo: local
hooks:
- id: run-make-generate-root
name: Run 'make generate' at repo root
entry: |
flock -x .git/pre-commit.lock sh -c '
echo "Running make generate at repo root"
make generate || exit $?
git diff --color=always | cat
'
language: system
files: ^(api/|pkg/apis/|pkg/generated/|internal/crdinstall/manifests/|packages/system/cozystack-controller/definitions/|packages/system/application-definition-crd/definition/|packages/system/backup-controller/definitions/|packages/system/backupstrategy-controller/definitions/|hack/update-codegen\.sh$|hack/boilerplate\.go\.txt$|Makefile$)
pass_filenames: false
- id: run-make-generate
name: Run 'make generate' in all app directories
entry: |

View file

@ -27,12 +27,6 @@ working with the **Cozystack** project.
- Read: [`contributing.md`](./docs/agents/contributing.md)
- Action: Read the file to understand git workflow, commit format, PR process
- **Issue and PR labeling, triage** (e.g., "label this issue", "what label should I use", "triage this", "categorize")
- Read: [`.github/labels.yml`](./.github/labels.yml)
- Action: Use labels defined there. Conventions follow the Kubernetes scheme — `kind/*` (type), `area/*` (subsystem), `priority/*` (urgency), `triage/*` (review state), `lifecycle/*` (auto-close), `do-not-merge/*` (PR blockers), `security/*` (severity)
- For `area/*`: accuracy outweighs reuse. If no existing `area/*` truly fits the change, propose a new one via PR (extend `.github/labels.yml` and the scope mapping in `.github/workflows/pr-labeler.yaml`) — do not shoehorn the change into a wrong area. `area/uncategorized` is the auto-labeler fallback; treat it as a signal to pick a fit, create a new area, or correct the PR title
- PR titles: a Conventional Commits header (`type(scope): description`, types from [`contributing.md`](./docs/agents/contributing.md)) auto-applies `kind/*` and `area/*` via `.github/workflows/pr-labeler.yaml`. Append `!` (or add a `BREAKING CHANGE:` footer) to apply `kind/breaking-change`
**Important rules:**
- ✅ **ONLY read the file if the task matches the documented process scope** - do not read files for tasks that don't match their purpose
- ✅ **ALWAYS read the file FIRST** before starting the task (when applicable)

View file

@ -22,7 +22,6 @@ build: build-deps
make -C packages/system/lineage-controller-webhook image
make -C packages/system/cilium image
make -C packages/system/linstor image
make -C packages/system/linstor-gui image
make -C packages/system/kubeovn-webhook image
make -C packages/system/kubeovn-plunger image
make -C packages/system/dashboard image
@ -83,19 +82,11 @@ test:
make -C packages/core/testing apply
make -C packages/core/testing test
unit-tests: helm-unit-tests bats-unit-tests go-unit-tests
unit-tests: helm-unit-tests bats-unit-tests
helm-unit-tests:
hack/helm-unit-tests.sh
# Scoped go test over the cozystack-api surface that this repo owns. Kept
# narrow intentionally - running `go test ./...` pulls in generated code
# round-trip suites whose behavior depends on tool versions outside this
# repo's control (kubebuilder, openapi-gen, etc.) and is better exercised
# from their generator workflows.
go-unit-tests:
go test ./pkg/registry/... ./pkg/config/... ./pkg/cmd/server/...
# Discover every hack/*.bats file that is NOT an e2e test and run it
# through cozytest.sh. Drop a new *.bats file in hack/ and it is picked
# up automatically on the next `make unit-tests` run.

View file

@ -36,9 +36,6 @@ type ConfigSpec struct {
// Kubernetes control-plane configuration.
// +kubebuilder:default:={}
ControlPlane ControlPlane `json:"controlPlane"`
// Optional image overrides for air-gapped or rate-limited registries.
// +kubebuilder:default:={}
Images Images `json:"images"`
}
type APIServer struct {
@ -69,9 +66,6 @@ type Addons struct {
// NVIDIA GPU Operator.
// +kubebuilder:default:={}
GpuOperator GPUOperatorAddon `json:"gpuOperator"`
// HAMi GPU virtualization middleware.
// +kubebuilder:default:={}
Hami HAMiAddon `json:"hami"`
// Ingress-NGINX controller.
// +kubebuilder:default:={}
IngressNginx IngressNginxAddon `json:"ingressNginx"`
@ -163,21 +157,6 @@ type GatewayAPIAddon struct {
Enabled bool `json:"enabled"`
}
type HAMiAddon struct {
// Enable HAMi (requires GPU Operator).
// +kubebuilder:default:=false
Enabled bool `json:"enabled"`
// Custom Helm values overrides.
// +kubebuilder:default:={}
ValuesOverride k8sRuntime.RawExtension `json:"valuesOverride"`
}
type Images struct {
// Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag.
// +kubebuilder:default:=""
WaitForKubeconfig string `json:"waitForKubeconfig,omitempty"`
}
type IngressNginxAddon struct {
// Enable the controller (requires nodes labeled `ingress-nginx`).
// +kubebuilder:default:=false

View file

@ -49,7 +49,6 @@ func (in *Addons) DeepCopyInto(out *Addons) {
in.Fluxcd.DeepCopyInto(&out.Fluxcd)
out.GatewayAPI = in.GatewayAPI
in.GpuOperator.DeepCopyInto(&out.GpuOperator)
in.Hami.DeepCopyInto(&out.Hami)
in.IngressNginx.DeepCopyInto(&out.IngressNginx)
in.MonitoringAgents.DeepCopyInto(&out.MonitoringAgents)
in.Velero.DeepCopyInto(&out.Velero)
@ -136,7 +135,6 @@ func (in *ConfigSpec) DeepCopyInto(out *ConfigSpec) {
}
in.Addons.DeepCopyInto(&out.Addons)
in.ControlPlane.DeepCopyInto(&out.ControlPlane)
out.Images = in.Images
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigSpec.
@ -262,37 +260,6 @@ func (in *GatewayAPIAddon) DeepCopy() *GatewayAPIAddon {
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *HAMiAddon) DeepCopyInto(out *HAMiAddon) {
*out = *in
in.ValuesOverride.DeepCopyInto(&out.ValuesOverride)
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HAMiAddon.
func (in *HAMiAddon) DeepCopy() *HAMiAddon {
if in == nil {
return nil
}
out := new(HAMiAddon)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Images) DeepCopyInto(out *Images) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Images.
func (in *Images) DeepCopy() *Images {
if in == nil {
return nil
}
out := new(Images)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *IngressNginxAddon) DeepCopyInto(out *IngressNginxAddon) {
*out = *in

View file

@ -92,9 +92,6 @@ type Bootstrap struct {
// Timestamp (RFC3339) for point-in-time recovery; empty means latest.
// +kubebuilder:default:=""
RecoveryTime string `json:"recoveryTime,omitempty"`
// Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name.
// +kubebuilder:default:=""
ServerName string `json:"serverName,omitempty"`
}
type Database struct {

View file

@ -29,6 +29,9 @@ type ConfigSpec struct {
// Deploy own Ingress Controller.
// +kubebuilder:default:=false
Ingress bool `json:"ingress"`
// Deploy own Gateway API Gateway (backed by Cilium Gateway API controller).
// +kubebuilder:default:=false
Gateway bool `json:"gateway"`
// Deploy own SeaweedFS.
// +kubebuilder:default:=false
Seaweedfs bool `json:"seaweedfs"`

View file

@ -26,9 +26,6 @@ type ConfigSpec struct {
// Ports to forward from outside the cluster.
// +kubebuilder:default:={22}
ExternalPorts []int `json:"externalPorts,omitempty"`
// Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect.
// +kubebuilder:default:=true
ExternalAllowICMP bool `json:"externalAllowICMP"`
// Requested running state of the VirtualMachineInstance
// +kubebuilder:default:="Always"
RunStrategy RunStrategy `json:"runStrategy"`

View file

@ -620,11 +620,6 @@ func (in *RestoreJobSpec) DeepCopyInto(out *RestoreJobSpec) {
*out = new(v1.TypedLocalObjectReference)
(*in).DeepCopyInto(*out)
}
if in.Options != nil {
in, out := &in.Options, &out.Options
*out = new(runtime.RawExtension)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreJobSpec.

View file

@ -132,16 +132,6 @@ type ComponentInstall struct {
// DependsOn is a list of component names that must be installed before this component
// +optional
DependsOn []string `json:"dependsOn,omitempty"`
// UpgradeCRDs controls how CRDs from the chart's crds/ directory are
// handled on HelmRelease upgrades. Maps to HelmRelease.Spec.Upgrade.CRDs.
// Empty string (default) preserves the helm-controller default (Skip).
// Use "CreateReplace" for operators that evolve their CRD set between
// versions. Warning: CreateReplace overwrites CRDs and may cause data
// loss if upstream drops fields from a CRD with live objects.
// +optional
// +kubebuilder:validation:Enum=Skip;Create;CreateReplace
UpgradeCRDs string `json:"upgradeCRDs,omitempty"`
}
// Component defines a single Helm release component within a package source

View file

@ -1,819 +0,0 @@
{
"uid": "gpu-efficiency",
"title": "GPU Efficiency Score",
"description": "Tensor saturation, util/watt and throttling — reveals inefficient GPU workloads",
"tags": [
"gpu",
"efficiency",
"finops"
],
"timezone": "browser",
"editable": true,
"graphTooltip": 1,
"time": {
"from": "now-1h",
"to": "now"
},
"fiscalYearStartMonth": 0,
"schemaVersion": 42,
"panels": [
{
"type": "row",
"collapsed": false,
"title": "Overall efficiency metrics",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 0
},
"id": 1,
"panels": []
},
{
"type": "stat",
"id": 2,
"targets": [
{
"expr": "avg(gpu:tensor_saturation:avg5m) * 100",
"refId": "A"
}
],
"title": "Avg Tensor Saturation",
"description": "Mean tensor core saturation across all GPUs. \u003c10% means GPUs are used inefficiently (workloads could move to CPU or optimize their code). Cluster-wide — DCGM metrics cannot be attributed to workload namespaces.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 5,
"w": 8,
"x": 0,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "area",
"colorMode": "background",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "red"
},
{
"value": 10,
"color": "orange"
},
{
"value": 30,
"color": "yellow"
},
{
"value": 60,
"color": "green"
}
]
}
},
"overrides": []
}
},
{
"type": "stat",
"id": 3,
"targets": [
{
"expr": "avg(gpu:util_per_watt:avg5m)",
"refId": "A"
}
],
"title": "Avg Utilization per Watt",
"description": "NVML utilization % per watt across all GPUs. Higher value = more efficient workload. Cluster-wide — DCGM metrics cannot be attributed to workload namespaces.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 5,
"w": 8,
"x": 8,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "area",
"colorMode": "background",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "none",
"decimals": 3,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "red"
},
{
"value": 0.5,
"color": "yellow"
},
{
"value": 1,
"color": "green"
}
]
}
},
"overrides": []
}
},
{
"type": "stat",
"id": 4,
"targets": [
{
"expr": "avg(gpu:power_throttle_fraction:rate5m)",
"refId": "A"
}
],
"title": "Avg Power Throttling",
"description": "Fraction of time GPUs hit the TDP cap and lose performance. \u003e5% means tenants underutilize billed FLOPS. Cluster-wide — rule aggregates by (Hostname, gpu, UUID) and drops namespace label.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 5,
"w": 8,
"x": 16,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "area",
"colorMode": "background",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "percentunit",
"decimals": 2,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 0.05,
"color": "yellow"
},
{
"value": 0.2,
"color": "red"
}
]
}
},
"overrides": []
}
},
{
"type": "row",
"collapsed": false,
"title": "NVML vs Tensor (mismatch detector)",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 6
},
"id": 10,
"panels": []
},
{
"type": "timeseries",
"id": 11,
"targets": [
{
"expr": "DCGM_FI_DEV_GPU_UTIL",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "NVML GPU Utilization",
"description": "Classic utilization metric. Shows activity of any engine (SM, copy, encoder). Cluster-wide — DCGM namespace is the exporter's own namespace, not the workload namespace.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 7
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 12,
"targets": [
{
"expr": "DCGM_FI_PROF_PIPE_TENSOR_ACTIVE * 100",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "Tensor Pipe Active",
"description": "Real tensor core load (HMMA). For AI/LLM inference it should be ≥20%, otherwise the workload is unoptimized. Cluster-wide — DCGM namespace is the exporter's own namespace, not the workload namespace.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 7
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "row",
"collapsed": false,
"title": "Per-GPU ranking",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 15
},
"id": 20,
"panels": []
},
{
"type": "table",
"id": 21,
"targets": [
{
"expr": "topk(20, gpu:tensor_saturation:avg5m * 100)",
"instant": true,
"range": false,
"format": "table",
"refId": "A"
}
],
"title": "Tensor Saturation per GPU (5m avg)",
"description": "Which GPUs are exercising tensor cores and which are not. Sorted descending. Grouped by Hostname/gpu/UUID — DCGM metrics cannot be attributed to workload namespaces.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 16
},
"repeatDirection": "h",
"transformations": [
{
"id": "organize",
"options": {
"excludeByName": {
"DCGM_FI_DRIVER_VERSION": true,
"Time": true,
"__name__": true,
"cluster": true,
"container": true,
"device": true,
"endpoint": true,
"gpu_driver_version": true,
"instance": true,
"job": true,
"modelName": true,
"namespace": true,
"pci_bus_id": true,
"pod": true,
"prometheus": true,
"service": true,
"tenant": true,
"tier": true,
"uid": true,
"unit": true
},
"indexByName": {
"Hostname": 0,
"UUID": 2,
"Value": 3,
"gpu": 1
},
"renameByName": {
"Hostname": "Node",
"UUID": "UUID",
"Value": "Saturation",
"gpu": "GPU"
}
}
}
],
"options": {
"frameIndex": 0,
"showHeader": true,
"showTypeIcons": false,
"sortBy": [
{
"displayName": "Saturation",
"desc": true
}
],
"footer": {
"show": false,
"reducer": []
},
"cellHeight": "sm"
},
"fieldConfig": {
"defaults": {},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "Saturation"
},
"properties": [
{
"id": "unit",
"value": "percent"
},
{
"id": "min",
"value": 0
},
{
"id": "max",
"value": 100
},
{
"id": "custom.cellOptions",
"value": {
"mode": "gradient",
"type": "gauge"
}
},
{
"id": "thresholds",
"value": {
"mode": "absolute",
"steps": [
{
"color": "red"
},
{
"color": "orange",
"value": 10
},
{
"color": "yellow",
"value": 30
},
{
"color": "green",
"value": 60
}
]
}
}
]
}
]
}
},
{
"type": "table",
"id": 22,
"targets": [
{
"expr": "topk(20, gpu:util_per_watt:avg5m)",
"instant": true,
"range": false,
"format": "table",
"refId": "A"
}
],
"title": "Utilization / Watt per GPU (5m avg)",
"description": "How efficiently each GPU spends watts. Low value = poor optimization. Grouped by Hostname/gpu/UUID — DCGM metrics cannot be attributed to workload namespaces.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 16
},
"repeatDirection": "h",
"transformations": [
{
"id": "organize",
"options": {
"excludeByName": {
"DCGM_FI_DRIVER_VERSION": true,
"Time": true,
"__name__": true,
"cluster": true,
"container": true,
"device": true,
"endpoint": true,
"gpu_driver_version": true,
"instance": true,
"job": true,
"modelName": true,
"namespace": true,
"pci_bus_id": true,
"pod": true,
"prometheus": true,
"service": true,
"tenant": true,
"tier": true,
"uid": true,
"unit": true
},
"indexByName": {
"Hostname": 0,
"UUID": 2,
"Value": 3,
"gpu": 1
},
"renameByName": {
"Hostname": "Node",
"UUID": "UUID",
"Value": "Util/Watt",
"gpu": "GPU"
}
}
}
],
"options": {
"frameIndex": 0,
"showHeader": true,
"showTypeIcons": false,
"sortBy": [
{
"displayName": "Util/Watt",
"desc": true
}
],
"footer": {
"show": false,
"reducer": []
},
"cellHeight": "sm"
},
"fieldConfig": {
"defaults": {},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "Util/Watt"
},
"properties": [
{
"id": "unit",
"value": "none"
},
{
"id": "decimals",
"value": 3
},
{
"id": "min",
"value": 0
},
{
"id": "max",
"value": 1.5
},
{
"id": "custom.cellOptions",
"value": {
"mode": "gradient",
"type": "gauge"
}
},
{
"id": "thresholds",
"value": {
"mode": "absolute",
"steps": [
{
"color": "red"
},
{
"color": "yellow",
"value": 0.5
},
{
"color": "green",
"value": 1
}
]
}
}
]
}
]
}
},
{
"type": "row",
"collapsed": false,
"title": "Throttling",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 24
},
"id": 30,
"panels": []
},
{
"type": "timeseries",
"id": 31,
"targets": [
{
"expr": "gpu:power_throttle_fraction:rate5m",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "Power throttle fraction per GPU",
"description": "Fraction of time the GPU was power-throttled. 1.0 = always throttled. Cluster-wide — rule aggregates by (Hostname, gpu, UUID) and drops namespace label.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 25
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percentunit",
"min": 0,
"max": 1,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 0.05,
"color": "yellow"
},
{
"value": 0.2,
"color": "red"
}
]
},
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 25,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 32,
"targets": [
{
"expr": "gpu:thermal_throttle_fraction:rate5m",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "Thermal throttle fraction per GPU",
"description": "Fraction of time the GPU was thermal-throttled. Cluster-wide — rule aggregates by (Hostname, gpu, UUID) and drops namespace label.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 25
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percentunit",
"min": 0,
"max": 1,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 0.05,
"color": "yellow"
},
{
"value": 0.2,
"color": "red"
}
]
},
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 25,
"showPoints": "never"
}
},
"overrides": []
}
}
],
"templating": {
"list": [
{
"type": "datasource",
"name": "ds_prometheus",
"label": "Prometheus",
"skipUrlSync": false,
"query": "prometheus",
"current": {
"selected": false,
"text": "default",
"value": "default"
},
"multi": false,
"allowCustomValue": true,
"includeAll": false,
"regex": "",
"auto": false,
"auto_min": "10s",
"auto_count": 30
}
]
},
"annotations": {}
}

File diff suppressed because it is too large Load diff

View file

@ -1,957 +0,0 @@
{
"uid": "gpu-performance",
"title": "GPU Performance",
"tags": [
"gpu",
"dcgm"
],
"timezone": "browser",
"editable": true,
"graphTooltip": 1,
"time": {
"from": "now-1h",
"to": "now"
},
"fiscalYearStartMonth": 0,
"schemaVersion": 42,
"panels": [
{
"type": "row",
"collapsed": false,
"title": "Overview",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 0
},
"id": 1,
"panels": []
},
{
"type": "stat",
"id": 2,
"targets": [
{
"expr": "cluster:gpu_count:total",
"refId": "A"
}
],
"title": "Total GPUs",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 0,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "none",
"colorMode": "value",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "blue"
}
]
}
},
"overrides": []
}
},
{
"type": "stat",
"id": 3,
"targets": [
{
"expr": "cluster:gpu_count:allocated",
"refId": "A"
}
],
"title": "Allocated",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 6,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "none",
"colorMode": "value",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 1,
"color": "yellow"
}
]
}
},
"overrides": []
}
},
{
"type": "stat",
"id": 4,
"targets": [
{
"expr": "cluster:gpu_util:avg",
"refId": "A"
}
],
"title": "Average utilization",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 12,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "area",
"colorMode": "value",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "blue"
},
{
"value": 30,
"color": "green"
},
{
"value": 80,
"color": "orange"
}
]
}
},
"overrides": []
}
},
{
"type": "stat",
"id": 5,
"targets": [
{
"expr": "cluster:gpu_power_watts:sum",
"refId": "A"
}
],
"title": "Power draw",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 18,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "area",
"colorMode": "value",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "watt",
"decimals": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
}
]
}
},
"overrides": []
}
},
{
"type": "row",
"collapsed": false,
"title": "Utilization",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 5
},
"id": 10,
"panels": []
},
{
"type": "timeseries",
"id": 11,
"targets": [
{
"expr": "DCGM_FI_DEV_GPU_UTIL{Hostname=~\"$Hostname\"}",
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
"refId": "A"
}
],
"title": "GPU Utilization (NVML)",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 6
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 12,
"targets": [
{
"expr": "DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{Hostname=~\"$Hostname\"} * 100",
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
"refId": "A"
}
],
"title": "Tensor Pipe Active (realistic load for LLM/AI)",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 6
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 13,
"targets": [
{
"expr": "DCGM_FI_PROF_GR_ENGINE_ACTIVE{Hostname=~\"$Hostname\"} * 100",
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
"refId": "A"
}
],
"title": "Graphics Engine Active",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 14
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 14,
"targets": [
{
"expr": "DCGM_FI_DEV_MEM_COPY_UTIL{Hostname=~\"$Hostname\"}",
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
"refId": "A"
}
],
"title": "Memory Copy Utilization",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 14
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false
}
},
"overrides": []
}
},
{
"type": "row",
"collapsed": false,
"title": "Memory",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 22
},
"id": 20,
"panels": []
},
{
"type": "timeseries",
"id": 21,
"targets": [
{
"expr": "DCGM_FI_DEV_FB_USED{Hostname=~\"$Hostname\"} * 1048576",
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
"refId": "A"
}
],
"title": "VRAM Used",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 23
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "bytes",
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 25,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 22,
"targets": [
{
"expr": "DCGM_FI_DEV_FB_FREE{Hostname=~\"$Hostname\"} * 1048576",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "VRAM Free",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 23
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"min"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "bytes",
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "row",
"collapsed": false,
"title": "Power \u0026 Temperature",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 31
},
"id": 30,
"panels": []
},
{
"type": "timeseries",
"id": 31,
"targets": [
{
"expr": "DCGM_FI_DEV_POWER_USAGE{Hostname=~\"$Hostname\"}",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "Power Usage per GPU",
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 32
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "watt",
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 32,
"targets": [
{
"expr": "DCGM_FI_DEV_GPU_TEMP{Hostname=~\"$Hostname\"}",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "GPU Temperature",
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 32
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "table",
"placement": "bottom",
"showLegend": false,
"calcs": [
"mean",
"max"
]
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "celsius",
"min": 20,
"max": 100,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 75,
"color": "yellow"
},
{
"value": 85,
"color": "red"
}
]
},
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "row",
"collapsed": false,
"title": "Health",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 40
},
"id": 40,
"panels": []
},
{
"type": "stat",
"id": 41,
"targets": [
{
"expr": "max by (Hostname, gpu) (DCGM_FI_DEV_XID_ERRORS{Hostname=~\"$Hostname\"})",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "XID errors (latest)",
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 5,
"w": 8,
"x": 0,
"y": 41
},
"repeatDirection": "h",
"options": {
"graphMode": "none",
"colorMode": "background",
"justifyMode": "auto",
"textMode": "value_and_name",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 1,
"color": "red"
}
]
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 42,
"targets": [
{
"expr": "rate(DCGM_FI_DEV_POWER_VIOLATION{Hostname=~\"$Hostname\"}[5m])",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "Power Violation (µs/s throttled due to power)",
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 5,
"w": 8,
"x": 8,
"y": 41
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": false,
"calcs": []
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "µs",
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 43,
"targets": [
{
"expr": "rate(DCGM_FI_DEV_THERMAL_VIOLATION{Hostname=~\"$Hostname\"}[5m])",
"legendFormat": "{{Hostname}}/{{gpu}}",
"refId": "A"
}
],
"title": "Thermal Violation (µs/s throttled due to thermals)",
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 5,
"w": 8,
"x": 16,
"y": 41
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": false,
"calcs": []
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "µs",
"custom": {
"drawStyle": "line",
"lineInterpolation": "linear",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": []
}
}
],
"templating": {
"list": [
{
"type": "datasource",
"name": "ds_prometheus",
"label": "Prometheus",
"skipUrlSync": false,
"query": "prometheus",
"current": {
"selected": false,
"text": "default",
"value": "default"
},
"multi": false,
"allowCustomValue": true,
"includeAll": false,
"regex": "",
"auto": false,
"auto_min": "10s",
"auto_count": 30
},
{
"type": "query",
"name": "Hostname",
"label": "Host",
"skipUrlSync": false,
"query": "label_values(DCGM_FI_DEV_GPU_UTIL, Hostname)",
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"current": {
"selected": true,
"text": "All",
"value": "$__all"
},
"multi": true,
"allowCustomValue": true,
"refresh": 2,
"sort": 1,
"includeAll": true,
"auto": false,
"auto_min": "10s",
"auto_count": 30
}
]
},
"annotations": {}
}

View file

@ -1,637 +0,0 @@
{
"uid": "gpu-quotas",
"title": "GPU Quotas \u0026 Allocation",
"tags": [
"gpu",
"quotas"
],
"timezone": "browser",
"editable": true,
"graphTooltip": 1,
"time": {
"from": "now-6h",
"to": "now"
},
"fiscalYearStartMonth": 0,
"schemaVersion": 42,
"panels": [
{
"type": "row",
"collapsed": false,
"title": "Allocation overview",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 0
},
"id": 1,
"panels": []
},
{
"type": "stat",
"id": 2,
"targets": [
{
"expr": "sum(kube_node_status_allocatable{resource=\"nvidia_com_gpu\"})",
"refId": "A"
}
],
"title": "GPU allocatable",
"description": "Total GPU capacity the cluster can schedule to pods.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 0,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "none",
"colorMode": "value",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "blue"
}
]
}
},
"overrides": []
}
},
{
"type": "stat",
"id": 3,
"targets": [
{
"expr": "cluster:gpu_count:allocated",
"refId": "A"
}
],
"title": "GPU requested",
"description": "Sum of GPU requests across all pods cluster-wide, including system namespaces.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 6,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "none",
"colorMode": "value",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 1,
"color": "yellow"
}
]
}
},
"overrides": []
}
},
{
"type": "gauge",
"id": 4,
"targets": [
{
"expr": "cluster:gpu_count:allocated / sum(kube_node_status_allocatable{resource=\"nvidia_com_gpu\"}) * 100",
"refId": "A"
}
],
"title": "Allocation ratio",
"description": "Percentage of allocatable GPUs currently requested by pods.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 12,
"y": 1
},
"repeatDirection": "h",
"options": {
"showThresholdLabels": false,
"showThresholdMarkers": true,
"sizing": "auto",
"minVizWidth": 75,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"minVizHeight": 75,
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"min": 0,
"max": 100,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "blue"
},
{
"value": 40,
"color": "green"
},
{
"value": 80,
"color": "yellow"
},
{
"value": 95,
"color": "red"
}
]
}
},
"overrides": []
}
},
{
"type": "stat",
"id": 5,
"targets": [
{
"expr": "count((sum by (namespace, pod) (kube_pod_container_resource_requests{resource=\"nvidia_com_gpu\"}) \u003e 0) * on(namespace, pod) group_left() (kube_pod_status_phase{phase=\"Pending\"} == 1)) or vector(0)",
"refId": "A"
}
],
"title": "Pending pods (GPU)",
"description": "Pods requesting GPUs that are stuck in Pending state — indicates capacity shortage.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 4,
"w": 6,
"x": 18,
"y": 1
},
"repeatDirection": "h",
"options": {
"graphMode": "none",
"colorMode": "background",
"justifyMode": "auto",
"textMode": "value",
"wideLayout": true,
"showPercentChange": false,
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"percentChangeColorMode": "standard",
"orientation": ""
},
"fieldConfig": {
"defaults": {
"unit": "short",
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
},
{
"value": 1,
"color": "red"
}
]
}
},
"overrides": []
}
},
{
"type": "row",
"collapsed": false,
"title": "Per namespace",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 5
},
"id": 10,
"panels": []
},
{
"type": "bargauge",
"id": 11,
"targets": [
{
"expr": "sum by (namespace) (namespace:gpu_count:allocated{namespace=~\"$namespace\"})",
"legendFormat": "{{namespace}}",
"refId": "A"
}
],
"title": "GPU requested per namespace",
"description": "GPU allocation breakdown by namespace — spot top consumers at a glance.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 6
},
"repeatDirection": "h",
"options": {
"displayMode": "gradient",
"valueMode": "color",
"namePlacement": "auto",
"showUnfilled": true,
"sizing": "auto",
"minVizWidth": 8,
"minVizHeight": 16,
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": false,
"calcs": []
},
"reduceOptions": {
"calcs": [
"lastNotNull"
]
},
"maxVizHeight": 300,
"orientation": "horizontal"
},
"fieldConfig": {
"defaults": {
"unit": "short",
"min": 0,
"max": 8,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "green"
}
]
}
},
"overrides": []
}
},
{
"type": "timeseries",
"id": 12,
"targets": [
{
"expr": "sum(kube_node_status_allocatable{resource=\"nvidia_com_gpu\"})",
"legendFormat": "Allocatable (total)",
"refId": "A"
},
{
"expr": "sum(namespace:gpu_count:allocated{namespace=~\"$namespace\"})",
"legendFormat": "Requested",
"refId": "B"
}
],
"title": "GPU allocated over time",
"description": "Requested vs allocatable GPUs over time — shows allocation pressure trends.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 6
},
"repeatDirection": "h",
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom",
"showLegend": false,
"calcs": []
},
"tooltip": {
"mode": "multi",
"sort": ""
}
},
"fieldConfig": {
"defaults": {
"unit": "short",
"custom": {
"drawStyle": "line",
"lineInterpolation": "stepAfter",
"fillOpacity": 10,
"showPoints": "never"
}
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "Allocatable (total)"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "blue",
"mode": "fixed"
}
}
]
}
]
}
},
{
"type": "row",
"collapsed": false,
"title": "Pods with GPU",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 14
},
"id": 20,
"panels": []
},
{
"type": "table",
"id": 21,
"targets": [
{
"expr": "kube_pod_container_resource_requests{resource=\"nvidia_com_gpu\", namespace=~\"$namespace\"} * on(namespace, pod) group_left(phase) (kube_pod_status_phase{phase=~\"Running|Pending\"} == 1)",
"instant": true,
"range": false,
"format": "table",
"refId": "requested"
},
{
"expr": "kube_pod_container_resource_limits{resource=\"nvidia_com_gpu\", namespace=~\"$namespace\"} * on(namespace, pod) group_left() (kube_pod_status_phase{phase=~\"Running|Pending\"} == 1)",
"instant": true,
"range": false,
"format": "table",
"refId": "limits"
}
],
"title": "Pods requesting GPU",
"description": "Per-pod GPU requests and limits with scheduling status — Running or Pending.",
"transparent": false,
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 15
},
"repeatDirection": "h",
"transformations": [
{
"id": "joinByField",
"options": {
"byField": "pod",
"mode": "outer"
}
},
{
"id": "organize",
"options": {
"excludeByName": {
"Time": true,
"Time 1": true,
"Time 2": true,
"__name__": true,
"__name__ 1": true,
"__name__ 2": true,
"cluster": true,
"cluster 1": true,
"cluster 2": true,
"container 2": true,
"endpoint": true,
"endpoint 1": true,
"endpoint 2": true,
"instance": true,
"instance 1": true,
"instance 2": true,
"job": true,
"job 1": true,
"job 2": true,
"namespace 2": true,
"node 2": true,
"prometheus": true,
"prometheus 1": true,
"prometheus 2": true,
"resource": true,
"resource 1": true,
"resource 2": true,
"service": true,
"service 1": true,
"service 2": true,
"tenant": true,
"tenant 1": true,
"tenant 2": true,
"tier": true,
"tier 1": true,
"tier 2": true,
"uid": true,
"uid 1": true,
"uid 2": true,
"unit": true,
"unit 1": true,
"unit 2": true
},
"indexByName": {
"Value #limits": 5,
"Value #requested": 4,
"container 1": 2,
"namespace 1": 0,
"node 1": 3,
"phase": 6,
"pod": 1
},
"renameByName": {
"Value #limits": "Limit",
"Value #requested": "Req",
"container 1": "Container",
"namespace 1": "Namespace",
"node 1": "Node",
"phase": "Status"
}
}
}
],
"options": {
"frameIndex": 0,
"showHeader": true,
"showTypeIcons": false,
"footer": {
"show": false,
"reducer": []
},
"cellHeight": "sm"
},
"fieldConfig": {
"defaults": {},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "Status"
},
"properties": [
{
"id": "custom.cellOptions",
"value": {
"mode": "basic",
"type": "color-background"
}
},
{
"id": "mappings",
"value": [
{
"options": {
"Failed": {
"color": "red",
"index": 2
},
"Pending": {
"color": "orange",
"index": 1
},
"Running": {
"color": "green",
"index": 0
}
},
"type": "value"
}
]
}
]
}
]
}
}
],
"templating": {
"list": [
{
"type": "datasource",
"name": "ds_prometheus",
"label": "Prometheus",
"skipUrlSync": false,
"query": "prometheus",
"current": {
"selected": false,
"text": "default",
"value": "default"
},
"multi": false,
"allowCustomValue": true,
"includeAll": false,
"regex": "",
"auto": false,
"auto_min": "10s",
"auto_count": 30
},
{
"type": "query",
"name": "namespace",
"label": "Namespace",
"skipUrlSync": false,
"query": "label_values(kube_pod_container_resource_requests{resource=\"nvidia_com_gpu\"}, namespace)",
"datasource": {
"type": "prometheus",
"uid": "$ds_prometheus"
},
"current": {
"selected": true,
"text": "All",
"value": "$__all"
},
"multi": true,
"allowCustomValue": true,
"refresh": 2,
"sort": 1,
"includeAll": true,
"auto": false,
"auto_min": "10s",
"auto_count": 30
}
]
},
"annotations": {}
}

File diff suppressed because it is too large Load diff

View file

@ -8,30 +8,6 @@ Project-side conventions for commits, branches, and pull requests in Cozystack.
- [ ] Commit is signed off with `--signoff`
- [ ] Branch is rebased on `upstream/main` (no extra commits)
- [ ] PR body includes description and release note
- [ ] Ran `make generate` in every package whose `values.yaml`, `values.schema.json`, `Chart.yaml`, or `README.md` was touched, and committed the regenerated files
## Regenerate Artifacts Before Committing
Several files in each package are produced by `make generate` from `values.yaml` + `values.schema.json` and must stay in sync with the hand-edited sources:
- `packages/(apps|extra)/<name>/README.md` — regenerated by `cozyvalues-gen` (parameter table, formatting).
- `packages/(apps|extra)/<name>/values.schema.json``cozyvalues-gen` rewrites ordering and derived fields.
- `packages/system/<name>-rd/cozyrds/<name>.yaml` — produced by `hack/update-crd.sh`, which `make generate` invokes.
**Before committing edits to any of those sources**, run `make generate` inside the package and stage the full diff:
```bash
make -C packages/<apps-or-extra>/<name> generate
git add packages/<apps-or-extra>/<name>/ packages/system/<name>-rd/
```
The repo's pre-commit CI job runs `make generate` in every package and then `git diff --exit-code`. Any unstaged generator output fails the job with exit code 123 and blocks the PR. Also rerun `make generate` after a `git commit --amend` if the amended change touched any of the sources above.
To locate packages a WIP branch likely needs to be regenerated:
```bash
git diff --name-only | xargs -n1 dirname | sort -u | grep ^packages/
```
## Commit Format
@ -43,11 +19,10 @@ git commit --signoff -m "type(scope): brief description"
**Types:** `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`
**Scopes** (examples — not an exhaustive list; pick the most specific scope that describes the change, and introduce a new one if a genuinely new area needs its own):
- System, e.g.: `dashboard`, `platform`, `operator`, `cilium`, `kube-ovn`, `linstor`, `fluxcd`, `cluster-api`
- Apps, e.g.: `postgres`, `mariadb`, `redis`, `kafka`, `clickhouse`, `virtual-machine`, `kubernetes`
- Other, e.g.: `api`, `hack`, `tests`, `ci`, `docs`, `agents`, `maintenance`
**Scopes:**
- System: `dashboard`, `platform`, `cilium`, `kube-ovn`, `linstor`, `fluxcd`, `cluster-api`
- Apps: `postgres`, `mariadb`, `redis`, `kafka`, `clickhouse`, `virtual-machine`, `kubernetes`
- Other: `api`, `hack`, `tests`, `ci`, `docs`, `maintenance`
Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer.
@ -58,49 +33,6 @@ git commit --signoff -m "fix(postgres): update operator to version 1.2.3"
git commit --signoff -m "docs(contributing): add installation guide"
```
## PR Title Auto-Labeling
`.github/workflows/pr-labeler.yaml` parses the PR title on `opened`, `edited`, `reopened`, and `synchronize` events and applies labels additively (never removes). The title is expected to follow Conventional Commits — same format as commit messages above.
**Type → `kind/*`:**
| type | label |
| --------- | ------------------ |
| feat | kind/feature |
| fix | kind/bug |
| docs | kind/documentation |
| chore | kind/cleanup |
| refactor | kind/cleanup |
| style, perf, test, build, ci, revert | (no kind label) |
**Scope → `area/*`** (full mapping in `.github/workflows/pr-labeler.yaml`):
| scope (examples) | label |
| --- | --- |
| agents, ai | area/ai |
| api, cozystack-api | area/api |
| build | area/build |
| ci | area/ci |
| dashboard | area/dashboard |
| postgres, mariadb, redis, etcd, kafka, clickhouse, postgres-operator, mariadb-operator | area/database |
| extra | area/extra |
| kubernetes | area/kubernetes |
| monitoring, vlogs, vmstack, grafana, workloadmonitor | area/monitoring |
| ingress, gateway, vpn, metallb, cilium, kube-ovn, cozy-proxy, ... | area/networking |
| platform, bundle, flux, fluxcd, cluster-api, talos, installer, cozyctl, cozystack-engine, cozy-lib | area/platform |
| backport, release | area/release |
| seaweedfs, bucket, linstor, velero, harbor, backups | area/storage |
| tests, e2e | area/testing |
| kubevirt, cdi, vmi, vm-import, virtual-machine, hami, gpu-operator | area/virtualization |
**Special handling:**
- `[Backport release-1.x]` prefix is stripped before parsing; `area/release` and `backport` labels are added.
- Composite scope (`feat(platform, system, apps): ...`) — each comma-separated part is mapped independently.
- `!` after type or `BREAKING CHANGE:` footer in the body → `kind/breaking-change`.
- Unmapped scope or non-conventional title → `area/uncategorized` (signals the PR needs manual area selection).
- Bracket-style fallback (`[scope] description`) maps `scope``area/*` but cannot infer `kind/*`.
### AI Agent Attribution
When an AI agent authors or materially assists with a commit, add an `Assisted-By:` trailer naming the model:

View file

@ -83,14 +83,6 @@ packages/<category>/<package-name>/
- Reference PR numbers when available
- Keep commits atomic and focused
### PackageSource CRD upgrade policy
Each component in a `PackageSource` may set `install.upgradeCRDs` to control how CRDs from the chart's `crds/` directory are handled on `HelmRelease` upgrades. Allowed values: `Skip` (default — helm-controller does not touch CRDs on upgrade), `Create` (create new CRDs only), `CreateReplace` (create new and overwrite existing).
Set `upgradeCRDs: CreateReplace` for operators whose upstream regularly adds new CRDs between versions (etcd-operator, cnpg, kubevirt, kamaji). Without it, new CRDs from a chart bump do not land on existing clusters — only fresh installs get them.
Do **not** set `CreateReplace` blindly: it overwrites every CRD in `crds/` and can cause silent data loss if upstream drops a field from a CRD that has live objects. Only enable it for operators whose schema evolution is additive-only. When in doubt, leave it unset and apply new CRDs manually.
### Documentation
Documentation is organized as follows:

View file

@ -1,37 +0,0 @@
<!--
https://github.com/cozystack/cozystack/releases/tag/v1.3.1
-->
# v1.3.1 (2026-04-28)
Patch release covering a TenantNamespace IDOR fix in the API, a destructive `post-upgrade` hook removed from the etcd chart, kamaji controller stability, a `linstor-csi` bump that fixes live migration on Protocol-A/B DRBD resources, the missing `linstor-gui` build wiring, and a velero RBAC fix that unblocked installs on bundles without Velero.
## Security
* **fix(api): prevent IDOR in TenantNamespace Get and Watch handlers**: Two IDOR (Insecure Direct Object Reference) vulnerabilities allowed authenticated users to read TenantNamespace metadata they had no RoleBinding for. The `Get` and `Watch` handlers now go through a new `hasAccessToNamespace()` helper that lists RoleBindings scoped only to the target namespace (orders of magnitude faster than the previous all-cluster scan), returns `NotFound` instead of leaking existence on unauthorized access, and applies the same check on the `Watch` filter path. Includes regression tests for the unauthorized paths. ([**@IvanHunters**](https://github.com/IvanHunters) in #2471, backport #2524)
## Features
* **feat(linstor): bump linstor-csi to v1.10.6 with Protocol-C dual-attach fix**: Live migration of KubeVirt VMs on Protocol-A/B (async) DRBD volumes no longer fails with `Protocol C required`. `linstor-csi` v1.10.6 now installs a `Protocol=C` override on the resource-definition during dual-attach and reverts it on detach, so `replicated-async` StorageClasses and other Protocol-A/B resource groups support live migration without manual `drbdadm adjust` intervention. ([**@kvaps**](https://github.com/kvaps) in #2496, backport #2505)
## Fixes
* **fix(backups): move velero-configmap Role to velero chart**: The `backupstrategy-controller` (a default package) declared a Role/RoleBinding scoped to the `cozy-velero` namespace for managing `ResourceModifier` ConfigMaps. On bundles where Velero was not enabled, that namespace did not exist and the HelmRelease failed with `namespaces "cozy-velero" not found`, blocking installation. The Role/RoleBinding now lives in the velero chart, so it is created only when velero is actually deployed. ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2459, backport #2467)
* **fix(etcd): remove destructive post-upgrade cert-regeneration hook**: The etcd chart ran a `post-upgrade` Helm hook on every upgrade that deleted etcd TLS Secrets (`etcd-ca-tls`, `etcd-peer-ca-tls`, `etcd-client-tls`, `etcd-peer-tls`, `etcd-server-tls`) and then deleted etcd pods, forcing cert-manager to re-issue the entire etcd CA chain. On clusters with Kamaji-managed tenant control planes this put every tenant `kube-apiserver` into CrashLoopBackOff until each DataStore was manually re-reconciled. The hook was a one-shot `2.6.0 → 2.6.1` migration that became a permanent footgun once chart versioning moved to `0.0.0+<git-hash>` (always `< 2.6.1` per semver) and after the underlying `rotationPolicy: Always` issue was fixed in `47d81f70`. The hook is now removed entirely. ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2462, backport #2511)
* **fix(kamaji): increase memory limits and add startup probe**: The kamaji controller frequently entered CrashLoopBackOff due to OOMKills (exit 137) within ~2025 seconds of startup, with the readiness probe failing while the controller was still finishing initialization. Memory limit raised from 500Mi to 512Mi, request from 100Mi to 256Mi, and a 60-second startup probe (12 attempts × 5s periods) is added so the controller has room to boot before liveness/readiness probes engage. ([**@IvanHunters**](https://github.com/IvanHunters) in #2421, backport #2491)
## Build
* **build(linstor): include linstor-gui in root image build target**: The `linstor-gui` package (added in #2382) was never wired into the root `Makefile`'s `build:` target, so CI never built or published the image. `ghcr.io/cozystack/cozystack/linstor-gui` returned `NAME_UNKNOWN` and `values.yaml` stayed pinned to `tag: 2.3.0` without a digest. The missing build line is added so the next CI run publishes the image and the per-package `Makefile` digest-pins `values.yaml` automatically. ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2498, backport #2518)
## Contributors
Thanks to everyone who contributed to this patch release:
* [**@IvanHunters**](https://github.com/IvanHunters)
* [**@kvaps**](https://github.com/kvaps)
* [**@myasnikovdaniil**](https://github.com/myasnikovdaniil)
**Full Changelog**: https://github.com/cozystack/cozystack/compare/v1.3.0...v1.3.1

View file

@ -1,145 +0,0 @@
#!/usr/bin/env bats
# -----------------------------------------------------------------------------
# Chart-wide invariant for packages/apps/kubernetes:
#
# Every Deployment in this chart that mounts <release>-admin-kubeconfig as a
# Secret volume MUST:
# - declare that volume optional: true (so kubelet does not FailedMount
# while Kamaji is still provisioning the Secret), AND
# - include the wait-for-kubeconfig init container (so the pod becomes
# Ready only after Kamaji publishes the Secret).
#
# The per-template unittests in packages/apps/kubernetes/tests/ lock in
# today's three Deployments (cluster-autoscaler, kccm, csi controller) by
# name. This invariant is stricter: any future Deployment added to this
# chart that mounts the same Secret but forgets the guard will fail here.
#
# Requires: helm, yq (mikefarah v4+), jq. All three are available on the
# project's CI runners and on the maintainer workstation.
# -----------------------------------------------------------------------------
@test "every Deployment mounting admin-kubeconfig has optional:true and wait-for-kubeconfig init" {
values_file="packages/apps/kubernetes/tests/values-ci.yaml"
[ -f "$values_file" ]
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
helm template invariant packages/apps/kubernetes \
--namespace tenant-root \
--values "$values_file" \
2>/dev/null > "$tmp/rendered.yaml"
# yq streams one JSON object per input document. jq -s slurps the stream
# into an array so we can treat all Deployments as a single collection.
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
| jq -s --raw-output '
map(select(.kind == "Deployment")) |
map({
name: .metadata.name,
volumes: (.spec.template.spec.volumes // []),
initNames: ((.spec.template.spec.initContainers // []) | map(.name)),
}) |
map(
.name as $n |
.initNames as $ins |
(.volumes[] | select(.secret.secretName | test("-admin-kubeconfig$")?))
| {
name: $n,
optional: (.secret.optional == true),
hasInit: ($ins | index("wait-for-kubeconfig") != null),
}
)
' > "$tmp/summary.json"
# At least one Deployment must match; if a refactor removes every
# admin-kubeconfig volume from this chart, the test must be updated
# deliberately rather than silently passing.
matched=$(jq 'length' "$tmp/summary.json")
[ "$matched" -ge 1 ]
offenders=$(jq --raw-output '.[] | select(.optional != true or .hasInit != true) | .name' "$tmp/summary.json")
if [ -n "$offenders" ]; then
echo "Deployments mounting *-admin-kubeconfig without optional:true + wait-for-kubeconfig init:" >&2
echo "$offenders" >&2
echo "Full summary:" >&2
cat "$tmp/summary.json" >&2
exit 1
fi
echo "Invariant holds for $matched Deployment(s)"
}
@test "chart emits zero admin-kubeconfig Deployments when tenant has no etcd DataStore" {
# Without a DataStore (parent Tenant has not populated _namespace.etcd yet)
# the control-plane-side Deployments must NOT render at all. If they did,
# the wait-for-kubeconfig init would CrashLoopBackOff indefinitely - there
# would be no KamajiControlPlane to provision the Secret - consuming the
# HelmRelease wait budget and triggering exactly the remediation cycle the
# rest of this chart tries to avoid. This test renders the whole chart
# with etcd empty and asserts no Deployment references the admin-kubeconfig
# Secret.
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
helm template invariant packages/apps/kubernetes \
--namespace tenant-root \
--values packages/apps/kubernetes/tests/values-ci-no-etcd.yaml \
2>/dev/null > "$tmp/rendered.yaml"
matched=$(
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
| jq -s '
map(select(.kind == "Deployment")) |
map(select(
(.spec.template.spec.volumes // [])
| any(.secret.secretName | test("-admin-kubeconfig$")?)
)) |
length
'
)
if [ "$matched" -ne 0 ]; then
echo "Expected zero Deployments mounting *-admin-kubeconfig when etcd is empty, got $matched:" >&2
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
| jq -s 'map(select(.kind == "Deployment") | .metadata.name)' >&2
exit 1
fi
echo "No admin-kubeconfig Deployments rendered for empty etcd (as expected)"
}
@test "chart emits zero admin-kubeconfig HelmReleases when tenant has no etcd DataStore" {
# Same principle as the Deployment variant above, extended to every child
# HelmRelease (cilium, coredns, csi, cert-manager, ...). They reference
# *-admin-kubeconfig via kubeConfig.secretRef and would otherwise sit in
# NotReady forever on an etcd-less tenant, polluting the HelmRelease list
# the operator sees and contradicting the "awaiting-etcd beacon only"
# contract of the soft-skip path.
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
helm template invariant packages/apps/kubernetes \
--namespace tenant-root \
--values packages/apps/kubernetes/tests/values-ci-no-etcd.yaml \
2>/dev/null > "$tmp/rendered.yaml"
matched=$(
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
| jq -s '
map(select(.kind == "HelmRelease")) |
map(select(.spec.kubeConfig.secretRef.name | test("-admin-kubeconfig$")?)) |
length
'
)
if [ "$matched" -ne 0 ]; then
echo "Expected zero HelmReleases referencing *-admin-kubeconfig when etcd is empty, got $matched:" >&2
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
| jq -s 'map(select(.kind == "HelmRelease") | .metadata.name)' >&2
exit 1
fi
echo "No admin-kubeconfig HelmReleases rendered for empty etcd (as expected)"
}

View file

@ -1,206 +0,0 @@
#!/usr/bin/env bats
# -----------------------------------------------------------------------------
# Cross-validation between GPU recording rules, the dashboards that consume
# them, and the DCGM Exporter metric set the cluster actually scrapes. Catches:
#
# 1. dangling references — a dashboard query mentions a recording rule name
# that doesn't exist in gpu-recording.rules.yaml. This is the bug the
# pre-merge review caught: gpu-efficiency.json shipped panels keyed on
# pod:tensor_saturation:avg5m without the rule being defined, so the
# panel showed "No data" everywhere.
#
# 2. typos in rule names — same bug class, manifested as a single-character
# difference between rule and reference.
#
# 3. undeclared DCGM metrics — a dashboard query or recording rule mentions
# a DCGM_FI_* metric that is neither in the upstream default CSV nor in
# the project's custom CSV (dcgm-custom-metrics.yaml), meaning DCGM
# Exporter would never emit it and the panel silently shows "No data".
# Example regression: gpu-fleet.json shipped a TDP panel referencing
# DCGM_FI_DEV_POWER_MGMT_LIMIT before the custom CSV declared it.
#
# Scope: only dashboards listed in packages/system/monitoring/dashboards-infra.list
# under the "gpu/" prefix (i.e. shipped to production), not every JSON file in
# dashboards/gpu/. Untracked drafts stay out of scope on purpose — adding one
# to dashboards-infra.list is what brings it under the test.
#
# Reverse direction (rule defined but never consumed) is intentionally NOT
# enforced: some rules exist for ad-hoc PromQL or upcoming dashboards. Treat
# unused rules as an editorial concern, not a regression.
#
# Title syntax constraints from cozytest.sh's awk parser:
# - Titles delimited by ASCII double quotes; embedded quotes truncate.
# - Only [A-Za-z0-9] from the title survives into the function name; titles
# differing only in punctuation collapse to the same function.
#
# Run with: hack/cozytest.sh hack/check-gpu-recording-rules.bats
# -----------------------------------------------------------------------------
REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
RULES_FILE="$REPO_ROOT/packages/system/monitoring-agents/alerts/gpu-recording.rules.yaml"
DASHBOARDS_LIST="$REPO_ROOT/packages/system/monitoring/dashboards-infra.list"
DASHBOARDS_DIR="$REPO_ROOT/dashboards"
DCGM_DEFAULT_CSV="$REPO_ROOT/hack/dcgm-default-counters.csv"
DCGM_CUSTOM_CSV="$REPO_ROOT/packages/system/gpu-operator/examples/dcgm-custom-metrics.yaml"
# Extract the set of "- record: NAME" entries from the rules YAML.
# Outputs one rule name per line, sorted and deduplicated.
extract_rules() {
awk '/^[[:space:]]*-[[:space:]]*record:[[:space:]]/ {
sub(/^[[:space:]]*-[[:space:]]*record:[[:space:]]*/, "")
sub(/[[:space:]]*$/, "")
print
}' "$RULES_FILE" | sort -u
}
# Extract the set of recording-rule references from a dashboard JSON.
# A recording-rule reference is matched by the pattern
# <segment>:<segment>(:<segment>)+
# where each <segment> is [a-z0-9_]. Raw DCGM metrics (DCGM_FI_*),
# kube-state-metrics (kube_*) and similar uppercase / single-word metric
# names do not match because the leading segment must be lowercase and the
# whole expression must contain at least two ':' characters.
extract_refs() {
json_file=$1
# Prometheus convention allows 2-segment rule names (level:metric); this
# regex is tuned to the 3+ segment convention used in this repo
# (level:metric:op — e.g. cluster:gpu_count:total). Update if future
# rules use 2 segments, otherwise they will be silently skipped.
grep -hoE '[a-z][a-z0-9_]*:[a-z0-9_]+:[a-z0-9_]+' "$json_file" | sort -u
}
# Resolve "gpu/foo" -> "$DASHBOARDS_DIR/gpu/foo.json"
list_tracked_gpu_dashboards() {
awk '/^gpu\// { print $0 ".json" }' "$DASHBOARDS_LIST"
}
# Extract the set of DCGM_FI_* metric names declared in a CSV file. Handles
# both the upstream-style default CSV (unindented) and the ConfigMap-style
# custom CSV (YAML-indented). A declaration line starts — after any leading
# whitespace — with "DCGM_FI_<NAME>," ; comment lines begin with "#" and are
# skipped. Uses POSIX awk's match()+RSTART/RLENGTH so no GNU extensions
# are required.
extract_csv_metrics() {
file=$1
awk '
{
line = $0
sub(/^[[:space:]]+/, "", line)
if (line ~ /^#/) next
if (match(line, /^DCGM_FI_[A-Z0-9_]+/)) {
print substr(line, RSTART, RLENGTH)
}
}
' "$file" | sort -u
}
# Extract the set of DCGM_FI_* metric references from a text file (dashboard
# JSON or rules YAML). A DCGM metric name has at least two underscore-delimited
# segments after the "DCGM_FI_" prefix (e.g. DCGM_FI_DEV_GPU_UTIL, DCGM_FI_PROF_
# PIPE_TENSOR_ACTIVE, DCGM_FI_DRIVER_VERSION). Requiring two segments keeps
# the matcher from latching onto glob stubs like "DCGM_FI_DEV_*_VIOLATION" that
# appear in comments.
extract_dcgm_refs() {
file=$1
grep -hoE 'DCGM_FI_[A-Z0-9]+(_[A-Z0-9]+)+' "$file" | sort -u
}
@test "every recording rule reference in tracked GPU dashboards has a matching record" {
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
extract_rules > "$TMP/rules.txt"
[ -s "$TMP/rules.txt" ] || { echo "no recording rules extracted from $RULES_FILE" >&2; exit 1; }
list_tracked_gpu_dashboards > "$TMP/dashboards.txt"
[ -s "$TMP/dashboards.txt" ] || { echo "no gpu/* dashboards listed in $DASHBOARDS_LIST" >&2; exit 1; }
failed=0
while IFS= read -r dashboard_rel; do
dashboard="$DASHBOARDS_DIR/$dashboard_rel"
if [ ! -f "$dashboard" ]; then
echo "ERROR: dashboard listed but file missing: $dashboard" >&2
failed=1
continue
fi
extract_refs "$dashboard" > "$TMP/refs.txt"
# comm -23: lines unique to refs.txt (referenced but not defined)
# Both inputs must be sorted; extract_* helpers already sort.
comm -23 "$TMP/refs.txt" "$TMP/rules.txt" > "$TMP/missing.txt"
if [ -s "$TMP/missing.txt" ]; then
echo "ERROR: $dashboard_rel references undefined recording rules:" >&2
sed 's/^/ - /' "$TMP/missing.txt" >&2
failed=1
fi
done < "$TMP/dashboards.txt"
[ "$failed" -eq 0 ]
}
@test "every DCGM metric referenced in tracked dashboards and rules is declared" {
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
[ -f "$DCGM_DEFAULT_CSV" ] || { echo "missing $DCGM_DEFAULT_CSV" >&2; exit 1; }
[ -f "$DCGM_CUSTOM_CSV" ] || { echo "missing $DCGM_CUSTOM_CSV" >&2; exit 1; }
{
extract_csv_metrics "$DCGM_DEFAULT_CSV"
extract_csv_metrics "$DCGM_CUSTOM_CSV"
} | sort -u > "$TMP/declared.txt"
[ -s "$TMP/declared.txt" ] || { echo "no DCGM metrics extracted from CSVs" >&2; exit 1; }
list_tracked_gpu_dashboards > "$TMP/dashboards.txt"
[ -s "$TMP/dashboards.txt" ] || { echo "no gpu/* dashboards listed in $DASHBOARDS_LIST" >&2; exit 1; }
failed=0
# Dashboard coverage — every dashboard's DCGM references must resolve.
while IFS= read -r dashboard_rel; do
dashboard="$DASHBOARDS_DIR/$dashboard_rel"
[ -f "$dashboard" ] || continue # handled by the existence test
extract_dcgm_refs "$dashboard" > "$TMP/refs.txt"
[ -s "$TMP/refs.txt" ] || continue # dashboard relies entirely on recording rules
comm -23 "$TMP/refs.txt" "$TMP/declared.txt" > "$TMP/missing.txt"
if [ -s "$TMP/missing.txt" ]; then
echo "ERROR: $dashboard_rel references DCGM metrics not declared in any CSV:" >&2
sed 's/^/ - /' "$TMP/missing.txt" >&2
failed=1
fi
done < "$TMP/dashboards.txt"
# Rules coverage — recording rules consume DCGM directly, so their set
# must be declared too, otherwise derived series on every dashboard
# collapse to empty.
extract_dcgm_refs "$RULES_FILE" > "$TMP/rule-refs.txt"
if [ -s "$TMP/rule-refs.txt" ]; then
comm -23 "$TMP/rule-refs.txt" "$TMP/declared.txt" > "$TMP/rule-missing.txt"
if [ -s "$TMP/rule-missing.txt" ]; then
echo "ERROR: gpu-recording.rules.yaml references DCGM metrics not declared in any CSV:" >&2
sed 's/^/ - /' "$TMP/rule-missing.txt" >&2
failed=1
fi
fi
[ "$failed" -eq 0 ]
}
@test "every tracked GPU dashboard listed in dashboards-infra.list exists on disk" {
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
list_tracked_gpu_dashboards > "$TMP/dashboards.txt"
[ -s "$TMP/dashboards.txt" ] || { echo "no gpu/* dashboards listed in $DASHBOARDS_LIST" >&2; exit 1; }
failed=0
while IFS= read -r dashboard_rel; do
dashboard="$DASHBOARDS_DIR/$dashboard_rel"
if [ ! -f "$dashboard" ]; then
echo "ERROR: $dashboard_rel listed in dashboards-infra.list but $dashboard does not exist" >&2
failed=1
fi
done < "$TMP/dashboards.txt"
[ "$failed" -eq 0 ]
}

View file

@ -1,104 +0,0 @@
# Snapshot of the upstream DCGM Exporter default-counters.csv used as a
# fixture by hack/check-gpu-recording-rules.bats. The test verifies that
# every DCGM_FI_* metric referenced by a tracked GPU dashboard is either
# declared here (upstream defaults) or in
# packages/system/gpu-operator/examples/dcgm-custom-metrics.yaml
# (the project's custom CSV).
#
# Source: https://github.com/NVIDIA/dcgm-exporter/blob/4.1.1-4.0.4/etc/default-counters.csv
# Pinned to the DCGM Exporter image tag shipped by the gpu-operator
# chart under packages/system/gpu-operator/charts/gpu-operator/values.yaml
# (dcgmExporter.version = 4.1.1-4.0.4-ubuntu22.04). When that image is
# bumped, refresh this file from the matching tag in the NVIDIA/dcgm-exporter
# repo and re-run `./hack/cozytest.sh hack/check-gpu-recording-rules.bats`.
# Format
# If line starts with a '#' it is considered a comment
# DCGM FIELD, Prometheus metric type, help message
# Clocks
DCGM_FI_DEV_SM_CLOCK, gauge, SM clock frequency (in MHz).
DCGM_FI_DEV_MEM_CLOCK, gauge, Memory clock frequency (in MHz).
# Temperature
DCGM_FI_DEV_MEMORY_TEMP, gauge, Memory temperature (in C).
DCGM_FI_DEV_GPU_TEMP, gauge, GPU temperature (in C).
# Power
DCGM_FI_DEV_POWER_USAGE, gauge, Power draw (in W).
DCGM_FI_DEV_TOTAL_ENERGY_CONSUMPTION, counter, Total energy consumption since boot (in mJ).
# PCIE
# DCGM_FI_PROF_PCIE_TX_BYTES, counter, Total number of bytes transmitted through PCIe TX via NVML.
# DCGM_FI_PROF_PCIE_RX_BYTES, counter, Total number of bytes received through PCIe RX via NVML.
DCGM_FI_DEV_PCIE_REPLAY_COUNTER, counter, Total number of PCIe retries.
# Utilization (the sample period varies depending on the product)
DCGM_FI_DEV_GPU_UTIL, gauge, GPU utilization (in %).
DCGM_FI_DEV_MEM_COPY_UTIL, gauge, Memory utilization (in %).
DCGM_FI_DEV_ENC_UTIL, gauge, Encoder utilization (in %).
DCGM_FI_DEV_DEC_UTIL , gauge, Decoder utilization (in %).
# Errors and violations
DCGM_FI_DEV_XID_ERRORS, gauge, Value of the last XID error encountered.
# DCGM_FI_DEV_POWER_VIOLATION, counter, Throttling duration due to power constraints (in us).
# DCGM_FI_DEV_THERMAL_VIOLATION, counter, Throttling duration due to thermal constraints (in us).
# DCGM_FI_DEV_SYNC_BOOST_VIOLATION, counter, Throttling duration due to sync-boost constraints (in us).
# DCGM_FI_DEV_BOARD_LIMIT_VIOLATION, counter, Throttling duration due to board limit constraints (in us).
# DCGM_FI_DEV_LOW_UTIL_VIOLATION, counter, Throttling duration due to low utilization (in us).
# DCGM_FI_DEV_RELIABILITY_VIOLATION, counter, Throttling duration due to reliability constraints (in us).
# Memory usage
DCGM_FI_DEV_FB_FREE, gauge, Framebuffer memory free (in MiB).
DCGM_FI_DEV_FB_USED, gauge, Framebuffer memory used (in MiB).
# ECC
# DCGM_FI_DEV_ECC_SBE_VOL_TOTAL, counter, Total number of single-bit volatile ECC errors.
# DCGM_FI_DEV_ECC_DBE_VOL_TOTAL, counter, Total number of double-bit volatile ECC errors.
# DCGM_FI_DEV_ECC_SBE_AGG_TOTAL, counter, Total number of single-bit persistent ECC errors.
# DCGM_FI_DEV_ECC_DBE_AGG_TOTAL, counter, Total number of double-bit persistent ECC errors.
# Retired pages
# DCGM_FI_DEV_RETIRED_SBE, counter, Total number of retired pages due to single-bit errors.
# DCGM_FI_DEV_RETIRED_DBE, counter, Total number of retired pages due to double-bit errors.
# DCGM_FI_DEV_RETIRED_PENDING, counter, Total number of pages pending retirement.
# NVLink
# DCGM_FI_DEV_NVLINK_CRC_FLIT_ERROR_COUNT_TOTAL, counter, Total number of NVLink flow-control CRC errors.
# DCGM_FI_DEV_NVLINK_CRC_DATA_ERROR_COUNT_TOTAL, counter, Total number of NVLink data CRC errors.
# DCGM_FI_DEV_NVLINK_REPLAY_ERROR_COUNT_TOTAL, counter, Total number of NVLink retries.
# DCGM_FI_DEV_NVLINK_RECOVERY_ERROR_COUNT_TOTAL, counter, Total number of NVLink recovery errors.
DCGM_FI_DEV_NVLINK_BANDWIDTH_TOTAL, counter, Total number of NVLink bandwidth counters for all lanes.
# DCGM_FI_DEV_NVLINK_BANDWIDTH_L0, counter, The number of bytes of active NVLink rx or tx data including both header and payload.
# VGPU License status
DCGM_FI_DEV_VGPU_LICENSE_STATUS, gauge, vGPU License status
# Remapped rows
DCGM_FI_DEV_UNCORRECTABLE_REMAPPED_ROWS, counter, Number of remapped rows for uncorrectable errors
DCGM_FI_DEV_CORRECTABLE_REMAPPED_ROWS, counter, Number of remapped rows for correctable errors
DCGM_FI_DEV_ROW_REMAP_FAILURE, gauge, Whether remapping of rows has failed
# Static configuration information. These appear as labels on the other metrics
DCGM_FI_DRIVER_VERSION, label, Driver Version
# DCGM_FI_NVML_VERSION, label, NVML Version
# DCGM_FI_DEV_BRAND, label, Device Brand
# DCGM_FI_DEV_SERIAL, label, Device Serial Number
# DCGM_FI_DEV_OEM_INFOROM_VER, label, OEM inforom version
# DCGM_FI_DEV_ECC_INFOROM_VER, label, ECC inforom version
# DCGM_FI_DEV_POWER_INFOROM_VER, label, Power management object inforom version
# DCGM_FI_DEV_INFOROM_IMAGE_VER, label, Inforom image version
# DCGM_FI_DEV_VBIOS_VERSION, label, VBIOS version of the device
# Datacenter Profiling (DCP) metrics
# NOTE: supported on Nvidia datacenter Volta GPUs and newer
DCGM_FI_PROF_GR_ENGINE_ACTIVE, gauge, Ratio of time the graphics engine is active.
# DCGM_FI_PROF_SM_ACTIVE, gauge, The ratio of cycles an SM has at least 1 warp assigned.
# DCGM_FI_PROF_SM_OCCUPANCY, gauge, The ratio of number of warps resident on an SM.
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE, gauge, Ratio of cycles the tensor (HMMA) pipe is active.
DCGM_FI_PROF_DRAM_ACTIVE, gauge, Ratio of cycles the device memory interface is active sending or receiving data.
# DCGM_FI_PROF_PIPE_FP64_ACTIVE, gauge, Ratio of cycles the fp64 pipes are active.
# DCGM_FI_PROF_PIPE_FP32_ACTIVE, gauge, Ratio of cycles the fp32 pipes are active.
# DCGM_FI_PROF_PIPE_FP16_ACTIVE, gauge, Ratio of cycles the fp16 pipes are active.
DCGM_FI_PROF_PCIE_TX_BYTES, gauge, The rate of data transmitted over the PCIe bus - including both protocol headers and data payloads - in bytes per second.
DCGM_FI_PROF_PCIE_RX_BYTES, gauge, The rate of data received over the PCIe bus - including both protocol headers and data payloads - in bytes per second.
Can't render this file because it has a wrong number of fields in line 12.

356
hack/e2e-apps/gateway.bats Normal file
View file

@ -0,0 +1,356 @@
#!/usr/bin/env bats
@test "Gateway API CRDs are installed and the cilium GatewayClass is Accepted" {
# Gateway API CRDs must exist — installed by packages/system/gateway-api-crds
kubectl wait crd/gatewayclasses.gateway.networking.k8s.io --for=condition=Established --timeout=60s
kubectl wait crd/gateways.gateway.networking.k8s.io --for=condition=Established --timeout=60s
kubectl wait crd/httproutes.gateway.networking.k8s.io --for=condition=Established --timeout=60s
# Cilium must have registered its built-in GatewayClass once gatewayAPI.enabled
# is true in the cilium values. This verifies the flip in
# packages/system/cilium/values.yaml propagated end-to-end.
timeout 120 sh -ec 'until kubectl get gatewayclass cilium >/dev/null 2>&1; do sleep 2; done'
kubectl wait gatewayclass/cilium --for=condition=Accepted --timeout=3m
}
@test "Cilium Gateway API controller reconciles a minimal Gateway to Programmed" {
# Use the pre-existing tenant-test namespace created by e2e-install-cozystack.bats.
kubectl apply -f - <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: gateway-e2e-probe
namespace: tenant-test
spec:
gatewayClassName: cilium
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: Same
EOF
# The controller must accept and program the Gateway.
kubectl -n tenant-test wait gateway/gateway-e2e-probe --for=condition=Accepted --timeout=2m
kubectl -n tenant-test wait gateway/gateway-e2e-probe --for=condition=Programmed --timeout=3m
# Cilium materialises a LoadBalancer Service named cilium-gateway-<gateway-name>
# for each programmed Gateway. Its existence is the observable proof that the
# full data-plane wiring kicked in.
kubectl -n tenant-test get svc cilium-gateway-gateway-e2e-probe
# Cleanup
kubectl -n tenant-test delete gateway/gateway-e2e-probe --ignore-not-found --timeout=1m
}
@test "cozystack-gateway-hostname-policy VAP and binding are installed" {
# Diagnose the hostname-policy enforcement path from the ground up.
# If these resources are missing, admission cannot reject anything.
kubectl get validatingadmissionpolicy cozystack-gateway-hostname-policy -o yaml
kubectl get validatingadmissionpolicybinding cozystack-gateway-hostname-policy -o yaml
# Binding MUST have validationActions: [Deny] — [Audit] / [] would let requests through silently.
actions=$(kubectl get validatingadmissionpolicybinding cozystack-gateway-hostname-policy -o jsonpath='{.spec.validationActions}')
echo "binding.validationActions=$actions"
case "$actions" in
*Deny*) ;;
*) echo "SETUP FAILURE: binding.validationActions lacks Deny (got '$actions')" >&2; return 1 ;;
esac
}
@test "tenant-test namespace carries namespace.cozystack.io/host label from tenant chart" {
# Diagnostic: the whole hostname-policy VAP keys off this label. If it is
# missing or empty, the VAP's matchCondition returns false, VAP skips, and
# EVERY Gateway in the namespace is admitted regardless of listener hostname.
# Make that bug loud instead of letting it fall through as a silent pass.
host_label=$(kubectl get namespace tenant-test -o jsonpath='{.metadata.labels.namespace\.cozystack\.io/host}')
if [ -z "$host_label" ]; then
echo "SETUP FAILURE: tenant-test namespace lacks namespace.cozystack.io/host label" >&2
kubectl get namespace tenant-test -o yaml >&2
return 1
fi
if [ "$host_label" != "test.example.org" ]; then
echo "SETUP FAILURE: tenant-test host label is '$host_label', expected 'test.example.org'" >&2
kubectl get namespace tenant-test -o yaml >&2
return 1
fi
}
@test "ValidatingAdmissionPolicy rejects Gateway with foreign hostname" {
# tenant-test namespace should only be allowed to publish its own
# domain suffix ('.test.example.org'); a listener hostname from the
# root tenant's apex must be denied by cozystack-gateway-hostname-policy.
# hack/cozytest.sh is /bin/sh (dash) with set -e — a failing command
# substitution propagates its exit status through variable assignment and
# kills the test. We specifically EXPECT admission to reject the apply, so
# use `if !` — set -e is disabled inside the if-condition, the exit status
# is captured, and $output is filled either way for the follow-up greps.
if ! output=$(kubectl apply -f - 2>&1 <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: hostname-hijack-probe
namespace: tenant-test
spec:
gatewayClassName: cilium
listeners:
- name: https
protocol: HTTPS
port: 443
hostname: "dashboard.example.org"
tls:
mode: Terminate
certificateRefs:
- name: noop
allowedRoutes:
namespaces:
from: Same
EOF
); then
# Happy path: admission rejected the Gateway. Verify the rejection came
# from our VAP and names the expected tenant host.
echo "$output" | grep -qi "ValidatingAdmissionPolicy"
echo "$output" | grep -q "must equal test.example.org"
else
echo "BUG: admission accepted cross-tenant hostname — Gateway 'hostname-hijack-probe' was created in tenant-test" >&2
echo "$output" >&2
return 1
fi
}
@test "cozystack-gateway-attached-namespaces-policy rejects Packages with tenant-* entries" {
# The platform Package default name is cozystack.cozystack-platform, managed by
# cozystack-api. Creating a dummy Package with tenant-alice in gateway.attachedNamespaces
# must fail at admission time.
if ! output=$(kubectl apply -f - 2>&1 <<'EOF'
apiVersion: cozystack.io/v1alpha1
kind: Package
metadata:
name: vap-reject-probe
spec:
variant: isp-full
components:
platform:
values:
gateway:
attachedNamespaces:
- tenant-alice
EOF
); then
echo "$output" | grep -qi "ValidatingAdmissionPolicy"
echo "$output" | grep -q "must not contain any tenant-"
else
echo "BUG: admission accepted tenant-* in attachedNamespaces — Package 'vap-reject-probe' was created" >&2
echo "$output" >&2
kubectl delete package vap-reject-probe --ignore-not-found
return 1
fi
}
@test "cozystack-tenant-host-policy blocks non-trusted callers from setting tenant.spec.host" {
# Impersonate a tenant-scoped ServiceAccount that is NOT in the trustedCaller
# group list. First grant RBAC to create Tenants — authorization runs BEFORE
# admission, so without this grant the apiserver returns a plain RBAC
# Forbidden and the test would fail grep-ing for 'ValidatingAdmissionPolicy'
# even though the VAP itself is fine.
kubectl apply -f - <<'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: vap-probe-tenant-create
namespace: tenant-test
rules:
- apiGroups: ["apps.cozystack.io"]
resources: ["tenants"]
verbs: ["create","get","list","watch","update","patch","delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: vap-probe-tenant-create
namespace: tenant-test
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: vap-probe-tenant-create
subjects:
- kind: ServiceAccount
name: default
namespace: tenant-test
EOF
if ! output=$(kubectl --as=system:serviceaccount:tenant-test:default \
--as-group=system:serviceaccounts \
--as-group=system:serviceaccounts:tenant-test \
apply -f - 2>&1 <<'EOF'
apiVersion: apps.cozystack.io/v1alpha1
kind: Tenant
metadata:
name: vaphostprobe
namespace: tenant-test
spec:
host: foreign.example.org
EOF
); then
kubectl -n tenant-test delete rolebinding vap-probe-tenant-create --ignore-not-found
kubectl -n tenant-test delete role vap-probe-tenant-create --ignore-not-found
echo "$output" | grep -qi "ValidatingAdmissionPolicy"
echo "$output" | grep -q "spec.host can only be set"
else
kubectl -n tenant-test delete tenants.apps.cozystack.io vaphostprobe --ignore-not-found
kubectl -n tenant-test delete rolebinding vap-probe-tenant-create --ignore-not-found
kubectl -n tenant-test delete role vap-probe-tenant-create --ignore-not-found
echo "BUG: admission accepted tenant.spec.host from untrusted SA — Tenant 'vaphostprobe' was created" >&2
echo "$output" >&2
return 1
fi
}
@test "cozystack-namespace-host-label-policy blocks non-trusted callers from changing the host label" {
# tenant-test namespace already has namespace.cozystack.io/host set by the
# cozystack tenant chart. Grant patch on namespaces cluster-wide to the
# impersonated SA — namespaces is a cluster-scoped resource so this needs a
# ClusterRole. Authorization runs before admission, so without this grant
# the test would fail with plain RBAC Forbidden rather than a VAP rejection.
kubectl apply -f - <<'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: vap-probe-namespace-patch
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get","list","watch","update","patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: vap-probe-namespace-patch
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: vap-probe-namespace-patch
subjects:
- kind: ServiceAccount
name: default
namespace: tenant-test
EOF
if ! output=$(kubectl --as=system:serviceaccount:tenant-test:default \
--as-group=system:serviceaccounts \
--as-group=system:serviceaccounts:tenant-test \
label namespace tenant-test \
namespace.cozystack.io/host=foreign.example.org --overwrite 2>&1); then
kubectl delete clusterrolebinding vap-probe-namespace-patch --ignore-not-found
kubectl delete clusterrole vap-probe-namespace-patch --ignore-not-found
echo "$output" | grep -qi "ValidatingAdmissionPolicy"
echo "$output" | grep -q "immutable"
else
# Revert label if apiserver somehow accepted the overwrite.
kubectl label namespace tenant-test namespace.cozystack.io/host=test.example.org --overwrite
kubectl delete clusterrolebinding vap-probe-namespace-patch --ignore-not-found
kubectl delete clusterrole vap-probe-namespace-patch --ignore-not-found
echo "BUG: admission accepted host label change from untrusted SA" >&2
echo "$output" >&2
return 1
fi
}
@test "cozystack-namespace-host-label-policy blocks non-trusted callers from setting the host label at CREATE" {
# Defense-in-depth: a non-trusted caller must not be able to stamp
# namespace.cozystack.io/host=X on a brand-new namespace either — only
# cozystack/Flux SAs may write the label. Authorization runs before
# admission, so grant cluster-wide namespace create to the impersonated SA
# first, otherwise the test would fail with plain RBAC Forbidden.
kubectl apply -f - <<'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: vap-probe-namespace-create
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["create","get","list","delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: vap-probe-namespace-create
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: vap-probe-namespace-create
subjects:
- kind: ServiceAccount
name: default
namespace: tenant-test
EOF
if ! output=$(kubectl --as=system:serviceaccount:tenant-test:default \
--as-group=system:serviceaccounts \
--as-group=system:serviceaccounts:tenant-test \
apply -f - 2>&1 <<'EOF'
apiVersion: v1
kind: Namespace
metadata:
name: vap-host-label-probe
labels:
namespace.cozystack.io/host: foreign.example.org
EOF
); then
kubectl delete clusterrolebinding vap-probe-namespace-create --ignore-not-found
kubectl delete clusterrole vap-probe-namespace-create --ignore-not-found
echo "$output" | grep -qi "ValidatingAdmissionPolicy"
echo "$output" | grep -q "immutable"
else
kubectl delete namespace vap-host-label-probe --ignore-not-found --wait=false
kubectl delete clusterrolebinding vap-probe-namespace-create --ignore-not-found
kubectl delete clusterrole vap-probe-namespace-create --ignore-not-found
echo "BUG: admission accepted first-time host label write from untrusted SA at CREATE — Namespace 'vap-host-label-probe' was created" >&2
echo "$output" >&2
return 1
fi
}
@test "HTTPRoute with a matching parentRef reaches Accepted status" {
# Put a Gateway and a route in the same namespace so allowedRoutes: Same accepts them.
kubectl apply -f - <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: gateway-route-probe
namespace: tenant-test
spec:
gatewayClassName: cilium
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: httproute-probe
namespace: tenant-test
spec:
parentRefs:
- name: gateway-route-probe
sectionName: http
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: kubernetes
namespace: default
port: 443
EOF
kubectl -n tenant-test wait gateway/gateway-route-probe --for=condition=Programmed --timeout=3m
timeout 120 sh -ec 'until kubectl -n tenant-test get httproute httproute-probe -o jsonpath="{.status.parents[0].conditions[?(@.type==\"Accepted\")].status}" 2>/dev/null | grep -q True; do sleep 2; done'
kubectl -n tenant-test delete httproute/httproute-probe --ignore-not-found
kubectl -n tenant-test delete gateway/gateway-route-probe --ignore-not-found
}

View file

@ -1,39 +0,0 @@
# Helpers for asserting that a Flux HelmRelease did not fall into an
# install/upgrade remediation cycle during an e2e run.
#
# Background: Flux helm-controller's ClearFailures() zeroes
# .status.installFailures / .status.upgradeFailures on every successful
# reconciliation (see the upstream ClearFailures method on
# HelmReleaseStatus). That makes those counters useless for a guard that
# runs after the HelmRelease has reached Ready - the values are always 0.
#
# What survives a successful reconciliation is .status.history, a bounded
# list of release Snapshots. Each Snapshot carries a status field that
# tracks the Helm release state: deployed, superseded, failed, uninstalled,
# and so on. A remediation cycle leaves the footprint behind: a snapshot
# with status "uninstalled" (from install/upgrade remediation) or "failed"
# (Helm release failure that remediation then uninstalled). Those stay in
# history even after a subsequent successful reinstall.
#
# helmrelease_has_remediation_cycle takes a newline-delimited list of
# snapshot statuses (whatever the caller extracted via kubectl -o jsonpath
# or equivalent) and returns 0 (detected) when any entry is "failed" or
# "uninstalled", 1 otherwise. Empty input is treated as "no history yet,
# no cycle observed".
helmrelease_has_remediation_cycle() {
statuses="$1"
if [ -z "${statuses}" ]; then
return 1
fi
# printf + grep over the pipe, rather than a heredoc plus while read.
# printf %s treats the status string as a literal payload, so any stray
# $ in a future caller's input does not trigger shell expansion. grep
# returns 0 iff at least one line matches the allowlist, which is
# exactly the contract the caller wants, so we can return its exit
# status directly.
if printf '%s\n' "${statuses}" | grep --extended-regexp --quiet '^(failed|uninstalled)$'; then
return 0
fi
return 1
}

View file

@ -1,5 +1,3 @@
. hack/e2e-apps/remediation-guard.sh
run_kubernetes_test() {
local version_expr="$1"
local test_name="$2"
@ -322,35 +320,6 @@ EOF
done
kubectl wait hr kubernetes-${test_name}-ingress-nginx -n tenant-test --timeout=5m --for=condition=ready
# Guard: parent HelmRelease must not have entered an install/upgrade remediation cycle.
# A non-zero installFailures/upgradeFailures indicates the helm-wait budget expired while
# admin-kubeconfig was still being provisioned, which would trigger uninstall remediation
# and churn the Cluster CR.
# Flux helm-controller v2 retains per-revision release Snapshots in
# .status.history; each Snapshot's .status reflects the Helm release
# state (deployed/superseded/failed/uninstalled). A remediation cycle
# leaves a "failed" or "uninstalled" entry behind that survives a later
# successful reinstall, unlike the installFailures/upgradeFailures
# counters (which ClearFailures zeroes on every successful reconcile).
# The shape is pinned by hack/remediation-guard.bats; the upstream
# types are github.com/fluxcd/helm-controller/api v2 Snapshot.
history_statuses=$(kubectl get hr -n tenant-test "kubernetes-${test_name}" \
-ojsonpath='{range .status.history[*]}{.status}{"\n"}{end}')
# Always emit the raw value so a silent future-Flux field rename shows
# up as "empty history on a Ready HR" in CI logs rather than vanishing.
echo "Parent HelmRelease history statuses:"
printf '%s\n' "${history_statuses:-<empty>}"
if [ -z "${history_statuses}" ]; then
echo "Unexpected empty .status.history on a Ready HelmRelease - Flux API shape may have changed." >&2
kubectl -n tenant-test describe hr "kubernetes-${test_name}" >&2
exit 1
fi
if helmrelease_has_remediation_cycle "${history_statuses}"; then
echo "Parent HelmRelease entered remediation cycle." >&2
kubectl -n tenant-test describe hr "kubernetes-${test_name}" >&2
exit 1
fi
# Clean up
pkill -f "port-forward.*${port}:" 2>/dev/null || true
rm -f "tenantkubeconfig-${test_name}"

View file

@ -1,127 +0,0 @@
#!/usr/bin/env bats
# -----------------------------------------------------------------------------
# Unit tests for hack/e2e-apps/remediation-guard.sh
#
# helmrelease_has_remediation_cycle takes a newline-delimited list of
# HelmRelease history snapshot status values (deployed/superseded/failed/
# uninstalled/...) and returns 0 when any entry is "failed" or "uninstalled"
# (meaning flux helm-controller performed install/upgrade remediation).
#
# This is used by the e2e script after the HelmRelease reaches Ready. The
# failure/upgrade counters (.status.installFailures / .status.upgradeFailures)
# are useless there because flux's ClearFailures zeroes them on successful
# reconciliation; .status.history retains the snapshot trail.
#
# cozytest.sh's awk parser recognizes only @test blocks and a bare `}` on
# its own line; there is no bats `run` or `$status`. Assertions are
# expressed as direct shell tests that exit non-zero on failure.
#
# Run with: hack/cozytest.sh hack/remediation-guard.bats
# -----------------------------------------------------------------------------
@test "empty history returns not-detected" {
. hack/e2e-apps/remediation-guard.sh
if helmrelease_has_remediation_cycle ""; then
echo "expected not-detected for empty history" >&2
exit 1
fi
}
@test "single deployed snapshot returns not-detected" {
. hack/e2e-apps/remediation-guard.sh
if helmrelease_has_remediation_cycle "deployed"; then
echo "expected not-detected for deployed-only history" >&2
exit 1
fi
}
@test "deployed then superseded returns not-detected" {
. hack/e2e-apps/remediation-guard.sh
statuses=$(printf 'deployed\nsuperseded\n')
if helmrelease_has_remediation_cycle "${statuses}"; then
echo "expected not-detected for deployed+superseded history" >&2
exit 1
fi
}
@test "single failed snapshot returns detected" {
. hack/e2e-apps/remediation-guard.sh
if ! helmrelease_has_remediation_cycle "failed"; then
echo "expected detected when history contains failed snapshot" >&2
exit 1
fi
}
@test "single uninstalled snapshot returns detected" {
# The exact signature of the install-remediation race: the first install
# exceeded flux's wait budget, remediation uninstalled, the next retry
# eventually succeeded. History still carries the uninstalled snapshot.
. hack/e2e-apps/remediation-guard.sh
if ! helmrelease_has_remediation_cycle "uninstalled"; then
echo "expected detected when history contains uninstalled snapshot" >&2
exit 1
fi
}
@test "uninstalled then deployed still returns detected" {
. hack/e2e-apps/remediation-guard.sh
statuses=$(printf 'uninstalled\ndeployed\n')
if ! helmrelease_has_remediation_cycle "${statuses}"; then
echo "expected detected despite later successful deploy" >&2
exit 1
fi
}
@test "deployed then failed still returns detected" {
. hack/e2e-apps/remediation-guard.sh
statuses=$(printf 'deployed\nfailed\n')
if ! helmrelease_has_remediation_cycle "${statuses}"; then
echo "expected detected when any entry is failed" >&2
exit 1
fi
}
@test "status.history extraction pins HR v2 status.history shape" {
# Pins the Flux HelmRelease v2 .status.history[].status shape that
# run-kubernetes.sh relies on. If a future flux release renames the
# field, the jsonpath returns nothing, the guard reports no cycle,
# and real remediation loops slip past the e2e assertion. This test
# uses yq to read the exact path used in the e2e script; the upstream
# Snapshot type lives at
# github.com/fluxcd/helm-controller/api/v2.Snapshot (via go.mod).
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cat > "$tmp/hr.yaml" <<'YAML'
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: kubernetes-test
namespace: tenant-test
status:
history:
- name: kubernetes-test
namespace: tenant-test
version: 1
status: uninstalled
- name: kubernetes-test
namespace: tenant-test
version: 2
status: deployed
YAML
# Default yq output is yaml scalar format, which for string values emits
# bare unquoted tokens - matching what kubectl -o jsonpath produces in
# e2e. Do not switch to JSON output here; that would quote the values
# and break the loop in helmrelease_has_remediation_cycle.
statuses=$(yq '.status.history[].status' "$tmp/hr.yaml")
[ -n "$statuses" ]
echo "$statuses" | grep --quiet '^uninstalled$'
. hack/e2e-apps/remediation-guard.sh
if ! helmrelease_has_remediation_cycle "$statuses"; then
echo "expected detected for pinned HR snippet with uninstalled + deployed history" >&2
exit 1
fi
}

View file

@ -118,19 +118,6 @@ spec:
ReleaseName is the name of the HelmRelease resource that will be created
If not specified, defaults to the component Name field
type: string
upgradeCRDs:
description: |-
UpgradeCRDs controls how CRDs from the chart's crds/ directory are
handled on HelmRelease upgrades. Maps to HelmRelease.Spec.Upgrade.CRDs.
Empty string (default) preserves the helm-controller default (Skip).
Use "CreateReplace" for operators that evolve their CRD set between
versions. Warning: CreateReplace overwrites CRDs and may cause data
loss if upstream drops fields from a CRD with live objects.
enum:
- Skip
- Create
- CreateReplace
type: string
type: object
libraries:
description: |-

View file

@ -45,16 +45,6 @@ const (
SecretCozystackValues = "cozystack-values"
)
// parseCRDPolicy maps ComponentInstall.UpgradeCRDs to a helmv2.CRDsPolicy.
// Empty / nil preserves the helm-controller default (Skip on upgrade);
// the CRD enum marker restricts the string to Skip/Create/CreateReplace.
func parseCRDPolicy(install *cozyv1alpha1.ComponentInstall) helmv2.CRDsPolicy {
if install == nil || install.UpgradeCRDs == "" {
return ""
}
return helmv2.CRDsPolicy(install.UpgradeCRDs)
}
// PackageReconciler reconciles Package resources
type PackageReconciler struct {
client.Client
@ -231,7 +221,6 @@ func (r *PackageReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ct
Remediation: &helmv2.UpgradeRemediation{
Retries: -1,
},
CRDs: parseCRDPolicy(component.Install),
},
},
}

View file

@ -1,138 +0,0 @@
/*
Copyright 2025 The Cozystack Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package operator
import (
"encoding/json"
"os"
"path/filepath"
"testing"
cozyv1alpha1 "github.com/cozystack/cozystack/api/v1alpha1"
helmv2 "github.com/fluxcd/helm-controller/api/v2"
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
"sigs.k8s.io/yaml"
)
func TestParseCRDPolicy(t *testing.T) {
tests := []struct {
name string
install *cozyv1alpha1.ComponentInstall
want helmv2.CRDsPolicy
}{
{
name: "nil install leaves flux default",
install: nil,
want: "",
},
{
name: "empty upgradeCRDs leaves flux default",
install: &cozyv1alpha1.ComponentInstall{},
want: "",
},
{
name: "Skip is passed through",
install: &cozyv1alpha1.ComponentInstall{UpgradeCRDs: "Skip"},
want: helmv2.Skip,
},
{
name: "Create is passed through",
install: &cozyv1alpha1.ComponentInstall{UpgradeCRDs: "Create"},
want: helmv2.Create,
},
{
name: "CreateReplace is passed through",
install: &cozyv1alpha1.ComponentInstall{UpgradeCRDs: "CreateReplace"},
want: helmv2.CreateReplace,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := parseCRDPolicy(tc.install)
if got != tc.want {
t.Errorf("parseCRDPolicy() = %q, want %q", got, tc.want)
}
})
}
}
// TestPackageSourceCRDHasUpgradeCRDsEnum guards the generated CRD schema: the
// invalid-value case from the spec is enforced at the API server via a
// kubebuilder enum marker, not in the reconciler. If someone drops the marker
// and forgets to regenerate, this test catches it.
func TestPackageSourceCRDHasUpgradeCRDsEnum(t *testing.T) {
path := filepath.Join("..", "crdinstall", "manifests", "cozystack.io_packagesources.yaml")
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
var crd apiextensionsv1.CustomResourceDefinition
if err := yaml.Unmarshal(data, &crd); err != nil {
t.Fatalf("unmarshal CRD: %v", err)
}
var field *apiextensionsv1.JSONSchemaProps
for i := range crd.Spec.Versions {
v := &crd.Spec.Versions[i]
if v.Schema == nil || v.Schema.OpenAPIV3Schema == nil {
continue
}
spec, ok := v.Schema.OpenAPIV3Schema.Properties["spec"]
if !ok {
continue
}
variants, ok := spec.Properties["variants"]
if !ok || variants.Items == nil || variants.Items.Schema == nil {
continue
}
components, ok := variants.Items.Schema.Properties["components"]
if !ok || components.Items == nil || components.Items.Schema == nil {
continue
}
install, ok := components.Items.Schema.Properties["install"]
if !ok {
continue
}
f, ok := install.Properties["upgradeCRDs"]
if !ok {
continue
}
field = &f
break
}
if field == nil {
t.Fatal("upgradeCRDs field not found in PackageSource CRD schema")
}
got := map[string]bool{}
for _, e := range field.Enum {
var s string
if err := json.Unmarshal(e.Raw, &s); err != nil {
t.Fatalf("unmarshal enum value %q: %v", e.Raw, err)
}
got[s] = true
}
for _, want := range []string{"Skip", "Create", "CreateReplace"} {
if !got[want] {
t.Errorf("enum value %q missing from upgradeCRDs; got %v", want, got)
}
}
}

View file

@ -0,0 +1,22 @@
{{- $gateway := .Values._namespace.gateway | default "" }}
{{- $host := .Values._namespace.host }}
{{- $harborHost := .Values.host | default (printf "%s.%s" .Release.Name $host) }}
{{- if $gateway }}
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: {{ .Release.Name }}
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: cozystack
namespace: {{ $gateway }}
sectionName: https
hostnames:
- {{ $harborHost | quote }}
rules:
- backendRefs:
- name: {{ .Release.Name }}
port: 80
{{- end }}

View file

@ -1,8 +1,10 @@
{{- $ingress := .Values._namespace.ingress }}
{{- $gateway := .Values._namespace.gateway | default "" }}
{{- $host := .Values._namespace.host }}
{{- $harborHost := .Values.host | default (printf "%s.%s" .Release.Name $host) }}
{{- $solver := (index .Values._cluster "solver") | default "http01" }}
{{- $clusterIssuer := (index .Values._cluster "issuer-name") | default "letsencrypt-prod" }}
{{- if not $gateway }}
---
apiVersion: networking.k8s.io/v1
kind: Ingress
@ -35,3 +37,4 @@ spec:
name: {{ .Release.Name }}
port:
number: 80
{{- end }}

View file

@ -4,9 +4,6 @@ KUBERNETES_PKG_TAG = $(shell awk '$$1 == "version:" {print $$2}' Chart.yaml)
include ../../../hack/common-envs.mk
include ../../../hack/package.mk
test:
helm unittest .
generate:
cozyvalues-gen -m 'kubernetes' -v values.yaml -s values.schema.json -r README.md -g ../../../api/apps/v1alpha1/kubernetes/types.go
../../../hack/update-crd.sh
@ -70,4 +67,3 @@ image-cluster-autoscaler:
echo "$(REGISTRY)/cluster-autoscaler:$(call settag,$(KUBERNETES_PKG_TAG))@$$(yq e '."containerimage.digest"' images/cluster-autoscaler.json -o json -r)" \
> images/cluster-autoscaler.tag
rm -f images/cluster-autoscaler.json

View file

@ -128,9 +128,6 @@ See the reference for components utilized in this service:
| `addons.gpuOperator` | NVIDIA GPU Operator. | `object` | `{}` |
| `addons.gpuOperator.enabled` | Enable GPU Operator. | `bool` | `false` |
| `addons.gpuOperator.valuesOverride` | Custom Helm values overrides. | `object` | `{}` |
| `addons.hami` | HAMi GPU virtualization middleware. | `object` | `{}` |
| `addons.hami.enabled` | Enable HAMi (requires GPU Operator). | `bool` | `false` |
| `addons.hami.valuesOverride` | Custom Helm values overrides. | `object` | `{}` |
| `addons.fluxcd` | FluxCD GitOps operator. | `object` | `{}` |
| `addons.fluxcd.enabled` | Enable FluxCD. | `bool` | `false` |
| `addons.fluxcd.valuesOverride` | Custom Helm values overrides. | `object` | `{}` |
@ -148,33 +145,31 @@ See the reference for components utilized in this service:
### Kubernetes Control Plane Configuration
| Name | Description | Type | Value |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------- | ---------- | ------- |
| `controlPlane` | Kubernetes control-plane configuration. | `object` | `{}` |
| `controlPlane.replicas` | Number of control-plane replicas. | `int` | `2` |
| `controlPlane.apiServer` | API Server configuration. | `object` | `{}` |
| `controlPlane.apiServer.resources` | CPU and memory resources for API Server. | `object` | `{}` |
| `controlPlane.apiServer.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.apiServer.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.apiServer.resourcesPreset` | Preset if `resources` omitted. | `string` | `large` |
| `controlPlane.controllerManager` | Controller Manager configuration. | `object` | `{}` |
| `controlPlane.controllerManager.resources` | CPU and memory resources for Controller Manager. | `object` | `{}` |
| `controlPlane.controllerManager.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.controllerManager.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.controllerManager.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
| `controlPlane.scheduler` | Scheduler configuration. | `object` | `{}` |
| `controlPlane.scheduler.resources` | CPU and memory resources for Scheduler. | `object` | `{}` |
| `controlPlane.scheduler.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.scheduler.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.scheduler.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
| `controlPlane.konnectivity` | Konnectivity configuration. | `object` | `{}` |
| `controlPlane.konnectivity.server` | Konnectivity Server configuration. | `object` | `{}` |
| `controlPlane.konnectivity.server.resources` | CPU and memory resources for Konnectivity. | `object` | `{}` |
| `controlPlane.konnectivity.server.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.konnectivity.server.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.konnectivity.server.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
| `images` | Optional image overrides for air-gapped or rate-limited registries. | `object` | `{}` |
| `images.waitForKubeconfig` | Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag. | `string` | `""` |
| Name | Description | Type | Value |
| --------------------------------------------------- | ------------------------------------------------ | ---------- | ------- |
| `controlPlane` | Kubernetes control-plane configuration. | `object` | `{}` |
| `controlPlane.replicas` | Number of control-plane replicas. | `int` | `2` |
| `controlPlane.apiServer` | API Server configuration. | `object` | `{}` |
| `controlPlane.apiServer.resources` | CPU and memory resources for API Server. | `object` | `{}` |
| `controlPlane.apiServer.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.apiServer.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.apiServer.resourcesPreset` | Preset if `resources` omitted. | `string` | `large` |
| `controlPlane.controllerManager` | Controller Manager configuration. | `object` | `{}` |
| `controlPlane.controllerManager.resources` | CPU and memory resources for Controller Manager. | `object` | `{}` |
| `controlPlane.controllerManager.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.controllerManager.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.controllerManager.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
| `controlPlane.scheduler` | Scheduler configuration. | `object` | `{}` |
| `controlPlane.scheduler.resources` | CPU and memory resources for Scheduler. | `object` | `{}` |
| `controlPlane.scheduler.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.scheduler.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.scheduler.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
| `controlPlane.konnectivity` | Konnectivity configuration. | `object` | `{}` |
| `controlPlane.konnectivity.server` | Konnectivity Server configuration. | `object` | `{}` |
| `controlPlane.konnectivity.server.resources` | CPU and memory resources for Konnectivity. | `object` | `{}` |
| `controlPlane.konnectivity.server.resources.cpu` | CPU available. | `quantity` | `""` |
| `controlPlane.konnectivity.server.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
| `controlPlane.konnectivity.server.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
## Parameter examples and reference

View file

@ -1 +0,0 @@
docker.io/library/busybox:1.37.0@sha256:1487d0af5f52b4ba31c7e465126ee2123fe3f2305d638e7827681e7cf6c83d5e

View file

@ -49,52 +49,3 @@ Selector labels
app.kubernetes.io/name: {{ include "kubernetes.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
wait-for-kubeconfig init container shared by the control-plane-side
Deployments (cluster-autoscaler, kccm, kcsi-controller) that mount the
*-admin-kubeconfig Secret provisioned asynchronously by Kamaji. The
Secret volume is declared optional so kubelet does not FailedMount while
Kamaji is still bootstrapping; this container polls the mounted path and
exits only when super-admin.svc appears, which happens after kubelet's
optional-Secret refresh cycle.
The 10m deadline stays strictly below the 15m HelmRelease
Install.Timeout set by cozystack-api for the Kubernetes kind (via the
release.cozystack.io/helm-install-timeout annotation) so the
CrashLoopBackOff surfaces before flux remediation fires and uninstalls
the Cluster CR.
The default image lives in images/busybox.tag and points directly at
docker.io by digest (not mirrored to ghcr.io like the other .tag files
here): the payload is a one-shot sh loop and the digest pin makes the
pull immutable. Operators in air-gapped or rate-limited environments
can override it via .Values.images.waitForKubeconfig (any registry
reference kubelet can pull). When the value is empty the chart falls
back to the bundled digest pin, preserving the prior default.
Call site owns the surrounding volumes block; the kubeconfig volume
must exist on the pod and mount at /etc/kubernetes/kubeconfig.
*/}}
{{- define "kubernetes.waitForAdminKubeconfig" -}}
- name: wait-for-kubeconfig
image: "{{ default (.Files.Get "images/busybox.tag" | trim) .Values.images.waitForKubeconfig }}"
command:
- sh
- -c
- |
set -eu
deadline=$(( $(date +%s) + 600 ))
until [ -s /etc/kubernetes/kubeconfig/super-admin.svc ]; do
if [ "$(date +%s)" -ge "$deadline" ]; then
echo "admin kubeconfig was not provisioned within 10m; exiting so the pod goes CrashLoopBackOff and surfaces in dashboards" >&2
exit 1
fi
echo "waiting for admin kubeconfig (provisioned by Kamaji, visible after kubelet Secret refresh)..."
sleep 5
done
volumeMounts:
- name: kubeconfig
mountPath: /etc/kubernetes/kubeconfig
readOnly: true
{{- end }}

View file

@ -1,14 +1,3 @@
{{- /*
Gate the control-plane-side workloads on the parent tenant having an etcd
DataStore. Without it no KamajiControlPlane is ever created, Kamaji never
provisions -admin-kubeconfig, and rendering these Deployments would cause
the wait-for-kubeconfig init to CrashLoopBackOff indefinitely, consuming
the parent HelmRelease install timeout and triggering the very uninstall
remediation cycle this chart is supposed to avoid. Rendering them only
when $etcd is set keeps the HelmRelease Ready while flux retries on its
interval and picks up the DataStore as soon as the Tenant chart finishes.
*/}}
{{- if .Values._namespace.etcd }}
---
apiVersion: apps/v1
kind: Deployment
@ -34,8 +23,6 @@ spec:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: "NoSchedule"
initContainers:
{{- include "kubernetes.waitForAdminKubeconfig" $ | nindent 6 }}
containers:
- image: "{{ $.Files.Get "images/cluster-autoscaler.tag" | trim }}"
name: cluster-autoscaler
@ -69,7 +56,6 @@ spec:
name: cloud-config
- secret:
secretName: {{ .Release.Name }}-admin-kubeconfig
optional: true
name: kubeconfig
serviceAccountName: {{ .Release.Name }}-cluster-autoscaler
terminationGracePeriodSeconds: 10
@ -119,4 +105,3 @@ rules:
- list
- update
- watch
{{- end }}

View file

@ -1,15 +1,4 @@
{{- $etcd := .Values._namespace.etcd }}
{{- /*
When $etcd is empty, the parent Tenant application has not populated
_namespace.etcd in cozystack-values yet - either the operator forgot to
set etcd: true on an ancestor Tenant, or the Tenant HelmRelease is still
reconciling. Either way, rendering a KamajiControlPlane with an empty
dataStoreName would be rejected by Kamaji's admission webhook and the
HelmRelease would fail to install, triggering remediation. Instead, emit
a single ConfigMap as a user-visible status beacon and skip the rest so
flux marks the HelmRelease Ready and retries its 5m reconcile loop until
the Tenant chart catches up.
*/}}
{{- $ingress := .Values._namespace.ingress }}
{{- $host := .Values._namespace.host }}
{{- $kubevirtmachinetemplateNames := list }}
@ -95,26 +84,6 @@ spec:
- name: default
pod: {}
{{- end }}
{{- if not $etcd }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-awaiting-etcd
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/name: kubernetes
app.kubernetes.io/instance: {{ .Release.Name }}
data:
status: "awaiting-etcd"
message: |
No DataStore is available for this tenant Kubernetes cluster. The parent
Tenant application has not populated _namespace.etcd. Set spec.etcd: true
on an ancestor Tenant (usually tenant-root) and wait for its HelmRelease
to reconcile - this HelmRelease will pick up the DataStore on its next
5m reconcile loop and provision the cluster.
{{- else }}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
@ -435,4 +404,3 @@ metadata:
spec:
{{- .spec | toYaml | nindent 2 }}
{{- end }}
{{- end }}

View file

@ -1,4 +1,3 @@
{{- if .Values._namespace.etcd }}
kind: Deployment
apiVersion: apps/v1
metadata:
@ -25,8 +24,6 @@ spec:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: "NoSchedule"
initContainers:
{{- include "kubernetes.waitForAdminKubeconfig" $ | nindent 6 }}
containers:
- name: csi-driver
imagePullPolicy: Always
@ -237,6 +234,4 @@ spec:
emptyDir: {}
- secret:
secretName: {{ .Release.Name }}-admin-kubeconfig
optional: true
name: kubeconfig
{{- end }}

View file

@ -1,4 +1,4 @@
{{- if and .Values.addons.certManager.enabled .Values._namespace.etcd }}
{{- if .Values.addons.certManager.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -8,7 +8,7 @@ cert-manager:
{{- end }}
{{- end }}
{{- if and .Values.addons.certManager.enabled .Values._namespace.etcd }}
{{- if .Values.addons.certManager.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -14,7 +14,6 @@ cilium:
{{- end }}
{{- end }}
{{- if .Values._namespace.etcd }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
@ -56,4 +55,3 @@ spec:
- name: {{ .Release.Name }}-gateway-api-crds
namespace: {{ .Release.Namespace }}
{{- end }}
{{- end }}

View file

@ -4,7 +4,6 @@ coredns:
clusterIP: "10.95.0.10"
{{- end }}
{{- if .Values._namespace.etcd }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
@ -43,4 +42,3 @@ spec:
{{- end }}
- name: {{ .Release.Name }}-cilium
namespace: {{ .Release.Namespace }}
{{- end }}

View file

@ -1,4 +1,3 @@
{{- if .Values._namespace.etcd }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
@ -40,4 +39,3 @@ spec:
- name: {{ .Release.Name }}
namespace: {{ .Release.Namespace }}
{{- end }}
{{- end }}

View file

@ -1,4 +1,4 @@
{{- if and .Values.addons.fluxcd.enabled .Values._namespace.etcd }}
{{- if .Values.addons.fluxcd.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -1,4 +1,4 @@
{{- if and $.Values.addons.gatewayAPI.enabled $.Values._namespace.etcd }}
{{- if $.Values.addons.gatewayAPI.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -1,12 +1,4 @@
{{- define "cozystack.defaultGpuOperatorValues" -}}
{{- if .Values.addons.hami.enabled }}
gpu-operator:
devicePlugin:
enabled: false
{{- end }}
{{- end }}
{{- if and .Values.addons.gpuOperator.enabled .Values._namespace.etcd }}
{{- if .Values.addons.gpuOperator.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
@ -37,12 +29,9 @@ spec:
force: true
remediation:
retries: -1
{{- $defaults := fromYaml (include "cozystack.defaultGpuOperatorValues" .) }}
{{- $overrides := deepCopy (default (dict) .Values.addons.gpuOperator.valuesOverride) }}
{{- $merged := mergeOverwrite (default (dict) $defaults) $overrides }}
{{- if $merged }}
{{- with .Values.addons.gpuOperator.valuesOverride }}
values:
{{- toYaml $merged | nindent 4 }}
{{- toYaml . | nindent 4 }}
{{- end }}
dependsOn:

View file

@ -1,49 +0,0 @@
{{- if and .Values.addons.hami.enabled .Values._namespace.etcd }}
{{- if not .Values.addons.gpuOperator.enabled }}
{{- fail "addons.hami requires addons.gpuOperator to be enabled" }}
{{- end }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: {{ .Release.Name }}-hami
labels:
cozystack.io/repository: system
cozystack.io/target-cluster-name: {{ .Release.Name }}
sharding.fluxcd.io/key: tenants
spec:
releaseName: hami
chartRef:
kind: ExternalArtifact
name: cozystack-kubernetes-application-kubevirt-kubernetes-hami
namespace: cozy-system
kubeConfig:
secretRef:
name: {{ .Release.Name }}-admin-kubeconfig
key: super-admin.svc
targetNamespace: cozy-hami
storageNamespace: cozy-hami
interval: 5m
timeout: 10m
install:
createNamespace: true
remediation:
retries: -1
upgrade:
force: true
remediation:
retries: -1
{{- with .Values.addons.hami.valuesOverride }}
values:
{{- toYaml . | nindent 4 }}
{{- end }}
dependsOn:
{{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
- name: {{ .Release.Name }}
namespace: {{ .Release.Namespace }}
{{- end }}
- name: {{ .Release.Name }}-cilium
namespace: {{ .Release.Namespace }}
- name: {{ .Release.Name }}-gpu-operator
namespace: {{ .Release.Namespace }}
{{- end }}

View file

@ -20,7 +20,7 @@ ingress-nginx:
node-role.kubernetes.io/ingress-nginx: ""
{{- end }}
{{- if and .Values.addons.ingressNginx.enabled .Values._namespace.etcd }}
{{- if .Values.addons.ingressNginx.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -1,4 +1,3 @@
{{- if .Values._namespace.etcd }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
@ -37,4 +36,3 @@ spec:
namespace: {{ .Release.Namespace }}
- name: {{ .Release.Name }}-prometheus-operator-crds
namespace: {{ .Release.Namespace }}
{{- end }}

View file

@ -1,6 +1,6 @@
{{- $targetTenant := .Values._namespace.monitoring }}
{{- $clusterDomain := (index .Values._cluster "cluster-domain") | default "cozy.local" }}
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
{{- if .Values.addons.monitoringAgents.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -1,4 +1,3 @@
{{- if .Values._namespace.etcd }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
@ -32,4 +31,3 @@ spec:
- name: {{ .Release.Name }}
namespace: {{ .Release.Namespace }}
{{- end }}
{{- end }}

View file

@ -1,4 +1,4 @@
{{- if and .Values.addons.velero.enabled .Values._namespace.etcd }}
{{- if .Values.addons.velero.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -1,4 +1,4 @@
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
{{- if .Values.addons.monitoringAgents.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -24,7 +24,7 @@ vertical-pod-autoscaler:
memory: 1600Mi
{{- end }}
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
{{- if .Values.addons.monitoringAgents.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -1,4 +1,4 @@
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
{{- if .Values.addons.monitoringAgents.enabled }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:

View file

@ -1,4 +1,3 @@
{{- if .Values._namespace.etcd }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
@ -34,4 +33,3 @@ spec:
- name: {{ .Release.Name }}
namespace: {{ .Release.Namespace }}
{{- end }}
{{- end }}

View file

@ -1,4 +1,3 @@
{{- if .Values._namespace.etcd }}
apiVersion: apps/v1
kind: Deployment
metadata:
@ -23,8 +22,6 @@ spec:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: "NoSchedule"
initContainers:
{{- include "kubernetes.waitForAdminKubeconfig" $ | nindent 6 }}
containers:
- name: kubevirt-cloud-controller-manager
args:
@ -58,7 +55,5 @@ spec:
name: cloud-config
- secret:
secretName: {{ .Release.Name }}-admin-kubeconfig
optional: true
name: kubeconfig
serviceAccountName: {{ .Release.Name }}-kccm
{{- end }}

View file

@ -1,155 +0,0 @@
suite: admin-kubeconfig wait guards
release:
name: test
namespace: tenant-root
values:
- values-ci.yaml
tests:
- it: cluster-autoscaler mounts admin-kubeconfig as optional
template: templates/cluster-autoscaler/deployment.yaml
documentSelector:
path: kind
value: Deployment
asserts:
- equal:
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.secretName
value: test-admin-kubeconfig
- equal:
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.optional
value: true
- it: cluster-autoscaler waits for admin-kubeconfig via initContainer
template: templates/cluster-autoscaler/deployment.yaml
documentSelector:
path: kind
value: Deployment
asserts:
- equal:
path: spec.template.spec.initContainers[0].name
value: wait-for-kubeconfig
- contains:
path: spec.template.spec.initContainers[0].volumeMounts
content:
name: kubeconfig
mountPath: /etc/kubernetes/kubeconfig
readOnly: true
- it: kccm mounts admin-kubeconfig as optional
template: templates/kccm/manager.yaml
documentSelector:
path: kind
value: Deployment
asserts:
- equal:
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.secretName
value: test-admin-kubeconfig
- equal:
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.optional
value: true
- it: kccm waits for admin-kubeconfig via initContainer
template: templates/kccm/manager.yaml
documentSelector:
path: kind
value: Deployment
asserts:
- equal:
path: spec.template.spec.initContainers[0].name
value: wait-for-kubeconfig
- contains:
path: spec.template.spec.initContainers[0].volumeMounts
content:
name: kubeconfig
mountPath: /etc/kubernetes/kubeconfig
readOnly: true
- it: csi controller mounts admin-kubeconfig as optional
template: templates/csi/deploy.yaml
documentSelector:
path: kind
value: Deployment
asserts:
- equal:
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.secretName
value: test-admin-kubeconfig
- equal:
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.optional
value: true
- it: csi controller waits for admin-kubeconfig via initContainer
template: templates/csi/deploy.yaml
documentSelector:
path: kind
value: Deployment
asserts:
- equal:
path: spec.template.spec.initContainers[0].name
value: wait-for-kubeconfig
- contains:
path: spec.template.spec.initContainers[0].volumeMounts
content:
name: kubeconfig
mountPath: /etc/kubernetes/kubeconfig
readOnly: true
- it: wait-for-kubeconfig defaults to bundled busybox digest pin
template: templates/cluster-autoscaler/deployment.yaml
documentSelector:
path: kind
value: Deployment
asserts:
- matchRegex:
path: spec.template.spec.initContainers[0].image
pattern: '^docker\.io/library/busybox:[^@]+@sha256:[0-9a-f]{64}$'
- it: wait-for-kubeconfig honours images.waitForKubeconfig override
template: templates/cluster-autoscaler/deployment.yaml
documentSelector:
path: kind
value: Deployment
set:
images:
waitForKubeconfig: "registry.example.test/library/busybox:1.37.0@sha256:0000000000000000000000000000000000000000000000000000000000000000"
asserts:
- equal:
path: spec.template.spec.initContainers[0].image
value: "registry.example.test/library/busybox:1.37.0@sha256:0000000000000000000000000000000000000000000000000000000000000000"
- it: cluster.yaml renders and wires dataStoreName when tenant has etcd
template: templates/cluster.yaml
documentSelector:
path: kind
value: KamajiControlPlane
asserts:
- equal:
path: spec.dataStoreName
value: tenant-root
- it: cluster.yaml skips Cluster resources when tenant has no etcd DataStore
# Must NOT fail rendering - the parent Tenant chart populates
# _namespace.etcd asynchronously, so rendering failures here would cause
# flux install remediation on every cold bootstrap. Instead, emit only a
# ConfigMap status beacon so the HelmRelease reports Ready while flux
# retries on its interval until the DataStore appears.
template: templates/cluster.yaml
set:
_namespace:
etcd: ""
monitoring: ""
ingress: ""
seaweedfs: ""
host: ""
asserts:
- hasDocuments:
count: 1
- isKind:
of: ConfigMap
- equal:
path: metadata.name
value: test-awaiting-etcd
- equal:
path: data.status
value: awaiting-etcd

View file

@ -1,99 +0,0 @@
suite: GPU Operator HelmRelease HAMi integration tests
templates:
- templates/helmreleases/gpu-operator.yaml
values:
- values-ci.yaml
tests:
- it: should disable devicePlugin when hami is enabled
set:
addons:
gpuOperator:
enabled: true
valuesOverride: {}
hami:
enabled: true
valuesOverride: {}
asserts:
- equal:
path: spec.values.gpu-operator.devicePlugin.enabled
value: false
- it: should not have values when hami is disabled and no overrides
set:
addons:
gpuOperator:
enabled: true
valuesOverride: {}
hami:
enabled: false
valuesOverride: {}
asserts:
- notExists:
path: spec.values
- it: should apply hami defaults when valuesOverride key is omitted
set:
addons:
gpuOperator:
enabled: true
hami:
enabled: true
asserts:
- equal:
path: spec.values.gpu-operator.devicePlugin.enabled
value: false
- it: should allow user overrides to merge with hami defaults
set:
addons:
gpuOperator:
enabled: true
valuesOverride:
gpu-operator:
driver:
enabled: false
hami:
enabled: true
valuesOverride: {}
asserts:
- equal:
path: spec.values.gpu-operator.devicePlugin.enabled
value: false
- equal:
path: spec.values.gpu-operator.driver.enabled
value: false
- it: should let user explicitly override devicePlugin.enabled to true with hami enabled
set:
addons:
gpuOperator:
enabled: true
valuesOverride:
gpu-operator:
devicePlugin:
enabled: true
driver:
enabled: false
hami:
enabled: true
valuesOverride: {}
asserts:
- equal:
path: spec.values.gpu-operator.devicePlugin.enabled
value: true
- equal:
path: spec.values.gpu-operator.driver.enabled
value: false
- it: should not render when gpuOperator is disabled
set:
addons:
gpuOperator:
enabled: false
valuesOverride: {}
hami:
enabled: false
valuesOverride: {}
asserts:
- hasDocuments:
count: 0

View file

@ -1,153 +0,0 @@
suite: HAMi HelmRelease tests
templates:
- templates/helmreleases/hami.yaml
values:
- values-ci.yaml
tests:
- it: should not render when hami is disabled
set:
addons:
hami:
enabled: false
valuesOverride: {}
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- hasDocuments:
count: 0
- it: should render HelmRelease when hami is enabled
set:
addons:
hami:
enabled: true
valuesOverride: {}
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- hasDocuments:
count: 1
- isKind:
of: HelmRelease
- it: should fail when gpuOperator is not enabled
set:
addons:
hami:
enabled: true
valuesOverride: {}
gpuOperator:
enabled: false
valuesOverride: {}
asserts:
- failedTemplate:
errorMessage: "addons.hami requires addons.gpuOperator to be enabled"
- it: should have correct metadata labels
set:
addons:
hami:
enabled: true
valuesOverride: {}
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- equal:
path: metadata.labels["cozystack.io/repository"]
value: system
- equal:
path: metadata.labels["sharding.fluxcd.io/key"]
value: tenants
- it: should use ExternalArtifact chartRef
set:
addons:
hami:
enabled: true
valuesOverride: {}
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- equal:
path: spec.chartRef.kind
value: ExternalArtifact
- equal:
path: spec.chartRef.namespace
value: cozy-system
- it: should target cozy-hami namespace
set:
addons:
hami:
enabled: true
valuesOverride: {}
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- equal:
path: spec.targetNamespace
value: cozy-hami
- equal:
path: spec.storageNamespace
value: cozy-hami
- it: should depend on gpu-operator and cilium
release:
name: test
namespace: test-ns
set:
addons:
hami:
enabled: true
valuesOverride: {}
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- contains:
path: spec.dependsOn
content:
name: test-cilium
namespace: test-ns
- contains:
path: spec.dependsOn
content:
name: test-gpu-operator
namespace: test-ns
- it: should not render spec.values when valuesOverride is empty
set:
addons:
hami:
enabled: true
valuesOverride: {}
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- hasDocuments:
count: 1
- notExists:
path: spec.values
- it: should pass through valuesOverride
set:
addons:
hami:
enabled: true
valuesOverride:
hami:
devicePlugin:
deviceSplitCount: 5
gpuOperator:
enabled: true
valuesOverride: {}
asserts:
- equal:
path: spec.values.hami.devicePlugin.deviceSplitCount
value: 5

View file

@ -1,9 +0,0 @@
_namespace:
etcd: ""
monitoring: ""
ingress: ""
seaweedfs: ""
host: ""
_cluster:
cluster-domain: cozy.local
nodeGroups: null

View file

@ -1,9 +0,0 @@
_namespace:
etcd: tenant-root
monitoring: ""
ingress: ""
seaweedfs: ""
host: ""
_cluster:
cluster-domain: cozy.local
nodeGroups: null

View file

@ -149,7 +149,6 @@
"fluxcd",
"gatewayAPI",
"gpuOperator",
"hami",
"ingressNginx",
"monitoringAgents",
"velero",
@ -269,28 +268,6 @@
}
}
},
"hami": {
"description": "HAMi GPU virtualization middleware.",
"type": "object",
"default": {},
"required": [
"enabled",
"valuesOverride"
],
"properties": {
"enabled": {
"description": "Enable HAMi (requires GPU Operator).",
"type": "boolean",
"default": false
},
"valuesOverride": {
"description": "Custom Helm values overrides.",
"type": "object",
"default": {},
"x-kubernetes-preserve-unknown-fields": true
}
}
},
"ingressNginx": {
"description": "Ingress-NGINX controller.",
"type": "object",
@ -653,18 +630,6 @@
}
}
}
},
"images": {
"description": "Optional image overrides for air-gapped or rate-limited registries.",
"type": "object",
"default": {},
"properties": {
"waitForKubeconfig": {
"description": "Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag.",
"type": "string",
"default": ""
}
}
}
}
}

View file

@ -94,10 +94,6 @@ host: ""
## @field {bool} enabled - Enable FluxCD.
## @field {object} valuesOverride - Custom Helm values overrides.
## @typedef {struct} HAMiAddon - HAMi GPU virtualization middleware.
## @field {bool} enabled - Enable HAMi (requires GPU Operator).
## @field {object} valuesOverride - Custom Helm values overrides.
## @typedef {struct} MonitoringAgentsAddon - Monitoring agents (Fluent Bit, VMAgents).
## @field {bool} enabled - Enable monitoring agents.
## @field {object} valuesOverride - Custom Helm values overrides.
@ -118,7 +114,6 @@ host: ""
## @field {GatewayAPIAddon} gatewayAPI - Gateway API addon.
## @field {IngressNginxAddon} ingressNginx - Ingress-NGINX controller.
## @field {GPUOperatorAddon} gpuOperator - NVIDIA GPU Operator.
## @field {HAMiAddon} hami - HAMi GPU virtualization middleware.
## @field {FluxCDAddon} fluxcd - FluxCD GitOps operator.
## @field {MonitoringAgentsAddon} monitoringAgents - Monitoring agents.
## @field {VerticalPodAutoscalerAddon} verticalPodAutoscaler - Vertical Pod Autoscaler.
@ -142,9 +137,6 @@ addons:
gpuOperator:
enabled: false
valuesOverride: {}
hami:
enabled: false
valuesOverride: {}
fluxcd:
enabled: false
valuesOverride: {}
@ -205,10 +197,3 @@ controlPlane:
server:
resources: {}
resourcesPreset: "micro"
## @typedef {struct} Images - Optional image overrides for chart-internal helpers.
## @field {string} [waitForKubeconfig] - Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag.
## @param {Images} images - Optional image overrides for air-gapped or rate-limited registries.
images:
waitForKubeconfig: ""

View file

@ -133,13 +133,12 @@ See:
### Bootstrap (recovery) parameters
| Name | Description | Type | Value |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------- |
| `bootstrap` | Bootstrap configuration. | `object` | `{}` |
| `bootstrap.enabled` | Whether to restore from a backup. | `bool` | `false` |
| `bootstrap.recoveryTime` | Timestamp (RFC3339) for point-in-time recovery; empty means latest. | `string` | `""` |
| `bootstrap.oldName` | Previous cluster name before deletion. | `string` | `""` |
| `bootstrap.serverName` | Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name. | `string` | `""` |
| Name | Description | Type | Value |
| ------------------------ | ------------------------------------------------------------------- | -------- | ------- |
| `bootstrap` | Bootstrap configuration. | `object` | `{}` |
| `bootstrap.enabled` | Whether to restore from a backup. | `bool` | `false` |
| `bootstrap.recoveryTime` | Timestamp (RFC3339) for point-in-time recovery; empty means latest. | `string` | `""` |
| `bootstrap.oldName` | Previous cluster name before deletion. | `string` | `""` |
## Parameter examples and reference

View file

@ -32,9 +32,6 @@ spec:
- name: {{ .Values.bootstrap.oldName }}
barmanObjectStore:
destinationPath: {{ .Values.backup.destinationPath }}
{{- if .Values.bootstrap.serverName }}
serverName: {{ .Values.bootstrap.serverName }}
{{- end }}
endpointURL: {{ .Values.backup.endpointURL }}
s3Credentials:
accessKeyId:

View file

@ -254,11 +254,6 @@
"description": "Timestamp (RFC3339) for point-in-time recovery; empty means latest.",
"type": "string",
"default": ""
},
"serverName": {
"description": "Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name.",
"type": "string",
"default": ""
}
}
}

View file

@ -154,7 +154,6 @@ backup:
## @field {bool} enabled - Whether to restore from a backup.
## @field {string} [recoveryTime] - Timestamp (RFC3339) for point-in-time recovery; empty means latest.
## @field {string} oldName - Previous cluster name before deletion.
## @field {string} [serverName] - Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name.
## @param {Bootstrap} bootstrap - Bootstrap configuration.
bootstrap:
@ -162,4 +161,3 @@ bootstrap:
# example: 2020-11-26 15:22:00.00000+00
recoveryTime: ""
oldName: ""
serverName: ""

View file

@ -80,6 +80,7 @@ tenant-u1
| `etcd` | Deploy own Etcd cluster. | `bool` | `false` |
| `monitoring` | Deploy own Monitoring Stack. | `bool` | `false` |
| `ingress` | Deploy own Ingress Controller. | `bool` | `false` |
| `gateway` | Deploy own Gateway API Gateway (backed by Cilium Gateway API controller). | `bool` | `false` |
| `seaweedfs` | Deploy own SeaweedFS. | `bool` | `false` |
| `schedulingClass` | The name of a SchedulingClass CR to apply scheduling constraints for this tenant's workloads. | `string` | `""` |
| `resourceQuotas` | Define resource quotas for the tenant. | `map[string]quantity` | `{}` |

View file

@ -12,7 +12,11 @@
(eq $exposeMode "loadBalancer")
(eq $exposeIngress .Release.Namespace)
.Values.ingress }}
{{- if and $isPublishingIngressLB $ipsList }}
{{- $hasOwnGateway := .Values.gateway | default false }}
{{- if and $hasOwnGateway (not $ipsList) }}
{{- fail "publishing.externalIPs is empty — a tenant Gateway has no CiliumLoadBalancerIPPool to draw from. The Gateway Service would stay in <pending> with no address. Set publishing.externalIPs on the platform (or publishing.exposure=loadBalancer with a BGP / L2-announce pool you manage separately) before enabling tenant.spec.gateway=true." }}
{{- end }}
{{- if and (or $isPublishingIngressLB $hasOwnGateway) $ipsList }}
apiVersion: cilium.io/v2
kind: CiliumLoadBalancerIPPool
metadata:

View file

@ -0,0 +1,32 @@
{{- if .Values.gateway }}
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: gateway
namespace: {{ include "tenant.name" . }}
labels:
sharding.fluxcd.io/key: tenants
internal.cozystack.io/tenantmodule: "true"
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
apps.cozystack.io/application.kind: Gateway
apps.cozystack.io/application.group: apps.cozystack.io
apps.cozystack.io/application.name: gateway
spec:
chartRef:
kind: ExternalArtifact
name: cozystack-gateway-application-default-gateway
namespace: cozy-system
interval: 5m
timeout: 10m
install:
remediation:
retries: -1
upgrade:
force: true
remediation:
retries: -1
valuesFrom:
- kind: Secret
name: cozystack-values
{{- end }}

View file

@ -29,6 +29,23 @@
{{- $ingress = $tenantName }}
{{- end }}
{{/*
Gateway reference is NOT inherited from parent. Unlike ingress (where nginx
serves from the publishing namespace and any tenant can attach via the shared
apex), a Gateway's allowedRoutes.namespaces selector is restrictive: a child
tenant namespace is not on the parent Gateway's whitelist, so an HTTPRoute
from the child would be NotAllowedByListeners. The parent Gateway's cert also
only covers <apex> and *.<apex> (one level), and a child apex is
<name>.<parent-apex> — two levels deep — so even if admission let the route
through, TLS would break. Require explicit tenant.spec.gateway=true on each
tenant that wants Gateway-attached apps; otherwise apps fall back to the
inherited Ingress.
*/}}
{{- $gateway := "" }}
{{- if .Values.gateway }}
{{- $gateway = $tenantName }}
{{- end }}
{{- $monitoring := $parentNamespace.monitoring | default "" }}
{{- if .Values.monitoring }}
{{- $monitoring = $tenantName }}
@ -61,6 +78,7 @@ metadata:
{{/* Labels for network policies */}}
namespace.cozystack.io/etcd: {{ $etcd | quote }}
namespace.cozystack.io/ingress: {{ $ingress | quote }}
namespace.cozystack.io/gateway: {{ $gateway | quote }}
namespace.cozystack.io/monitoring: {{ $monitoring | quote }}
namespace.cozystack.io/seaweedfs: {{ $seaweedfs | quote }}
namespace.cozystack.io/host: {{ $computedHost | quote }}
@ -92,6 +110,7 @@ stringData:
_namespace:
etcd: {{ $etcd | quote }}
ingress: {{ $ingress | quote }}
gateway: {{ $gateway | quote }}
monitoring: {{ $monitoring | quote }}
seaweedfs: {{ $seaweedfs | quote }}
host: {{ $computedHost | quote }}

View file

@ -139,3 +139,78 @@ tests:
asserts:
- hasDocuments:
count: 0
- it: gateway=true in any tenant renders pool with namespace-only selector
set:
gateway: true
_cluster:
expose-ingress: tenant-root
expose-external-ips: "192.0.2.10"
asserts:
- hasDocuments:
count: 1
- equal:
path: metadata.name
value: root-exposure
- equal:
path: spec.blocks
value:
- cidr: 192.0.2.10/32
- equal:
path: spec.serviceSelector.matchLabels["io.kubernetes.service.namespace"]
value: tenant-root
- it: gateway=true in a non-publishing tenant renders its own pool (not tied to publishing tenant)
set:
gateway: true
_cluster:
expose-ingress: tenant-root
expose-external-ips: "192.0.2.10"
release:
name: tenant-u1
namespace: tenant-u1
asserts:
- hasDocuments:
count: 1
- equal:
path: metadata.name
value: u1-exposure
- equal:
path: spec.serviceSelector.matchLabels["io.kubernetes.service.namespace"]
value: tenant-u1
- it: gateway=true + ingress=true + loadBalancer mode renders a single shared pool (no overlapping CIDR conflict)
set:
gateway: true
ingress: true
_cluster:
expose-ingress: tenant-root
expose-external-ips: "192.0.2.10"
expose-mode: loadBalancer
asserts:
- hasDocuments:
count: 1
- equal:
path: metadata.name
value: root-exposure
- it: gateway=true with ingress=false still renders pool (gateway alone is enough)
set:
gateway: true
ingress: false
_cluster:
expose-ingress: tenant-root
expose-external-ips: "192.0.2.10"
asserts:
- hasDocuments:
count: 1
- it: gateway=true with empty externalIPs fails chart render with explicit message
set:
gateway: true
_cluster:
expose-ingress: tenant-root
expose-external-ips: ""
asserts:
- failedTemplate:
errorMessage: "publishing.externalIPs is empty — a tenant Gateway has no CiliumLoadBalancerIPPool to draw from. The Gateway Service would stay in <pending> with no address. Set publishing.externalIPs on the platform (or publishing.exposure=loadBalancer with a BGP / L2-announce pool you manage separately) before enabling tenant.spec.gateway=true."

View file

@ -22,6 +22,11 @@
"type": "boolean",
"default": false
},
"gateway": {
"description": "Deploy own Gateway API Gateway (backed by Cilium Gateway API controller).",
"type": "boolean",
"default": false
},
"seaweedfs": {
"description": "Deploy own SeaweedFS.",
"type": "boolean",

View file

@ -14,6 +14,9 @@ monitoring: false
## @param {bool} ingress - Deploy own Ingress Controller.
ingress: false
## @param {bool} gateway - Deploy own Gateway API Gateway (backed by Cilium Gateway API controller).
gateway: false
## @param {bool} seaweedfs - Deploy own SeaweedFS.
seaweedfs: false

View file

@ -36,32 +36,31 @@ virtctl ssh <user>@<vm>
### Common parameters
| Name | Description | Type | Value |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----------- |
| `external` | Enable external access from outside the cluster. | `bool` | `false` |
| `externalMethod` | Method to pass through traffic to the VM. | `string` | `PortList` |
| `externalPorts` | Ports to forward from outside the cluster. | `[]int` | `[22]` |
| `externalAllowICMP` | Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect. | `bool` | `true` |
| `runStrategy` | Requested running state of the VirtualMachineInstance | `string` | `Always` |
| `instanceType` | Virtual Machine instance type. | `string` | `u1.medium` |
| `instanceProfile` | Virtual Machine preferences profile. | `string` | `ubuntu` |
| `disks` | List of disks to attach. | `[]object` | `[]` |
| `disks[i].name` | Disk name. | `string` | `""` |
| `disks[i].bus` | Disk bus type (e.g. "sata"). | `string` | `""` |
| `networks` | Networks to attach the VM to. | `[]object` | `[]` |
| `networks[i].name` | Network attachment name. | `string` | `""` |
| `subnets` | Deprecated: use networks instead. | `[]object` | `[]` |
| `subnets[i].name` | Network attachment name. | `string` | `""` |
| `gpus` | List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM). | `[]object` | `[]` |
| `gpus[i].name` | The name of the GPU resource to attach. | `string` | `""` |
| `cpuModel` | Model specifies the CPU model inside the VMI. List of available models https://github.com/libvirt/libvirt/tree/master/src/cpu_map | `string` | `""` |
| `resources` | Resource configuration for the virtual machine. | `object` | `{}` |
| `resources.cpu` | Number of CPU cores allocated. | `quantity` | `""` |
| `resources.memory` | Amount of memory allocated. | `quantity` | `""` |
| `resources.sockets` | Number of CPU sockets (vCPU topology). | `quantity` | `""` |
| `sshKeys` | List of SSH public keys for authentication. | `[]string` | `[]` |
| `cloudInit` | Cloud-init user data. | `string` | `""` |
| `cloudInitSeed` | Seed string to generate SMBIOS UUID for the VM. | `string` | `""` |
| Name | Description | Type | Value |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----------- |
| `external` | Enable external access from outside the cluster. | `bool` | `false` |
| `externalMethod` | Method to pass through traffic to the VM. | `string` | `PortList` |
| `externalPorts` | Ports to forward from outside the cluster. | `[]int` | `[22]` |
| `runStrategy` | Requested running state of the VirtualMachineInstance | `string` | `Always` |
| `instanceType` | Virtual Machine instance type. | `string` | `u1.medium` |
| `instanceProfile` | Virtual Machine preferences profile. | `string` | `ubuntu` |
| `disks` | List of disks to attach. | `[]object` | `[]` |
| `disks[i].name` | Disk name. | `string` | `""` |
| `disks[i].bus` | Disk bus type (e.g. "sata"). | `string` | `""` |
| `networks` | Networks to attach the VM to. | `[]object` | `[]` |
| `networks[i].name` | Network attachment name. | `string` | `""` |
| `subnets` | Deprecated: use networks instead. | `[]object` | `[]` |
| `subnets[i].name` | Network attachment name. | `string` | `""` |
| `gpus` | List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM). | `[]object` | `[]` |
| `gpus[i].name` | The name of the GPU resource to attach. | `string` | `""` |
| `cpuModel` | Model specifies the CPU model inside the VMI. List of available models https://github.com/libvirt/libvirt/tree/master/src/cpu_map | `string` | `""` |
| `resources` | Resource configuration for the virtual machine. | `object` | `{}` |
| `resources.cpu` | Number of CPU cores allocated. | `quantity` | `""` |
| `resources.memory` | Amount of memory allocated. | `quantity` | `""` |
| `resources.sockets` | Number of CPU sockets (vCPU topology). | `quantity` | `""` |
| `sshKeys` | List of SSH public keys for authentication. | `[]string` | `[]` |
| `cloudInit` | Cloud-init user data. | `string` | `""` |
| `cloudInitSeed` | Seed string to generate SMBIOS UUID for the VM. | `string` | `""` |
## U Series

View file

@ -9,10 +9,7 @@ metadata:
{{- if .Values.external }}
service.kubernetes.io/service-proxy-name: "cozy-proxy"
annotations:
networking.cozystack.io/wholeIP: {{ ternary "true" "false" (eq .Values.externalMethod "WholeIP") | quote }}
{{- if eq .Values.externalMethod "PortList" }}
networking.cozystack.io/allowICMP: {{ ternary "true" "false" (ne .Values.externalAllowICMP false) | quote }}
{{- end }}
networking.cozystack.io/wholeIP: "true"
{{- end }}
spec:
type: {{ ternary "LoadBalancer" "ClusterIP" .Values.external }}

View file

@ -26,11 +26,6 @@
"type": "integer"
}
},
"externalAllowICMP": {
"description": "Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect.",
"type": "boolean",
"default": true
},
"runStrategy": {
"description": "Requested running state of the VirtualMachineInstance",
"type": "string",

View file

@ -31,9 +31,6 @@ externalMethod: PortList
externalPorts:
- 22
## @param {bool} externalAllowICMP - Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect.
externalAllowICMP: true
## @enum {string} RunStrategy - Requested running state of the VirtualMachineInstance
## @value Always - VMI should always be running
## @value Halted - VMI should never be running

View file

@ -13,6 +13,7 @@ spec:
- name: default
dependsOn:
- cozystack.networking
- cozystack.gateway-api-crds
components:
- name: cert-manager-crds
path: system/cert-manager-crds

View file

@ -2,7 +2,7 @@
apiVersion: cozystack.io/v1alpha1
kind: PackageSource
metadata:
name: cozystack.hami
name: cozystack.gateway-api-crds
spec:
sourceRef:
kind: OCIRepository
@ -11,14 +11,10 @@ spec:
path: /
variants:
- name: default
dependsOn:
- cozystack.gpu-operator
dependsOn: []
components:
- name: hami
path: system/hami
valuesFiles:
- values.yaml
- name: gateway-api-crds
path: system/gateway-api-crds
install:
privileged: true
namespace: cozy-hami
releaseName: hami
namespace: cozy-gateway-api
releaseName: gateway-api-crds

View file

@ -0,0 +1,23 @@
---
apiVersion: cozystack.io/v1alpha1
kind: PackageSource
metadata:
name: cozystack.gateway-application
spec:
sourceRef:
kind: OCIRepository
name: cozystack-packages
namespace: cozy-system
path: /
variants:
- name: default
dependsOn:
- cozystack.gateway-api-crds
- cozystack.cert-manager
libraries:
- name: cozy-lib
path: library/cozy-lib
components:
- name: gateway
path: extra/gateway
libraries: ["cozy-lib"]

View file

@ -52,8 +52,6 @@ spec:
path: system/cilium
- name: kubernetes-gpu-operator
path: system/gpu-operator
- name: kubernetes-hami
path: system/hami
- name: kubernetes-vertical-pod-autoscaler
path: system/vertical-pod-autoscaler
- name: kubernetes-prometheus-operator-crds

View file

@ -13,7 +13,8 @@ spec:
- name: noop
components: []
- name: cilium
dependsOn: []
dependsOn:
- cozystack.gateway-api-crds
components:
- name: cilium
path: system/cilium
@ -34,7 +35,8 @@ spec:
dependsOn:
- cilium
- name: cilium-kilo
dependsOn: []
dependsOn:
- cozystack.gateway-api-crds
components:
- name: cilium
path: system/cilium
@ -60,7 +62,8 @@ spec:
- cilium
# Generic Cilium variant for non-Talos clusters (kubeadm, k3s, RKE2, etc.)
- name: cilium-generic
dependsOn: []
dependsOn:
- cozystack.gateway-api-crds
components:
- name: cilium
path: system/cilium
@ -81,7 +84,8 @@ spec:
dependsOn:
- cilium
- name: kubeovn-cilium
dependsOn: []
dependsOn:
- cozystack.gateway-api-crds
components:
- name: cilium
path: system/cilium
@ -112,7 +116,8 @@ spec:
- cilium
# Generic KubeOVN+Cilium variant for non-Talos clusters (kubeadm, k3s, RKE2, etc.)
- name: kubeovn-cilium-generic
dependsOn: []
dependsOn:
- cozystack.gateway-api-crds
components:
- name: cilium
path: system/cilium

View file

@ -31,6 +31,8 @@ stringData:
expose-ingress: {{ .Values.publishing.ingressName | quote }}
expose-external-ips: {{ .Values.publishing.externalIPs | join "," | quote }}
expose-mode: {{ .Values.publishing.exposure | default "externalIPs" | quote }}
gateway-enabled: {{ .Values.gateway.enabled | default false | quote }}
gateway-attached-namespaces: {{ .Values.gateway.attachedNamespaces | default (list) | join "," | quote }}
cluster-domain: {{ .Values.networking.clusterDomain | quote }}
api-server-endpoint: {{ .Values.publishing.apiServerEndpoint | quote }}
{{- with .Values.branding }}

View file

@ -10,7 +10,6 @@
{{include "cozystack.platform.package.default" (list "cozystack.kubevirt-cdi" $) }}
{{include "cozystack.platform.package.optional.default" (list "cozystack.vm-default-images" $) }}
{{include "cozystack.platform.package.optional.default" (list "cozystack.gpu-operator" $) }}
{{include "cozystack.platform.package.optional.default" (list "cozystack.hami" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.kamaji" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.capi-operator" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.capi-provider-bootstrap-kubeadm" $) }}

View file

@ -105,6 +105,9 @@
{{- /* cozystack-api DaemonSet */ -}}
{{- $apiValues := dict "cozystackAPI" (dict "nodeSelector" $genericNodeSelector) -}}
{{- $_ := set $cozystackEngineComponents "cozystack-api" (dict "values" $apiValues) -}}
{{- /* lineage-controller-webhook DaemonSet */ -}}
{{- $lineageValues := dict "lineageControllerWebhook" (dict "nodeSelector" $genericNodeSelector) -}}
{{- $_ := set $cozystackEngineComponents "lineage-controller-webhook" (dict "values" $lineageValues) -}}
{{- end -}}
{{- if .Values.authentication.oidc.enabled }}
{{include "cozystack.platform.package" (list "cozystack.cozystack-engine" "oidc" $ $cozystackEngineComponents) }}
@ -115,6 +118,7 @@
{{- end }}
# Common Packages
{{include "cozystack.platform.package.default" (list "cozystack.gateway-api-crds" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.cert-manager" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.flux-plunger" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.victoria-metrics-operator" $) }}
@ -123,6 +127,7 @@
{{- $_ := set $tenantComponents "tenant" (dict "values" $tenantClusterValues) }}
{{include "cozystack.platform.package" (list "cozystack.tenant-application" "default" $ $tenantComponents) }}
{{include "cozystack.platform.package.default" (list "cozystack.ingress-application" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.gateway-application" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.seaweedfs-application" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.info-application" $) }}
{{include "cozystack.platform.package.default" (list "cozystack.monitoring-application" $) }}

View file

@ -83,6 +83,15 @@ publishing:
certificates:
solver: http01 # "http01" or "dns01"
issuerName: letsencrypt-prod
# Rate-limit note: every tenant with gateway=true issues one
# Certificate for its wildcard + apex hostnames. If many tenants share
# the same apex domain with issuerName=letsencrypt-prod, the cluster
# can hit the Let's Encrypt limits (50 certs / registered domain /
# week, 5 duplicate certs / week, 300 new orders / account / 3h).
# Use letsencrypt-stage for non-production clusters or gate the
# number of certificates per tenant via tenant.spec.resourceQuotas
# with count/certificates.cert-manager.io. See
# https://letsencrypt.org/docs/rate-limits/.
# Authentication configuration
authentication:
oidc:
@ -94,6 +103,42 @@ authentication:
# userinfo, logout) through this URL while keeping browser redirects on the external URL.
# Example: http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy
keycloakInternalUrl: ""
# Gateway API configuration
gateway:
# Enable Gateway API support across the platform. When true:
# - cert-manager ClusterIssuers use an http01.gatewayHTTPRoute solver
# that attaches to the root tenant's Gateway, so Certificate issuance
# works for tenants whose ingress is via Gateway API rather than
# ingress-nginx.
# - Exposed services (dashboard, keycloak, harbor, bucket, grafana,
# cozystack-api, vm-exportproxy, cdi-uploadproxy) render an
# HTTPRoute attached to the tenant's Gateway.
# Per-tenant opt-in is still governed by tenant.spec.gateway=true, which
# materialises the Gateway itself. This flag only controls platform-wide
# Gateway API integration (cert-manager solver + per-service HTTPRoutes).
enabled: false
# Namespaces that are allowed to attach HTTPRoute or TLSRoute to a tenant
# Gateway. The publishing tenant namespace is always included implicitly.
# Additional entries must be the exact namespace name (matched on the
# built-in kubernetes.io/metadata.name label). Tenants not on this list
# cannot attach a route to somebody else's Gateway, which prevents
# hostname hijacking across tenants.
#
# The `default` namespace is included because the Kubernetes API's
# TLSRoute (shipped by the cozystack-api package) has to live next to
# the `kubernetes` Service it points at, which is always in `default`.
attachedNamespaces:
- cozy-cert-manager
- cozy-dashboard
- cozy-keycloak
- cozy-system
- cozy-harbor
- cozy-bucket
- cozy-kubevirt
- cozy-kubevirt-cdi
- cozy-monitoring
- cozy-linstor-gui
- default
# Pod scheduling configuration
scheduling:
globalAppTopologySpreadConstraints: ""

View file

@ -5,6 +5,3 @@ include ../../../hack/package.mk
generate:
cozyvalues-gen -v values.yaml -s values.schema.json -r README.md
../../../hack/update-crd.sh
test:
helm unittest .

View file

@ -0,0 +1,39 @@
{{- $shouldUpdateCerts := true }}
{{- $configMap := lookup "v1" "ConfigMap" .Release.Namespace "etcd-deployed-version" }}
{{- if $configMap }}
{{- $deployedVersion := index $configMap "data" "version" }}
{{- if $deployedVersion | semverCompare ">= 2.6.1" }}
{{- $shouldUpdateCerts = false }}
{{- end }}
{{- end }}
{{- if $shouldUpdateCerts }}
---
apiVersion: batch/v1
kind: Job
metadata:
name: etcd-hook
annotations:
helm.sh/hook: post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
spec:
template:
metadata:
labels:
policy.cozystack.io/allow-to-apiserver: "true"
spec:
serviceAccountName: etcd-hook
containers:
- name: kubectl
image: docker.io/alpine/k8s:1.33.4
command:
- sh
args:
- -exc
- |-
kubectl --namespace={{ .Release.Namespace }} delete secrets etcd-ca-tls etcd-peer-ca-tls
sleep 10
kubectl --namespace={{ .Release.Namespace }} delete secrets etcd-client-tls etcd-peer-tls etcd-server-tls
kubectl --namespace={{ .Release.Namespace }} delete pods --selector=app.kubernetes.io/instance=etcd,app.kubernetes.io/managed-by=etcd-operator,app.kubernetes.io/name=etcd,cozystack.io/service=etcd
restartPolicy: Never
{{- end }}

View file

@ -0,0 +1,26 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
annotations:
helm.sh/hook: post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
name: etcd-hook
rules:
- apiGroups:
- ""
resources:
- secrets
- pods
verbs:
- get
- list
- watch
- delete
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- list
- watch

View file

@ -0,0 +1,15 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: etcd-hook
annotations:
helm.sh/hook: post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: etcd-hook
subjects:
- kind: ServiceAccount
name: etcd-hook
namespace: {{ .Release.Namespace | quote }}

View file

@ -0,0 +1,7 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: etcd-hook
annotations:
helm.sh/hook: post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded

View file

@ -0,0 +1,6 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: etcd-deployed-version
data:
version: {{ .Chart.Version }}

Some files were not shown because too many files have changed in this diff Show more