Empty commit to retrigger E2E that hit two known flakes:
- kubernetes-previous (CSINode registration race per
flake-kubernetes-previous-csi memory): nfs-test-pvc timed out waiting
for condition.
- vminstance: VMI did not get IP in 20s timeout (KubeVirt flake).
Neither is related to the admission-chain or VAP changes in this PR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Empty commit to retrigger E2E that flaked on KubeVirt vminstance test
(Create a VM Instance — VMI did not get IP in 20s timeout). Same flake
that hit #2489 earlier in this session; not related to the
admission-chain or VAP changes in this PR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The custom REST handler at pkg/registry/apps/application/rest.go is the
storage backend for every kind in apps.cozystack.io/v1alpha1 (Tenant,
MariaDB, Postgres, Kubernetes, ...). genericapiserver passes the
admission chain — mutating webhooks, validating webhooks, and
ValidatingAdmissionPolicies — to each storage method as a callback:
createValidation for Create, updateValidation for Update,
deleteValidation for Delete. Update was correctly invoking its callback
since 23e399bd, but Create and Delete accepted the parameter and never
called it. Admission was silently bypassed for both verbs on every
apps.cozystack.io resource.
The ValidatingAdmissionPolicy cozystack-tenant-host-policy added in
this PR is the first in-tree policy that depends on Create-side
enforcement. The accompanying e2e probe
'cozystack-tenant-host-policy blocks non-trusted callers from setting
tenant.spec.host' caught the gap.
Mirrors the existing updateValidation pattern: invoke createValidation
in Create after the cheap format/length checks but before any state
change, and invoke deleteValidation in Delete after the HelmRelease has
been resolved (so admission sees the same shape it would on UPDATE) but
before the actual r.c.Delete call.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The gateway.bats e2e test creates a Tenant named 'vap-host-probe' to
verify that a non-trusted ServiceAccount cannot set tenant.spec.host.
Tenant name validation rejects dashes (lowercase letters and digits
only), so the apply was rejected by the name validator before reaching
the cozystack-tenant-host-policy ValidatingAdmissionPolicy. Under the
test's set -e, the subsequent grep for 'ValidatingAdmissionPolicy' in
the error output then exits non-zero, failing the test.
Renaming the probe to 'vaphostprobe' lets the request pass the name
validator and reach the VAP, which is what the test is meant to
exercise.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Empty commit to retrigger Build job that failed on Docker Hub anonymous
pull rate limit (429 Too Many Requests) for alpine:latest.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
#2468 introduced packages/apps/tenant/templates/cilium-lb-pool.yaml as the per-tenant owner of the LoadBalancer IP pool. Extend that template with the .Values.gateway branch: the pool now renders when either _cluster.expose-mode=loadBalancer (with .Values.ingress=true on the publishing tenant) OR .Values.gateway=true. Cilium LB IPAM forbids overlapping CIDRs across pools regardless of serviceSelector, so a single pool per tenant is required — the same CIDRs cannot back both ingress and gateway from two separate pools without one of them being marked Conflicting.
Delete packages/extra/gateway/templates/cilium-lb-pool.yaml — the gateway chart no longer renders its own pool. The empty-externalIPs render-time fail moves to the tenant template, guarded on the gateway branch so the legacy ingress path remains unchanged.
Gateway tests lose the pool-rendering asserts that previously lived in packages/extra/gateway/tests/gateway_test.yaml. The tenant chart tests in packages/apps/tenant/tests/exposure_test.yaml gain five gateway-path cases covering pool rendering, non-publishing-tenant rendering, the simultaneous ingress+gateway single-pool case, gateway-only with ingress=false, and the empty-externalIPs fail.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Previous pattern relied on a "__SUCCEEDED__" marker captured via
`output=$(kubectl apply … && echo "__SUCCEEDED__")`. On dash
(hack/cozytest.sh is #!/bin/sh with set -e) a failing command
substitution propagates its non-zero exit through the assignment and
kills the test before the follow-up grep can run — so admission
correctly rejecting the probe read as a test failure. The marker
approach also hides what actually happened behind a string comparison.
Rewrite each VAP negative-path test as `if ! output=$(…); then …
else …; fi`. The if-condition disables set -e per POSIX, the exit
code is consumed directly (admission reject = happy path), and on the
unexpected-success branch we print the captured admission response
and clean up the resource that slipped through. Happy-path cleanup
only removes the scaffolding RBAC because the probe resource was
never created.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Root cause of the install regression on run 24885268333: the Flux
ServiceAccount in cozystack is `system:serviceaccount:cozy-fluxcd:flux`,
not `flux-system:flux`. My trustedCaller list had
`system:serviceaccounts:flux-system`, so once the namespace-host-label
VAP was broadened to CREATE, Flux's first-time write of
`namespace.cozystack.io/host` on tenant-test got rejected:
namespaces "tenant-test" is forbidden: ValidatingAdmissionPolicy
'cozystack-namespace-host-label-policy' denied request: namespace
label namespace.cozystack.io/host is immutable once set (was ,
requested test.example.org); only cluster-admins and cozystack/Flux
service accounts may change it. Request user:
system:serviceaccount:cozy-fluxcd:flux
Replace `flux-system` with `cozy-fluxcd` in the trustedCaller group
list of both cozystack-tenant-host-policy and
cozystack-namespace-host-label-policy, plus the README.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The policy previously only fired on UPDATE and only when oldObject already
carried namespace.cozystack.io/host, which left two paths unguarded:
- CREATE of a namespace with the label pre-set.
- UPDATE that adds the label for the first time (oldObject has no label,
so the matchCondition was false and the VAP was skipped).
Both paths required namespace create/update on labels, which is normally
cluster-admin-only, but the VAP is meant to be the source of truth for
this label's integrity. Close the gap:
- operations: [CREATE, UPDATE]
- matchCondition now fires when either object or oldObject carries the
label (renamed had-host-label -> touches-host-label).
- oldHost is computed with a null-safe ternary so CREATE (where
oldObject is null) evaluates cleanly.
The existing validation (newHost == oldHost || trustedCaller) then
naturally denies first-time label writes from non-trusted callers while
still allowing cozy-system / cozy-cert-manager / flux-system / kube-system
SAs to stamp the label during the tenant chart apply.
Update packages/extra/gateway/README.md layer 5 description to match and
add an e2e test that asserts a namespace CREATE with the label from an
untrusted SA is rejected.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Observed on run 24880685728: the hostname-policy VAP correctly rejected
the foreign-hostname Gateway (apiserver returned the exact "Gateway
listener hostname must equal test.example.org" message), but the bats
test still failed. Root cause: the guard
echo "$output" | grep -q "__SUCCEEDED__" && { return 1; }
is an AND list at statement level. When the marker is ABSENT (the
intended happy path — admission rejected, no marker captured), grep
exits 1, && short-circuits, the overall compound command returns 1,
and cozytest.sh's set -e kills the test.
Rewrite each of the four VAP negative-path guards as an if-statement:
if echo "$output" | grep -q "__SUCCEEDED__"; then
echo "BUG: …" >&2
return 1
fi
Bash's set -e explicitly ignores non-zero exits from commands inside
if-conditions, so the happy path falls through to the follow-up greps
that verify the expected rejection message.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from coderabbitai on hack/e2e-apps/gateway.bats:170 —
the VAP negative-path tests impersonate
system:serviceaccount:tenant-test:default, but that SA has no RBAC to
create apps.cozystack.io/tenants or to update/patch namespaces.
Authorization runs BEFORE admission, so without those grants the
apiserver returns a plain RBAC Forbidden, the test's grep for
"ValidatingAdmissionPolicy" fails, and the test appears to be a VAP
regression when the VAP is actually fine.
Create an ad-hoc Role + RoleBinding in tenant-test granting create/update
on apps.cozystack.io/tenants to default SA (scoped to tenant-test) for
the tenant-host-policy test, and a ClusterRole + ClusterRoleBinding
granting update/patch on core namespaces for the namespace-host-label
policy test. Both are deleted after the test runs so the sandbox stays
clean for subsequent tests.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address observation from kitsunoff on packages/system/cilium/values.yaml:21 —
envoy.enabled: true turns on a per-node envoy DaemonSet but the upstream
default leaves envoy.resources at {} (no requests, no limits). PR
description cites ~100 MB RAM/node at idle; under L7 traffic that grows
without a cap, and the lack of requests also lets the DaemonSet schedule
onto any node with zero reserved headroom.
Pin the baseline matching the commented-out example in the upstream
Helm values: 100m CPU + 512Mi memory requests, 1Gi memory limit. That
gives the scheduler something to plan around and prevents a runaway
listener from OOM-ing the node without drawing attention.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from coderabbitai on
packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:133 —
the matchCondition required object.spec.host != "", so an UPDATE that
unset or emptied the host field skipped the VAP entirely. A tenant with
empty spec.host inherits the apex from its parent, so clearing a
previously-set host is itself an effective apex change that the policy's
intent already covers. A non-trusted caller with UPDATE permission on
the Tenant could flip spec.host: foo.example.org → "" without ever
hitting the trustedCaller gate.
Rewrite the gate so the VAP fires whenever EITHER side of the update
has a non-empty host, and compute hostChanged over both oldHost and
newHost (each with its own CEL variable and ternary fallback to ""
when the field is absent). Leaves the validation unchanged: the change
is allowed only when hostChanged=false or variables.trustedCaller=true.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/extra/gateway/templates/issuer.yaml:21 — the per-tenant Issuer was
emitting an http01.gatewayHTTPRoute solver while the accompanying
Certificate in gateway.yaml requests both the apex and the *.<apex>
wildcard. Let's Encrypt refuses to issue wildcard certificates through
HTTP-01 (see LE challenge-types docs and the cert-manager CRD schema
note at crd-cert-manager.io_clusterissuers.yaml:728-729). Under the
default publishing.certificates.solver=http01, the Order would loop
forever, the Secret would never be created, and the Gateway HTTPS
listeners would sit without a cert. cert-manager-issuers ClusterIssuers
already branch on _cluster.solver (http01 vs dns01 Cloudflare); the
per-tenant Issuer was ignoring that signal entirely and the E2E flag
that would have exercised the full issuance flow was reverted in
a4bef14, so CI missed the regression.
Fail the render when publishing.certificates.solver=http01 with an
explicit message pointing at the root cause and the dns01/selfsigned
alternatives, and switch the emitted solver to dns01.cloudflare (same
shape cluster-issuers.yaml uses). Add a unit test for the http01-fail
path and update the existing Issuer test asserts to match the dns01
solver shape.
Follow-up: add an E2E assertion that actually issues a Certificate
through the per-tenant Issuer and waits for .status.conditions[Ready]=True
once the gateway.enabled=true install path is re-enabled.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Previous matchConditions on namespaceObject.metadata.labels were evaluating
to false or skipping silently despite the label being present
(verified via diagnostic test on run 24875535058: VAP exists, binding
has validationActions: [Deny], tenant-test namespace has
namespace.cozystack.io/host=test.example.org, yet admission still
accepted a Gateway with listener hostname dashboard.example.org).
Restructure the VAP so matchCondition does NOT touch namespaceObject —
it just checks the Gateway's declared namespace starts with tenant-
via object.metadata.namespace. The namespaceObject-based label lookup
moves into a CEL variable with a ternary that returns "" when the label
is missing, and the validation accepts if tenantHost is empty (skip) or
if every listener hostname matches. This way the gate fires predictably
for every tenant-* Gateway and the namespaceObject access happens in a
context where its shape is easier to reason about.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Run 24870368740 showed the VAP still not rejecting cross-tenant hostname
after the CEL chain fix. Diagnostic test on run 24873181209 confirmed:
1. tenant-test namespace DOES have namespace.cozystack.io/host=test.example.org.
2. admission still CREATES a Gateway with listener hostname dashboard.example.org
(__SUCCEEDED__ marker appeared — kubectl apply returned 0).
So the label is present but the VAP's matchCondition evaluates to false
for some reason, skipping the policy entirely. The prime suspect is
has(namespaceObject.metadata) on a Kubernetes unstructured representation
where the has() macro semantics don't match the typed Namespace resource
my chain fix assumed.
Simplify the matchCondition to use only the in-operator for map-key
presence ("k" in map) — this is the canonical CEL pattern for
K8s-style unstructured labels and does not rely on has() working
consistently across nested selection:
namespaceObject != null &&
"namespace.cozystack.io/host" in namespaceObject.metadata.labels &&
namespaceObject.metadata.labels["..."] != ""
Also add a diagnostic test to hack/e2e-apps/gateway.bats that dumps the
VAP + binding yaml and asserts validationActions includes Deny — so if
anything structural breaks with the VAP installation, the failure is
visible in the first few lines of output.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The VAP 'ValidatingAdmissionPolicy rejects Gateway with foreign hostname'
silently fails to reject when tenant-test namespace lacks the
namespace.cozystack.io/host label — matchCondition returns false, VAP
skips, Gateway is admitted. That's what happened on run 24870368740.
Add a dedicated test BEFORE the VAP tests that asserts tenant-test
actually has the expected host label. If it doesn't, the test fails with
a clear SETUP FAILURE message and dumps the full namespace yaml to CI
logs, so the real root-cause (label missing — tenant chart bug, helm
timing, or namespaceObject representation issue) is visible instead of
being obscured by the downstream VAP test failure.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
hack/cozytest.sh is a pure-shell bats-compat runner, not real bats. The
'run' builtin that bats ships (captures cmd output/status into $output
and $status) is unavailable, so every test using 'run kubectl …' fails
with 'run: not found' as soon as it's exercised. Previous runs didn't
surface this because install-cozystack was failing before the gateway
bats file could execute — now that the install path is green, the VAP
tests actually run and hit this trap.
Rewrite the four VAP negative-path tests to capture kubectl output and
exit status manually. Uses '&& echo __SUCCEEDED__' inside the command
substitution so the happy-path (admission accepted something it
shouldn't) is detectable, and captures stderr via 2>&1 so admission
rejection messages land in $output for grep assertions.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Second CEL bug in the same VAP surfaced in E2E run 24862782568 (gateway
bats test on tenant-test namespace): matchCondition
has(namespaceObject.metadata.labels) errored at admission time with
"no such key: metadata" — apiserver rejected the Gateway with
"ValidatingAdmissionPolicy denied request: expression '...' resulted in
error: no such key: metadata".
In Kubernetes CEL the has() macro checks field presence at the last
selection only — has(a.b.c) requires a.b to already be resolvable.
When the intermediate step is missing or the object is represented as
an unstructured map with a missing key, CEL raises 'no such key' at
evaluation rather than letting has() short-circuit to false.
Chain the checks per Kubernetes CEL docs recommendation (Check if a
field is set → has(a.b) && has(a.b.c) && ...):
namespaceObject != null &&
has(namespaceObject.metadata) &&
has(namespaceObject.metadata.labels) &&
"namespace.cozystack.io/host" in namespaceObject.metadata.labels &&
namespaceObject.metadata.labels["namespace.cozystack.io/host"] != ""
The null-check handles the cluster-scoped case that K8s documents as
'namespaceObject is null when the incoming object is cluster-scoped',
the has() chain handles the unstructured-map case that surfaced in the
gateway-e2e-probe on tenant-test.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The cozystack-tenant-host-policy VAP failed to install because its CEL
variable used has(oldObject), which is not a valid CEL macro: has()
requires a field selection expression like has(x.y), not a bare
identifier check. The apiserver rejected the policy with
"invalid argument to has() macro", Helm install of cozystack-basics
failed, Flux triggered uninstall remediation which deleted the namespaces
created by the chart (cozy-public, tenant-root), and every downstream
HelmRelease that had to create resources in cozy-public or tenant-root
(kubevirt-cdi, the tenant chart, etc.) then failed with
"namespace ... because it is being terminated". Root cause for the
whole install-cozystack E2E regression on this branch.
Replace has(oldObject) with oldObject != null. On CREATE oldObject is
null and the ternary returns true (hostChanged); on UPDATE oldObject is
set and the rest of the condition runs as before.
Found via the cozyreport artifact from run 24858285228 rerun — the
kubectl describe hr cozystack-basics output exposed the VAP compilation
error that the top-level HelmRelease status message truncated with
"1 error occurred: ...".
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/extra/gateway/templates/cilium-lb-pool.yaml:9: the previous
behaviour rendered the tenant Gateway without a matching
CiliumLoadBalancerIPPool when publishing.externalIPs was empty. Cilium LB
IPAM then had no pool to serve the tenant's cilium-gateway-cozystack
Service, so the Service sat in <pending> forever with no address and the
operator got a silently non-working Gateway.
Replace the conditional skip with an explicit helm fail that names the
root cause. Also guard the CIDR suffix append on the absence of "/" to
accept pre-CIDR input (consistent with the same guard already shipped on
packages/extra/ingress/templates/cilium-lb-pool.yaml).
Update the unit test that asserted hasDocuments: count: 0 to assert the
new failed-template error message, and add expose-external-ips to every
other test so they don't accidentally cross the empty-IPs guard.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
hack/e2e-apps/gateway.bats:79: only cozystack-gateway-hostname-policy was
exercised end-to-end. Commit 8770b87 specifically retrofitted
matchConditions onto the other three VAPs to stop failurePolicy: Fail from
turning missing fields into denials, which is a high-regression-risk area,
so each VAP needs its own test.
Add:
- cozystack-gateway-attached-namespaces-policy rejects a Package with a
tenant-* entry in spec.components.platform.values.gateway.attachedNamespaces.
- cozystack-tenant-host-policy rejects a Tenant.spec.host set by an
impersonated tenant-namespace ServiceAccount (non-trusted group list).
- cozystack-namespace-host-label-policy rejects a kubectl label overwrite
on namespace.cozystack.io/host by the same non-trusted SA.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/extra/gateway/README.md:17: the security model section described
only two layers, but packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml
ships four ValidatingAdmissionPolicies plus a render-time fail in
cozystack-basics. Rewrite the section to cover each VAP's scope and what
it guards against, plus the listener allowedRoutes whitelist and the
render-time fail — six layers, each one explained.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/system/cozystack-api/templates/api-tlsroute.yaml:10: the
kubernetes-api TLSRoute lives in the default namespace (it has to sit
next to the kubernetes Service it points at), but default was missing
from the platform's gateway.attachedNamespaces default list. The tenant
Gateway's listener allowedRoutes selector then rejected the TLSRoute,
so Kubernetes API through Gateway API silently failed to work.
Add default to the default whitelist, plus a comment explaining why it
has to be there.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/apps/tenant/templates/namespace.yaml:32: the previous
$gateway := $parentNamespace.gateway | default "" pattern made a child
tenant without gateway: true inherit the parent's gateway namespace into
_namespace.gateway. Harbor and Bucket in that child would then render
HTTPRoutes with parentRefs.namespace pointing at the parent, which fails
in two ways:
1. The parent Gateway's listener allowedRoutes selector whitelists only
the parent namespace plus publishing.gateway.attachedNamespaces (a static
cozy-* list). The child tenant namespace is not there, so the HTTPRoute
ends up NotAllowedByListeners.
2. The parent wildcard Certificate covers <apex> and *.<apex> only. A
child tenant apex is <name>.<parent-apex>, so a harbor hostname would
be harbor.<name>.<parent-apex> — two subdomain levels deep — and no
valid cert would be presented.
Drop the inheritance and require explicit tenant.spec.gateway=true on
each tenant that should host Gateway-attached apps. Tenants without
their own gateway fall back to the inherited ingress reference, which is
the same behaviour the ingress path already has.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/system/bucket/templates/httproute.yaml:9: same HTTP->HTTPS
redirect regression as dashboard's HTTPRoute. Pin sectionName: https so
the bucket UI HTTPRoute stays on the TLS listener and the generic
http-to-https-redirect on port 80 actually fires for cleartext requests.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/apps/harbor/templates/httproute.yaml:10: same HTTP->HTTPS redirect
regression as dashboard's HTTPRoute. Pin sectionName: https so port 80
stays owned by the generic redirect HTTPRoute on the tenant Gateway.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/system/keycloak/templates/httproute.yaml:11: same HTTP->HTTPS
redirect regression as dashboard's HTTPRoute — without sectionName, the
route attached to both listeners and a specific-hostname route on port 80
won the precedence contest against the generic redirect HTTPRoute. Pin
sectionName: https so the keycloak route stays on the TLS-terminating
listener exclusively.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/system/dashboard/templates/httproute.yaml:11: without sectionName,
the HTTPRoute attached to both http:80 and https:443. Under Gateway API
hostname-precedence rules, a route with a specific hostname wins over the
generic http-to-https-redirect HTTPRoute on the http listener, so HTTP
traffic to http://dashboard.example.org/ was proxied directly to the
backend with no TLS — a regression against ingress-nginx behaviour.
Pin sectionName: https so port 80 stays fully owned by the redirect route.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:120:
startsWith("system:serviceaccount:cozy-") without a trailing colon trusted
any ServiceAccount in any namespace whose name begins with cozy- — including
a hypothetical attacker-controlled cozy-evil namespace. Creating such a
namespace requires cluster-admin today, so this is not an active escalation
path, but it is an unnecessarily wide trust boundary for a defense-in-depth
check.
Switch both tenant-host-policy and namespace-host-label-policy to a
group-based check that matches the exact system:serviceaccounts:<ns> group
for cozy-system, cozy-cert-manager, flux-system and kube-system. Every SA
in those namespaces is a member of that group automatically, so there is no
behaviour regression for cozystack's own controllers.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from kitsunoff on
packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:22:
the first VAP had no matchConditions, so with failurePolicy: Fail and a
validation expression of variables.tenantHost != "", any Gateway in a
namespace without the namespace.cozystack.io/host label (kube-system,
default, any non-cozystack namespace) was rejected cluster-wide, including
clusters running with gateway.enabled: false.
Hoist the namespace-label precondition into matchConditions so the VAP only
fires for cozystack-managed tenant namespaces, then simplify the validation
body — the explicit tenantHost-nonempty check becomes redundant.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The bundle file begins with a variant-switching '# Networking'
comment followed by '{{- if eq .Values.bundles.system.variant ...}}'.
The left-strip on that if eats the newline after '# Networking' and
the next emitted '---' from the first package helper ends up
appended to the comment as '# Networking---', which YAML treats as
pure comment (no document separator). That hides the separator
between whatever was emitted before the variant block and the first
Package inside it, so the two documents merge and duplicate-key
errors fire at Flux post-render time.
Pre-existing behaviour was safe only because the variant block
itself produced the first cozystack.io Package, so no previous
Package existed to duplicate keys with. The previous commit put a
brand-new 'cozystack.gateway-api-crds' Package include right before
the '# Networking' comment — that Package's apiVersion then
collided with the networking Package's apiVersion because the
separator between them was consumed by the comment.
Move the gateway-api-crds include down to the 'Common Packages'
section, which has no strip-dash interaction with surrounding
comments. Package ordering in the bundle does not determine install
ordering — Flux honours the dependsOn chain on cozystack.networking
and cozystack.cert-manager, both of which declare a dependsOn on
cozystack.gateway-api-crds.
Verified locally that the rendered platform chart now emits every
Package preceded by '# Source:' comment on its own line, with the
helper-emitted '---' on a dedicated line. 'apiVersion' keys no
longer duplicate within any single YAML document.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Three of the four ValidatingAdmissionPolicies walked deeply into
object fields inside validation CEL expressions. With failurePolicy:
Fail and non-structural CRD schemas (Package, Tenant) the evaluator
could reject the request instead of skipping — which would cascade
into HelmReleases never being produced and the e2e install hanging
on 'waiting for more than 10 HelmReleases'.
Move path existence checks into matchConditions (Kubernetes 1.30+).
matchConditions are evaluated before validations; a false matchCondition
cleanly skips the validation rather than forcing failurePolicy. Switch
to the 'key in map' operator where possible — it is the supported safe
form for Kubernetes JSON without structural schema guarantees.
Policies fixed:
- cozystack-gateway-attached-namespaces-policy: matchCondition
'has-gateway-attached-namespaces' checks every level of the path
spec.components.platform.values.gateway.attachedNamespaces using
'key in map'. Validation runs only on Packages that actually carry
the field.
- cozystack-tenant-host-policy: matchCondition 'has-host-field' skips
Tenants without a non-empty spec.host. The hostChanged variable no
longer needs defensive has() checks.
- cozystack-namespace-host-label-policy: matchCondition 'had-host-label'
skips namespaces that never had the namespace.cozystack.io/host label.
The immutability rule now only fires on true UPDATE-of-set-label.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Security review left two residual trust-assumption gaps:
1. tenant.spec.host is user-controlled. A tenant admin with RBAC
permission to create Tenant CRs could claim an arbitrary host
(dashboard.example.org), have the tenant chart write that host
into their namespace label, and thereby satisfy the per-namespace
host VAP with a hijacked value.
2. namespace.cozystack.io/host is written by cozystack charts but
there is no runtime check that prevents a later update to the
label from an untrusted principal. A tenant with Update on their
own namespace could rewrite the label to whatever they want and
bypass the per-namespace host VAP.
Both close with parallel VAPs. 'Make it impossible — forbid it twice.'
cozystack-tenant-host-policy
matchConstraints: apps.cozystack.io/v1alpha1 tenants, CREATE/UPDATE.
Rejects setting or changing spec.host unless the caller is
cluster-admin (system:masters group) or a cozystack/Flux/kube-system
service account. Tenants are still free to leave spec.host empty
(the normal default) — the tenant chart will compute the
inherited host from the parent namespace. Admins who need to
override do so explicitly via kubectl / cozystack-api.
cozystack-namespace-host-label-policy
matchConstraints: core/v1 namespaces, UPDATE only.
Rejects any change to the namespace.cozystack.io/host label once
it has been set, unless the caller is in the same trusted caller
whitelist. CREATE is unrestricted (initial label write happens
there, by the chart). Result: the label is effectively
write-once-by-cozystack — no tenant can rewrite their own host
label after it is provisioned.
Both policies use failurePolicy: Fail and validationActions: [Deny],
so a missing paramRef or CEL error fails closed.
Combined with the two earlier VAPs (Gateway listener hostname +
Package attachedNamespaces) and the listener allowedRoutes namespace
whitelist, the hijack surface for multi-tenant Gateway API is now
guarded at four independent admission points and one runtime-proxy
allow-list. Bypass requires compromising a trusted service account
or obtaining cluster-admin credentials — at which point Gateway API
isolation is not the weakest link anyway.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replaces the earlier ConfigMap-parameterised hostname VAP with two
independent ValidatingAdmissionPolicies that both enforce at runtime
and do not depend on the publisher's assumption that 'tenant host is
always <name>.<platform apex>'.
1. cozystack-gateway-hostname-policy
Now reads the tenant's real apex from the namespace label
namespace.cozystack.io/host, which the tenant chart already writes
onto every child tenant namespace and which this commit adds to
tenant-root itself. No CRD lookup, no paramRef, no assumption that
the tenant host is a subdomain of the cluster-wide publishing.host.
CEL:
tenantHost := namespaceObject.metadata.labels['namespace.cozystack.io/host']
every listener.hostname must equal tenantHost or endsWith '.' + tenantHost
Consequence:
- tenant-alice with spec.host = alice.example.org gets label
alice.example.org → may publish alice.example.org and
*.alice.example.org, nothing else.
- tenant-corp with spec.host = customer1.io (fully independent
domain, NOT a subdomain of the platform apex) gets label
customer1.io → may publish customer1.io and *.customer1.io.
Previous VAP would have rejected this legitimate case.
- A namespace without the label (no cozystack-managed tenant) falls
through tenantHost == '' → all listener hostnames rejected.
2. cozystack-gateway-attached-namespaces-policy (new)
Runtime counterpart to the render-time 'fail' guard on
_cluster.gateway-attached-namespaces. Watches Package CRs
(cozystack.io/v1alpha1) and rejects any UPDATE that introduces a
tenant-* entry under spec.components.platform.values.gateway.attachedNamespaces.
'kubectl edit packages.cozystack.io cozystack.cozystack-platform'
bypasses the Helm render path; this policy catches it.
CEL walks the array via .all(ns, !ns.startsWith('tenant-')) with
full path presence guards so it ignores packages that do not touch
the gateway block.
3. tenant-root namespace label
cozystack-basics/templates/tenant-root.yaml now writes
labels.namespace.cozystack.io/host = {{ _cluster.root-host }}
so root tenant's Gateway resources get validated through the same
label-based VAP as child tenants. No ConfigMap-driven param, no
split path between root and child.
Both policies use failurePolicy: Fail and validationActions: [Deny].
They install unconditionally (no gateway.enabled gate) — Gateway CRDs
are installed on every bundle variant, so the policies must guard
every path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two correctness fixes that make extra/gateway actually work in a real
cluster, not just in the tests:
1. _cluster.host does not exist
The Secret projected from platform's apps.yaml only exports
_cluster.root-host — the cluster-wide publishing apex (e.g.
example.org). There is no _cluster.host key. The gateway template was
reading _cluster.host, which always evaluated to empty, which always
tripped the 'publishing.host must be set' fail guard. extra/gateway
therefore never rendered in a real cluster; the helm-unittest cases
passed only because they explicitly put a host field under _cluster,
which never happens in production. Switch to _namespace.host — the
per-tenant apex that cozystack-basics (for tenant-root) and the
tenant chart's namespace.yaml (for child tenants) both populate.
2. Guard against 'non-publishing namespace' blocked child tenants
The previous 'if not (eq $exposeIngress .Release.Namespace)' guard
forced extra/gateway to render only in the publishing tenant
namespace. Child tenants with spec.gateway=true deploy extra/gateway
into their own namespace, so the guard always fired and no child
Gateway was ever materialised — despite earlier commits claiming
child-tenant ACME works. The guard is removed; extra/gateway now
renders in whichever namespace it is released into, which matches
the per-tenant HelmRelease template.
Side-effects:
- CiliumLoadBalancerIPPool no longer carries the '== publishing
namespace' condition — each tenant-scoped release produces its
own pool, selecting its own cilium-gateway-cozystack Service by
io.kubernetes.service.namespace. Different pools do not conflict
as Kubernetes resources; IP allocation across pools with
overlapping blocks is an operator concern (document).
- issuer.yaml unchanged — the Issuer's parentRef already had no
namespace, so the solver attaches to the local Gateway regardless
of which tenant namespace we render in.
- A new helm-unittest case exercises the child-tenant path
(tenant-alice with host=alice.example.org): Certificate dnsNames,
Gateway listener hostnames, allowedRoutes whitelist, and
CiliumLoadBalancerIPPool selector all resolve to the child's
apex and namespace.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Remove the child-tenant ACME HTTP-01 entry from Known limitations in
the gateway README. With the per-tenant Issuer added in the previous
commit, each tenant that opts into gateway: true gets its own ACME
account, its own HTTPRoute solver targeting its own Gateway, and no
dependency on the publishing tenant's Gateway.
In its place, document the actual remaining constraint: the
per-tenant Issuer template currently supports two issuer names
(letsencrypt-prod, letsencrypt-stage) — any other value fails the
render with an actionable error. Supporting more ACME providers is
a one-case extension of templates/issuer.yaml, so operators know
exactly where to hook in.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two correctness fixes for multi-tenant Gateway ACME, plus a small
default-tightening.
1. Per-tenant Issuer (child-tenant ACME closes)
packages/extra/gateway/templates/issuer.yaml (new): render a
cert-manager.io/v1 Issuer named 'gateway' in the tenant's own
namespace. It carries its own ACME account (privateKeySecretRef
'gateway-acme-account') and an http01.gatewayHTTPRoute solver whose
parentRef points to the local Gateway 'cozystack' at sectionName http
— no namespace on parentRef, so it attaches in-namespace only.
packages/extra/gateway/templates/gateway.yaml: the wildcard
Certificate now references issuerRef kind: Issuer, name: gateway
(namespace-scoped) instead of the previous cluster-scoped
letsencrypt-prod. The result: every tenant that enables
tenant.spec.gateway=true gets a fully self-contained ACME flow —
its own account, its own HTTPRoute, its own Gateway, no cross-
namespace dependency on the publishing tenant. Child tenants with
gateway: true now work end-to-end for ACME HTTP-01.
The template maps 'publishing.certificates.issuerName' to a
concrete ACME server URL: letsencrypt-prod →
acme-v02.api.letsencrypt.org/directory, letsencrypt-stage →
acme-staging-v02.api.letsencrypt.org/directory. Any other value
fails the render with an explicit error pointing at the file to
extend.
Three new helm-unittest cases cover Certificate→Issuer linkage,
staging-server selection, and the unknown-issuer fail path.
2. Hostname VAP installs unconditionally
packages/system/cozystack-basics/templates/gateway-hostname-policy.yaml:
drop the 'eq gateway-enabled true' guard. The ValidatingAdmissionPolicy
now installs whenever publishing.host is set, regardless of whether
gateway.enabled is flipped. Rationale: Gateway CRDs ship
unconditionally (gateway-api-crds is loaded in every bundle), so a
tenant could create a Gateway manually even with gateway.enabled=false.
The VAP must guard that path too.
3. Restore opt-in default in e2e
hack/e2e-install-cozystack.bats: revert the gateway.enabled: true
line so the default e2e platform configuration matches production
defaults. Gateway API remains strictly opt-in. The
'exposed services render HTTPRoute/TLSRoute but not Ingress' test
(which required platform-wide gateway.enabled=true) is removed; the
VAP cross-tenant reject test stays and now works against the
unconditionally-installed policy.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
hack/e2e-install-cozystack.bats: add 'gateway: {enabled: true}' to the
values used for the cozystack.cozystack-platform Package. The e2e
pipeline therefore exercises the Gateway API path as its primary
integration scenario — dashboard, keycloak render as HTTPRoutes;
cozystack-api, vm-exportproxy, cdi-uploadproxy render as TLSRoutes;
the legacy Ingresses for those services should not exist.
hack/e2e-apps/gateway.bats: two new test cases.
1. 'exposed services render HTTPRoute/TLSRoute but not Ingress when
gateway.enabled=true' — waits for the HTTPRoutes for dashboard and
keycloak to reach Accepted, confirms the three TLSRoutes exist in
their respective namespaces, and asserts (negative tests) that the
old Ingress objects are gone. Catches regressions where a developer
forgets to wrap an Ingress in the gateway-enabled conditional.
2. 'ValidatingAdmissionPolicy rejects Gateway with foreign hostname' —
tries to apply a Gateway in tenant-test that claims
dashboard.example.org (a domain outside the tenant's allowed suffix
test.example.org), asserts kubectl fails with the expected VAP error
and the expected message.
Together these cover the two most important regressions this PR is
shipping against: Ingress leaking through when it should not, and
the hostname-hijacking defence silently disappearing.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
README now shows the exact Tenant manifest snippet for capping the
number of Certificate objects a tenant may create
(count/certificates.cert-manager.io: 10), and calls out that the
default is unlimited so operators on shared-apex multi-tenant clusters
must set this before handing gateway=true to untrusted tenants.
Also rewrites the 'Known limitations' section to reflect what this
PR actually ships: TLS passthrough (cozystack-api, vm-exportproxy,
cdi-uploadproxy) and per-tenant apps (harbor, bucket) are no longer
limitations — they all migrate to Gateway API when gateway.enabled=true.
What's left:
- Child-tenant ACME HTTP-01 still relies on the publishing tenant's
Gateway; a follow-up PR will add namespace-scoped Issuers.
- Upstream application-level gaps (harbor ACL / bucket upstream
features) — carried as upstream PRs, not a cozystack blocker.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Render-time guard at the top of gateway-hostname-policy.yaml walks
_cluster.gateway-attached-namespaces and fails the chart render with
an explicit message if any entry begins with 'tenant-'.
Adding a tenant namespace to the attached whitelist would defeat the
whole point of the whitelist: that tenant could then attach arbitrary
HTTPRoutes to the publishing tenant's Gateway and pick any hostname
inside the apex domain. Catching this at 'helm template' time beats
catching it after the Gateway listener is already programmed and
traffic can flow.
The message spells out the correct recourse — child tenants that
need their own Gateway must flip tenant.spec.gateway=true, which
materialises their own Gateway in their own namespace rather than
borrowing the publishing tenant's.
Applies whether gateway.enabled is true or false — validation on the
attachedNamespaces list is orthogonal to whether the policy itself
is installed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Tenants can now opt their own apps into Gateway API without touching
the publishing tenant. The mechanism mirrors the existing
_namespace.ingress contract exactly.
packages/apps/tenant/templates/namespace.yaml:
- Compute a new $gateway value: if the tenant sets .Values.gateway
it becomes the tenant's own namespace, otherwise it inherits from
the parent namespace (empty means no Gateway anywhere in the chain).
- Ship $gateway out through the cozystack-values Secret under
_namespace.gateway and through a namespace label
namespace.cozystack.io/gateway.
packages/apps/harbor and packages/system/bucket:
- Existing Ingress wrapped in '{{ if not $gateway }}' so it only
renders when no Gateway is attached anywhere up the tenant chain.
- New templates/httproute.yaml render when $gateway is non-empty:
apiVersion gateway.networking.k8s.io/v1 HTTPRoute, parentRef to
the 'cozystack' Gateway in whichever namespace $gateway points
at, hostname <release>.<host> (harbor) or <bucketName>.<host>
(bucket), backendRef to the existing Service on its current port.
Per-tenant resolution: a tenant with .Values.gateway=true gets its
own Gateway in its own namespace, and harbor/bucket deployed inside
that tenant attach to that namespace's Gateway — no cross-namespace
references, no coupling to the publishing tenant.
Default remains ingress-nginx: tenants that do not opt in keep
rendering Ingress verbatim.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Wire the three HTTPS-passthrough services through Gateway API so
ingress-nginx is no longer required for them when gateway.enabled=true.
extra/gateway chart: new values.tlsPassthroughServices (defaults to
[api, vm-exportproxy, cdi-uploadproxy]). For each service in that list
AND in _cluster.expose-services, the Gateway emits an extra
port-443 listener with protocol: TLS, tls.mode: Passthrough, hostname
<svc>.<host>, and allowedRoutes.kinds restricted to TLSRoute. The
specific hostname wins over the wildcard HTTPS Terminate listener at
SNI-matching time per Gateway API spec.
Per-service packages:
- packages/system/cozystack-api: existing Ingress wrapped in
'!= gateway-enabled=true'. New api-tlsroute.yaml renders
apiVersion gateway.networking.k8s.io/v1alpha2 TLSRoute in the
'default' namespace (where the kubernetes Service lives) targeting
the tls-api listener, hostname api.<root-host>, backendRef kubernetes:443.
- packages/system/kubevirt: same treatment for
vm-exportproxy-ingress.yaml → vm-exportproxy-tlsroute.yaml,
hostname vm-exportproxy.<root-host>, backendRef vm-exportproxy:443
in cozy-kubevirt.
- packages/system/kubevirt-cdi: same for cdi-uploadproxy —
hostname cdi-uploadproxy.<root-host>, backendRef cdi-uploadproxy:443
in cozy-kubevirt-cdi.
TLSRoute apiVersion v1alpha2 is intentional: Cilium 1.19 vendors
Gateway API CRDs v1.4 (TLSRoute is v1alpha2 there) and v1.5.1 ships
both v1 and v1alpha2 in the experimental channel, so v1alpha2 is
compatible with both. A follow-up can bump to v1 once Cilium v1.20
(Gateway API v1.5) lands in the platform.
Backward compatibility: default gateway.enabled=false keeps the
existing Ingress path verbatim for all three services.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add operator-facing documentation for the Gateway API feature:
packages/extra/gateway/README.md grows three sections:
- Security model — explains the two layers that protect cross-tenant
isolation (Gateway listener allowedRoutes namespace whitelist +
ValidatingAdmissionPolicy for tenant hostname ownership). Makes it
explicit which namespaces are on the default whitelist so operators
who add a new system component know where to register it.
- Rate limits — spells out the Let's Encrypt quotas (50 certs /
registered domain / week, 5 duplicate certs / week, 300 new orders /
account / 3h), lists mitigations (letsencrypt-stage, resourceQuotas
with count/certificates.cert-manager.io, self-signed ClusterIssuer,
internal ACME).
- Known limitations — TLS passthrough services still use ingress-nginx,
tenant-scoped apps (harbor, bucket) not yet wired to per-tenant
Gateway, child-tenant ACME HTTP-01 needs a namespace-scoped Issuer.
All three are tracked as follow-up work rather than hidden failure
modes.
packages/core/platform/values.yaml adds a comment block under
publishing.certificates.issuerName reminding operators of the
rate-limit consequences before they flip gateway.enabled=true on a
production cluster.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Three bats test cases verify the end-to-end infrastructure brought up
by this PR:
1. Gateway API CRDs are installed (gatewayclasses, gateways, httproutes)
and the 'cilium' GatewayClass is Accepted by the controller —
proves gateway-api-crds installed and cilium.gatewayAPI.enabled
propagated.
2. A minimal Gateway in tenant-test reconciles to Programmed and the
controller materialises its cilium-gateway-<name> LoadBalancer
Service — proves envoy.enabled kicked in and the data-plane wiring
is live.
3. An HTTPRoute with a matching parentRef reaches Accepted status —
proves Cilium's HTTPRoute attachment logic works.
Not covered (because the test harness has no reachable Let's Encrypt
endpoint and no way to flip cluster-wide gateway.enabled mid-pipeline
without trampling other tests):
- cert-manager HTTP-01 solver via gatewayHTTPRoute.
- The ValidatingAdmissionPolicy that enforces tenant hostname
ownership (installed only when gateway.enabled=true at platform
level).
- Full tenant.spec.gateway=true flow with extra/gateway chart and
CiliumLoadBalancerIPPool.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When gateway.enabled is true and publishing.host is set, cozystack-basics
now installs a ValidatingAdmissionPolicy (plus its Binding and a
ConfigMap that carries the apex host and publishing namespace as
parameters) that rejects any Gateway whose listeners reference a
hostname outside the tenant's own domain suffix.
Scheme:
- Publishing tenant namespace ($publishing.ingressName, default
tenant-root): allowed suffix is the platform-wide $publishing.host.
- Any namespace prefixed with 'tenant-': allowed suffix is
<name-stripped-of-tenant-prefix>.$publishing.host.
- Any other namespace: the CEL variable resolves to "" and every
listener with a hostname is rejected — only cluster-admin-managed
namespaces should be creating Gateways anyway.
The VAP is scoped to CREATE and UPDATE on gateway.networking.k8s.io/v1
(and v1beta1) Gateways cluster-wide; the Binding uses
validationActions: [Deny] with parameterNotFoundAction: Deny so that
if the ConfigMap is missing the policy fails safe instead of
open-allowing. messageExpression spells out the tenant-scoped allowed
suffix in the rejection, so operators can see at a glance why the
request was denied.
This closes the hostname-hijacking half of the multi-tenant threat
model on top of the namespace whitelist on Gateway listeners. The two
work in layers: the whitelist stops a non-authorised namespace from
attaching any HTTPRoute; the VAP stops an authorised namespace from
claiming someone else's hostname on its own Gateway.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Each package now ships a conditional templates/httproute.yaml which is
rendered only when the cluster-wide gateway.enabled flag is true
(propagated via _cluster.gateway-enabled). The existing Ingress
template is wrapped in the inverse condition, so exactly one of the
two is rendered at a time and traffic does not split.
Dashboard: HTTPRoute hostname dashboard.<root-host>, backendRef
incloud-web-gatekeeper:8000 (same as the existing Ingress).
Keycloak: HTTPRoute hostname <ingress.host> (default
keycloak.<root-host>), backendRef keycloak-http:80.
Both routes parentRef the tenant Gateway named 'cozystack' in the
publishing namespace and rely on the namespace whitelist on Gateway
listeners (see 'fix(gateway): restrict tenant Gateway listener
allowedRoutes') to ensure only authorised namespaces can attach.
Known gap (follow-up): services that rely on TLS passthrough
(cozystack-api, vm-exportproxy, cdi-uploadproxy) still render their
existing Ingress regardless of gateway.enabled, because Gateway API
passthrough requires a separate TLSRoute + a dedicated
mode: Passthrough listener on the Gateway. Harbor and bucket are
tenant-scoped apps that need _namespace.gateway plumbing — also
follow-up.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Security fix: before this change the Gateway listeners used
'allowedRoutes.namespaces.from: Selector' with the tenant namespace
label, but when per-service HTTPRoutes had to live outside the
tenant namespace that label model broke — so the natural fix was to
relax to 'from: All'. That would have let any namespace in the
cluster attach an HTTPRoute with an arbitrary hostname to the
publishing tenant's Gateway, which is hostname hijacking across
tenants.
The correct solution is to pin each listener to an explicit namespace
whitelist using the built-in 'kubernetes.io/metadata.name' label that
kube-apiserver assigns to every namespace by default. Namespaces not
on the whitelist literally cannot attach HTTPRoutes to this Gateway,
so a non-publishing tenant cannot create a hijacking HTTPRoute in its
own namespace.
The whitelist is the union of:
- The publishing tenant namespace (always, implicitly added).
- 'publishing.gateway.attachedNamespaces' in the platform chart —
a list of system namespaces that host exposed services and need to
attach HTTPRoutes (cozy-cert-manager for ACME, cozy-dashboard,
cozy-keycloak, cozy-system, cozy-harbor, cozy-bucket,
cozy-kubevirt, cozy-kubevirt-cdi, cozy-monitoring,
cozy-linstor-gui).
Plumbed through cozystack-values as '_cluster.gateway-attached-namespaces'
(CSV of namespaces). The gateway chart concats this CSV with the
release namespace, de-dupes via 'uniq', and emits the result as the
matchExpressions 'values' list on every listener.
Tests: 3 new helm-unittest cases in packages/extra/gateway/tests/
covering the default (publishing namespace only), CSV parsing with
empty-entry filtering, and the invariant that HTTPS listeners share
the whitelist with the HTTP listener.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add a cluster-wide 'gateway.enabled' flag in packages/core/platform/values.yaml
(default false) and plumb it into cozystack-values as _cluster.gateway-enabled.
When set to true, the cert-manager ClusterIssuers in cert-manager-issuers
switch their http01 solver from the ingress-nginx backend (ingressClassName)
to the Gateway API backend (gatewayHTTPRoute). Both letsencrypt-prod and
letsencrypt-stage are updated in lockstep.
The new solver attaches to the tenant Gateway named 'cozystack' in the
publishing namespace via parentRefs with sectionName: http, so cert-manager
places its ephemeral HTTPRoute on the Gateway's HTTP (:80) listener. Path
matching on /.well-known/acme-challenge/ is more specific than the sibling
HTTPRoute that redirects HTTP to HTTPS, so ACME challenges arrive correctly.
Default behaviour (gateway.enabled=false) is unchanged — ingress-nginx
path remains the only solver on existing clusters.
Tests: packages/system/cert-manager-issuers/tests/solver_test.yaml adds
4 helm-unittest cases covering the ingress fallback, the gateway path for
both issuers, dns01 unaffected by the flag, and the empty-string default.
Picked up automatically by hack/helm-unit-tests.sh via the new Makefile
test target.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The previous commit in this PR regenerated schema and CRD artefacts
using a local 'dev' build of cozyvalues-gen, which sorts
properties alphabetically. CI installs the pinned upstream release
(v1.3.0), which preserves values.yaml source order, so the committed
outputs diverged from what CI regenerates and pre-commit failed with
exit code 123.
Re-ran 'make generate' locally with the release binary from
https://github.com/cozystack/cozyvalues-gen/releases/tag/v1.3.0 to
produce the same output the CI pipeline expects.
Files refreshed:
- api/apps/v1alpha1/tenant/types.go — field order reverted to
values.yaml source order (host, etcd, monitoring, ingress, gateway,
seaweedfs, schedulingClass, resourceQuotas).
- packages/apps/tenant/values.schema.json — same reordering.
- packages/system/tenant-rd/cozyrds/tenant.yaml — regenerated from
the refreshed schema, gateway bool field now properly sorted.
- packages/extra/gateway/values.schema.json — regenerated for the
same reason; gateway package itself has only one field so the diff
is cosmetic, but keeps the pipeline clean.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Introduces a full per-tenant Gateway API setup backed by Cilium. The
publishing tenant (the one whose namespace matches
publishing.ingressName) can opt into a Gateway via tenant.spec.gateway
instead of or alongside the existing Service.spec.externalIPs-based
ingress-nginx deployment.
Package layout:
- packages/extra/gateway/ — new Helm chart that renders, per release:
* cert-manager Certificate for the apex and wildcard hostnames.
* One Gateway with three listeners — HTTP:80 (ACME challenges and
redirect), HTTPS:443 wildcard *.<host>, HTTPS:443 apex <host>.
Each listener uses allowedRoutes.namespaces.from=Selector keyed on
label cozystack.io/gateway=<tenant namespace> so system services
can attach HTTPRoutes cross-namespace without ReferenceGrants.
* HTTPRoute http-to-https-redirect (301) on the HTTP listener.
cert-manager's own ACME solver HTTPRoute for
/.well-known/acme-challenge/ will take precedence via Gateway
API's more-specific-path matching rules.
* CiliumLoadBalancerIPPool scoped to the tenant namespace and the
Gateway's cilium-gateway-cozystack Service, so the IPs from
publishing.externalIPs get announced via LB IPAM.
* Render-time guards: fails loudly if released outside the
publishing namespace or if publishing.host is unset.
- packages/apps/tenant/ — new values key 'gateway: false' and a
conditional HelmRelease template that references the gateway
ExternalArtifact when the tenant opts in. Mirrors the existing
ingress toggle verbatim, including labels and Flux reconcile config.
- packages/core/platform/sources/gateway-application.yaml — new
PackageSource cozystack.gateway-application with a single 'default'
variant that wires extra/gateway as a tenant module and dependsOn
cozystack.gateway-api-crds + cozystack.cert-manager.
- packages/core/platform/templates/bundles/system.yaml — loads
cozystack.gateway-application in every system variant so the
ExternalArtifact is materialised before any tenant tries to use it.
- api/apps/v1alpha1/tenant/types.go, values.schema.json, README.md,
packages/system/tenant-rd/cozyrds/tenant.yaml — regenerated to
include the new Gateway bool field on TenantSpec.
- packages/extra/gateway/tests/gateway_test.yaml — 6 helm-unittest
cases covering the IPv4/IPv6/mixed IP paths, empty-entry filtering,
the no-IPs-still-renders-Gateway path, the non-publishing-namespace
guard, and the empty-host guard.
Default behaviour is unchanged: tenants default to gateway=false and
the platform only materialises the ExternalArtifact (no resources
inside any tenant namespace unless it opts in).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Register a dedicated cozystack.gateway-api-crds PackageSource and wire
it into the system bundle before networking/cert-manager load. Without
this the host cluster has no gateway.networking.k8s.io CRDs, Cilium's
Gateway API controller cannot bootstrap (no GatewayClass registered),
and cert-manager's gateway-shim controller silently disables Gateway
API certificate automation.
- New PackageSource cozystack.gateway-api-crds → installs
packages/system/gateway-api-crds into namespace cozy-gateway-api.
- cozystack.networking (variants cilium, cilium-kilo, cilium-generic,
kubeovn-cilium, kubeovn-cilium-generic) dependsOn
cozystack.gateway-api-crds so Cilium only starts once the CRDs are
present.
- cozystack.cert-manager dependsOn cozystack.gateway-api-crds, closing
the startup race that the #2208 discussion flagged — cert-manager's
gateway-shim discovers CRDs at process start, so racing with CRD
install left Gateway API solvers permanently disabled until the pod
was restarted.
- system bundle loads cozystack.gateway-api-crds unconditionally for
every variant (isp-full, isp-hosted, isp-full-generic).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Flip envoy.enabled and gatewayAPI.enabled on the vendored Cilium chart.
Cilium becomes the Gateway API implementation for Cozystack:
cilium-envoy DaemonSet starts on each node (approximately 100MB RAM
per node in steady state), and the Cilium operator begins reconciling
gateway.networking.k8s.io resources and publishing the built-in
'cilium' GatewayClass.
This value flip alone does not create any Gateway objects. The
platform installs only an idle Envoy DaemonSet and a GatewayClass
until per-tenant gateway manifests are actually created by the
platform chart.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Align vendored Gateway API CRDs with the latest stable upstream
release (v1.5.1, Oct 2026). TLSRoute graduates to GA in v1.5, still
shipped from the experimental channel to keep the superset of
optional fields. Cilium 1.19 consumes Gateway API CRDs through
standard backward-compatible versioning, so a v1.5 schema is
compatible with the Cilium 1.19 controller.
Notable additions over the previous v1.2.0 vendoring:
- ListenerSet (was XListenerSet / GEP-1713) — shared Gateway with
listeners delegated from separate LS objects. Will enable a
one-Gateway-per-cluster topology for the upcoming Gateway API work.
- BackendTLSPolicy for upstream-to-upstream TLS termination.
- XMesh (experimental) for mesh-specific routing hooks.
- ValidatingAdmissionPolicy 'safe-upgrades.gateway.networking.k8s.io'
plus its binding, shipped by upstream to prevent mixing channels
and downgrading CRD versions.
Stepping stone for the upcoming Gateway API via Cilium work; isolated
from that change so it can be rolled back on its own if any downstream
consumer breaks on the newer schema.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-24 17:03:39 +03:00
51 changed files with 10153 additions and 1334 deletions
{{- fail "publishing.externalIPs is empty — a tenant Gateway has no CiliumLoadBalancerIPPool to draw from. The Gateway Service would stay in <pending> with no address. Set publishing.externalIPs on the platform (or publishing.exposure=loadBalancer with a BGP / L2-announce pool you manage separately) before enabling tenant.spec.gateway=true." }}
{{- end }}
{{- if and (or $isPublishingIngressLB $hasOwnGateway) $ipsList }}
- it:gateway=true + ingress=true + loadBalancer mode renders a single shared pool (no overlapping CIDR conflict)
set:
gateway:true
ingress:true
_cluster:
expose-ingress:tenant-root
expose-external-ips:"192.0.2.10"
expose-mode:loadBalancer
asserts:
- hasDocuments:
count:1
- equal:
path:metadata.name
value:root-exposure
- it:gateway=true with ingress=false still renders pool (gateway alone is enough)
set:
gateway:true
ingress:false
_cluster:
expose-ingress:tenant-root
expose-external-ips:"192.0.2.10"
asserts:
- hasDocuments:
count:1
- it:gateway=true with empty externalIPs fails chart render with explicit message
set:
gateway:true
_cluster:
expose-ingress:tenant-root
expose-external-ips:""
asserts:
- failedTemplate:
errorMessage:"publishing.externalIPs is empty — a tenant Gateway has no CiliumLoadBalancerIPPool to draw from. The Gateway Service would stay in <pending> with no address. Set publishing.externalIPs on the platform (or publishing.exposure=loadBalancer with a BGP / L2-announce pool you manage separately) before enabling tenant.spec.gateway=true."
| `gatewayClassName` | GatewayClass to attach the tenant Gateway to. Must exist cluster-wide. Default matches the Cilium-managed class. | `string` | `cilium` |
| `tlsPassthroughServices` | Names (from publishing.exposedServices) whose traffic is TLS-passthrough rather than TLS-terminate. For each such service a dedicated HTTPS listener with tls.mode=Passthrough is rendered on the Gateway, and the service is expected to attach a TLSRoute instead of an HTTPRoute. | `[]string` | `[api, vm-exportproxy, cdi-uploadproxy]` |
## Security model
Six independent layers protect cross-tenant isolation. Compromising one of them does not bypass the others; admission-time checks (layers 2–5) fail closed (`failurePolicy: Fail`, `validationActions: [Deny]`).
1. **Namespace whitelist on listeners.** The Gateway's `allowedRoutes.namespaces.from: Selector` matches the built-in `kubernetes.io/metadata.name` label (written by kube-apiserver, unspoofable). It accepts routes from the publishing tenant's namespace plus `publishing.gateway.attachedNamespaces` in the platform chart (default includes the `cozy-*` namespaces for platform services and `default` for the Kubernetes API TLSRoute). A namespace outside the list literally cannot attach any `HTTPRoute` or `TLSRoute` to this Gateway.
2. **`cozystack-gateway-hostname-policy`** — `ValidatingAdmissionPolicy` on `gateway.networking.k8s.io/v1 Gateway` CREATE/UPDATE. Reads `namespaceObject.metadata.labels["namespace.cozystack.io/host"]` and rejects any listener hostname that is not equal to that value or a subdomain of it. `matchConditions` gate the VAP to cozystack-managed namespaces only — Gateways in unrelated namespaces (e.g. `kube-system`) are not touched.
3. **`cozystack-gateway-attached-namespaces-policy`** — VAP on `cozystack.io/v1alpha1 Package` CREATE/UPDATE. Rejects any `tenant-*` entry in `spec.components.platform.values.gateway.attachedNamespaces`. Catches direct `kubectl edit packages.cozystack.io` that would bypass the helm render-time guard in layer 6.
4. **`cozystack-tenant-host-policy`** — VAP on `apps.cozystack.io/v1alpha1 Tenant` CREATE/UPDATE. Rejects setting or changing `spec.host` unless the caller's groups contain `system:masters`, `system:serviceaccounts:cozy-system`, `system:serviceaccounts:cozy-cert-manager`, `system:serviceaccounts:cozy-fluxcd` or `system:serviceaccounts:kube-system`. Closes the path where a tenant user sets `spec.host=dashboard.example.org` on their own tenant to have the tenant chart write a hijacked label into the namespace.
5. **`cozystack-namespace-host-label-policy`** — VAP on core `v1 Namespace` CREATE/UPDATE. Rejects any set or change of the `namespace.cozystack.io/host` label, except by the same trusted-caller whitelist as layer 4. This closes both first-time label writes on CREATE and first-time adds on UPDATE — only cozystack/Flux service accounts (which apply the tenant chart) can stamp the label.
6. **Render-time `fail` in cozystack-basics.** The cozystack-basics chart fails the helm render if `_cluster.gateway-attached-namespaces` contains any `tenant-*` entry. Triggers on the helm-install path before the cluster ever sees the values — complements layer 3 which triggers at `kubectl apply` time.
For `tenant-root` the allowed host suffix is `publishing.host`; for any `tenant-<name>` that inherits from its parent the suffix is `<name>.<parent apex>`. A child tenant with an independent apex (`customer1.io` instead of a subdomain) is handled correctly because the VAP reads the per-namespace label rather than assuming a subdomain hierarchy.
## Rate limits
cert-manager issues one `Certificate` per Gateway release. With `issuerName: letsencrypt-prod` (the default), the `Certificate` for a tenant Gateway counts against the [Let's Encrypt rate limits](https://letsencrypt.org/docs/rate-limits/):
- 50 new certificates per registered domain per week.
- 5 duplicate certificates per week for the same set of hostnames.
- 300 new orders per account per 3 hours.
A cluster where many tenants share the same apex domain can exhaust these quickly. Mitigations:
- Use `publishing.certificates.issuerName: letsencrypt-stage` for non-production clusters (staging does not count against prod quotas).
- Limit the number of simultaneous tenant Gateways per cluster via the platform's package quota, or cap it via `tenant.spec.resourceQuotas` with `count/certificates.cert-manager.io` to limit how many `Certificate` objects a tenant may create.
- For bare-metal or air-gapped deployments consider an internal ACME server or the self-signed `ClusterIssuer` (`selfsigned-cluster-issuer`) that ships alongside the Let's Encrypt issuers.
Recommended tenant-level quota to contain a misbehaving tenant:
```yaml
apiVersion: apps.cozystack.io/v1alpha1
kind: Tenant
spec:
gateway: true
resourceQuotas:
count/certificates.cert-manager.io: "10"
```
The default for a fresh tenant is unlimited; operators running shared-apex multi-tenant clusters should set this explicitly (or stage it via the tenant-application default values) before opening `gateway: true` to non-trusted tenants.
## Known limitations
- **Upstream application gaps** — some chart-level features (harbor ACL integrations, bucket upstream limitations) remain on ingress-nginx workflows in upstream docs; cozystack tracks those separately as upstream PRs.
- **Supported ACME issuers** — `publishing.certificates.issuerName` for Gateway-based tenants must be `letsencrypt-prod` or `letsencrypt-stage` (the chart maps those names to concrete ACME server URLs). To support another ACME provider, extend `templates/issuer.yaml` with an additional branch.
{{- fail "_namespace.host is empty — the tenant has no apex domain. Either set tenant.spec.host on this tenant or inherit it from a parent that has host set." }}
{{- fail (printf "packages/extra/gateway currently supports publishing.certificates.issuerName=letsencrypt-prod or letsencrypt-stage (got %q). For other issuers, extend templates/issuer.yaml to map the name to the corresponding ACME server URL." $issuerName) }}
{{- end }}
{{- /*
Let's Encrypt refuses to issue wildcard certificates via HTTP-01 (see
https://letsencrypt.org/docs/challenge-types/#http-01-challenge). The
tenant Gateway's Certificate requests apex + *.apex, so HTTP-01 would
loop the Order forever and the HTTPS listeners would never get a cert.
Require dns01 (or selfsigned) for Gateway API tenants — fail fast so
the operator sees the mismatch at install time instead of during a
stuck cert-manager challenge.
*/}}
{{- if eq $solver "http01" }}
{{- fail "publishing.certificates.solver=http01 cannot issue the wildcard certificate the tenant Gateway needs (Let's Encrypt refuses HTTP-01 for wildcard identifiers). Set publishing.certificates.solver=dns01 (with a Cloudflare API token Secret in cozy-cert-manager) before enabling tenant.spec.gateway=true, or use the self-signed ClusterIssuer for air-gapped clusters." }}
{{- else if ne $solver "dns01" }}
{{- fail (printf "packages/extra/gateway currently supports publishing.certificates.solver=dns01 only (got %q). Extend templates/issuer.yaml to add additional solver types." $solver) }}
- it:empty _namespace.host fails with explicit error
set:
_cluster:
solver:dns01
expose-ingress:tenant-root
expose-external-ips:"192.0.2.10"
_namespace:
host:""
asserts:
- template:templates/gateway.yaml
failedTemplate:
errorMessage:"_namespace.host is empty — the tenant has no apex domain. Either set tenant.spec.host on this tenant or inherit it from a parent that has host set."
- it:allowedRoutes whitelist defaults to publishing namespace only when no extra namespaces are set
- it:chart render fails when publishing.certificates.solver=http01 because Let's Encrypt refuses HTTP-01 for wildcard certs
set:
_cluster:
solver:http01
expose-ingress:tenant-root
expose-external-ips:"192.0.2.10"
issuer-name:letsencrypt-prod
_namespace:
host:example.org
asserts:
- template:templates/issuer.yaml
failedTemplate:
errorMessage:"publishing.certificates.solver=http01 cannot issue the wildcard certificate the tenant Gateway needs (Let's Encrypt refuses HTTP-01 for wildcard identifiers). Set publishing.certificates.solver=dns01 (with a Cloudflare API token Secret in cozy-cert-manager) before enabling tenant.spec.gateway=true, or use the self-signed ClusterIssuer for air-gapped clusters."
- it:Issuer picks the staging ACME server when publishing.certificates.issuerName=letsencrypt-stage
- it:unknown issuer-name fails chart render with an explicit error
set:
_cluster:
solver:dns01
expose-ingress:tenant-root
expose-external-ips:"192.0.2.10"
issuer-name:my-custom-ca
_namespace:
host:example.org
asserts:
- template:templates/issuer.yaml
failedTemplate:
errorMessage:'packages/extra/gateway currently supports publishing.certificates.issuerName=letsencrypt-prod or letsencrypt-stage (got "my-custom-ca"). For other issuers, extend templates/issuer.yaml to map the name to the corresponding ACME server URL.'
- it:HTTPS and HTTPS-apex listeners share the same allowedRoutes whitelist
"description":"GatewayClass to attach the tenant Gateway to. Must exist cluster-wide. Default matches the Cilium-managed class.",
"type":"string",
"default":"cilium"
},
"tlsPassthroughServices":{
"description":"Names (from publishing.exposedServices) whose traffic is TLS-passthrough rather than TLS-terminate. For each such service a dedicated HTTPS listener with tls.mode=Passthrough is rendered on the Gateway, and the service is expected to attach a TLSRoute instead of an HTTPRoute.",
## @param {string} gatewayClassName - GatewayClass to attach the tenant Gateway to. Must exist cluster-wide. Default matches the Cilium-managed class.
gatewayClassName:cilium
## @param {[]string} tlsPassthroughServices - Names (from publishing.exposedServices) whose traffic is TLS-passthrough rather than TLS-terminate. For each such service a dedicated HTTPS listener with tls.mode=Passthrough is rendered on the Gateway, and the service is expected to attach a TLSRoute instead of an HTTPRoute.
{{- fail (printf "publishing.gateway.attachedNamespaces must not contain tenant-* namespaces (got %q). Adding a tenant namespace here lets that tenant attach HTTPRoutes to the publishing tenant's Gateway and hijack hostnames. Child tenants that need their own Gateway must opt in via tenant.spec.gateway=true instead." $ns) }}
"spec.components.platform.values.gateway.attachedNamespaces must not contain any tenant-* namespace; use tenant.spec.gateway=true instead of adding a tenant namespace to the publishing Gateway's whitelist"
"tenant.spec.host can only be set or changed by cluster-admins (system:masters) or by cozystack/Flux service accounts; request user is "+ request.userInfo.username + ". A tenant with empty spec.host inherits the apex from its parent tenant."
reason:Forbidden
---
apiVersion:admissionregistration.k8s.io/v1
kind:ValidatingAdmissionPolicyBinding
metadata:
name:cozystack-tenant-host-policy
labels:
internal.cozystack.io/managed-by-cozystack:""
spec:
policyName:cozystack-tenant-host-policy
validationActions:[Deny]
---
apiVersion:admissionregistration.k8s.io/v1
kind:ValidatingAdmissionPolicy
metadata:
name:cozystack-namespace-host-label-policy
labels:
internal.cozystack.io/managed-by-cozystack:""
spec:
failurePolicy:Fail
matchConstraints:
resourceRules:
- apiGroups:[""]
apiVersions:["v1"]
operations:["CREATE","UPDATE"]
resources:["namespaces"]
matchConditions:
- name:touches-host-label
expression:>-
(has(object.metadata.labels) && "namespace.cozystack.io/host" in object.metadata.labels) ||
(oldObject != null && has(oldObject.metadata.labels) && "namespace.cozystack.io/host" in oldObject.metadata.labels)
variables:
- name:oldHost
expression:>-
(oldObject != null && has(oldObject.metadata.labels) && "namespace.cozystack.io/host" in oldObject.metadata.labels)
"namespace label namespace.cozystack.io/host is immutable once set (was "+ variables.oldHost + ", requested " + variables.newHost + "); only cluster-admins and cozystack/Flux service accounts may change it. Request user: " + request.userInfo.username
{"title":"Chart Values","type":"object","properties":{"host":{"description":"The hostname used to access tenant services (defaults to using the tenant name as a subdomain for its parent tenant host).","type":"string","default":""},"etcd":{"description":"Deploy own Etcd cluster.","type":"boolean","default":false},"monitoring":{"description":"Deploy own Monitoring Stack.","type":"boolean","default":false},"ingress":{"description":"Deploy own Ingress Controller.","type":"boolean","default":false},"seaweedfs":{"description":"Deploy own SeaweedFS.","type":"boolean","default":false},"schedulingClass":{"description":"The name of a SchedulingClass CR to apply scheduling constraints for this tenant's workloads.","type":"string","default":""},"resourceQuotas":{"description":"Define resource quotas for the tenant.","type":"object","default":{},"additionalProperties":{"pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true}}}}
{"title":"Chart Values","type":"object","properties":{"host":{"description":"The hostname used to access tenant services (defaults to using the tenant name as a subdomain for its parent tenant host).","type":"string","default":""},"etcd":{"description":"Deploy own Etcd cluster.","type":"boolean","default":false},"monitoring":{"description":"Deploy own Monitoring Stack.","type":"boolean","default":false},"ingress":{"description":"Deploy own Ingress Controller.","type":"boolean","default":false},"gateway":{"description":"Deploy own Gateway API Gateway (backed by Cilium Gateway API controller).","type":"boolean","default":false},"seaweedfs":{"description":"Deploy own SeaweedFS.","type":"boolean","default":false},"schedulingClass":{"description":"The name of a SchedulingClass CR to apply scheduling constraints for this tenant's workloads.","type":"string","default":""},"resourceQuotas":{"description":"Define resource quotas for the tenant.","type":"object","default":{},"additionalProperties":{"pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true}}}}