3364290761
## What this PR does Adds an opt-in `publishing.exposure=loadBalancer` mode for the ingress-nginx Service as a migration path away from `Service.spec.externalIPs`, which is deprecated upstream in Kubernetes v1.36 ([KEP-5707](https://github.com/kubernetes/enhancements/issues/5707), [kubernetes#137293](https://github.com/kubernetes/kubernetes/pull/137293)). The `AllowServiceExternalIPs` feature gate is expected to default to off around v1.40 and the implementation to be removed around v1.43. Stacked on top of #2464 (cilium v1.19.3 bump) — depends on it for the `CiliumLoadBalancerIPPool` at `cilium.io/v2`. ### Behavior - New platform value `publishing.exposure` — enum `externalIPs | loadBalancer`, default `externalIPs` (current behavior unchanged on upgrade). - Plumbed through `cozystack-values` into each tenant's ingress HelmRelease via the new `expose-mode` key. - When `exposure=loadBalancer` and the current namespace matches `publishing.ingressName`, the Service becomes `type: LoadBalancer` with `externalTrafficPolicy: Local` and a `CiliumLoadBalancerIPPool` announces the addresses from `publishing.externalIPs` via Cilium LB IPAM. - The pool uses a namespace-only `serviceSelector` (`io.kubernetes.service.namespace: <ns>`) — any LoadBalancer Service in the tenant namespace draws from it. See "Pool ownership" below. - IPv4 addresses get `/32` CIDRs, IPv6 addresses get `/128`. Mixed families supported. Pre-CIDR entries (`192.0.2.10/32`) are accepted without double-suffixing. - Unknown values, `loadBalancer` with empty externalIPs, and stray empty entries from `publishing.externalIPs` are rejected at render time with explicit error messages. ### Pool ownership The `CiliumLoadBalancerIPPool` is rendered from `packages/apps/tenant/templates/cilium-lb-pool.yaml`, not from the ingress chart. The tenant chart is the per-tenant owner of cross-cutting resources (Namespace, `cozystack-values` Secret, HelmReleases for ingress and gateway), so one pool per tenant lives there. Cilium LB IPAM rejects overlapping CIDRs across pools regardless of `serviceSelector` — the last-added pool gets `cilium.io/PoolConflict` and stops allocating. Keeping the pool in the ingress chart would collide with the Gateway-API PR (#2470), which materialises its own LoadBalancer Service from the same `publishing.externalIPs` range when `tenant.spec.gateway=true`. Moving the pool to the tenant chart with a namespace-only selector lets a single pool back both services (ingress-nginx today, a Cilium Gateway Service once #2470 lands). Only the ingress-loadBalancer signal is wired here (`_cluster.expose-mode=loadBalancer` + `.Values.ingress=true` + publishing tenant). #2470 rebases on top of this PR, drops its own `packages/extra/gateway/templates/cilium-lb-pool.yaml`, and adds an OR branch for `.Values.gateway` in the tenant template. ### Scope Only the ingress-nginx Service is migrated by this setting. Other cozystack components that still write `Service.spec.externalIPs` directly (notably the `vpn` app at `packages/apps/vpn/templates/service.yaml`) need separate follow-up before Kubernetes v1.40. ### Tests - `packages/extra/ingress/tests/exposure_test.yaml` — 10 helm-unittest cases on the ingress-nginx Service: type / externalTrafficPolicy / externalIPs assertion for both modes, unknown-mode rejection (case-sensitive enum), empty-IPs failure, empty-entry filtering, non-publishing-tenant fallback. - `packages/apps/tenant/tests/exposure_test.yaml` — 9 cases on the pool itself: IPv4, IPv6, mixed, pre-CIDR input, empty-entry filtering, `ingress=false` in publishing tenant (no pool), non-publishing tenant (no pool), empty externalIPs (no pool). - Both suites are auto-discovered by `hack/helm-unit-tests.sh` via the `test:` target in each package's Makefile. ### Caveats (copied from the inline `values.yaml` comment) - `loadBalancer` mode uses `externalTrafficPolicy: Local`. The external IP must already be routed to a node that hosts an ingress pod (floating IP / upstream router / podAntiAffinity). - Cilium does not announce the IP on its own unless L2 announcements or BGP are enabled in the Cilium values (disabled by default in cozystack). - Switching the value on a running cluster causes the ingress-nginx Service to be recreated (`upgrade.force: true` on the HelmRelease + Service kind change); expect a brief interruption of ingress traffic. ### Release note ```release-note feat(ingress): add opt-in publishing.exposure=loadBalancer mode that uses type: LoadBalancer + CiliumLoadBalancerIPPool instead of the deprecated Service.spec.externalIPs. Default (externalIPs) preserves existing behavior. ``` |
||
|---|---|---|
| .gemini | ||
| .github | ||
| api | ||
| cmd | ||
| dashboards | ||
| docs | ||
| examples/backups/vmi | ||
| hack | ||
| img | ||
| internal | ||
| packages | ||
| pkg | ||
| tools/openapi-gen | ||
| .coderabbit.yaml | ||
| .gitignore | ||
| .pre-commit-config.yaml | ||
| ADOPTERS.md | ||
| AGENTS.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| CONTRIBUTOR_LADDER.md | ||
| go.mod | ||
| go.sum | ||
| GOVERNANCE.md | ||
| LICENSE | ||
| MAINTAINERS.md | ||
| Makefile | ||
| README.md | ||
| SECURITY.md | ||
Cozystack
Cozystack is a free platform and framework for building clouds.
Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.
If you have Feature Requests please use the Discussion's Feature Request section.
Community
You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.