## What this PR does
Documentation: https://github.com/cozystack/website/pull/517
Integrates [HAMi](https://github.com/Project-HAMi/HAMi) v2.8.1 (CNCF
Sandbox) into Cozystack as a system-level package for fractional GPU
sharing in tenant Kubernetes clusters.
The integration covers three layers:
- **System chart** (`packages/system/hami/`): Vendors upstream HAMi Helm
chart with device plugin, scheduler extender, mutating webhook, and
RuntimeClass configuration. The broken DRA subchart was removed — it
renders resources even when disabled and references unpublished images.
- **Kubernetes app addon** (`packages/apps/kubernetes/`): HAMi exposed
as an optional toggle (`hami.enabled`). Automatically disables GPU
Operator's native device plugin when active to avoid conflicts. Enforces
hard dependency on GPU Operator.
- **Platform registration** (`packages/core/platform/`): HAMi declared
as PackageSource with gpu-operator dependency, included in the iaas
bundle.
**Known limitation**: HAMi-core relies on a private glibc symbol
(`_dl_sym`) removed in glibc 2.34, which breaks compute isolation on
modern container images (Ubuntu 22.04+, PyTorch/TensorFlow official
images). Alpine/musl is entirely incompatible. See the package README
for details and upstream issue references.
### Screenshots
N/A — no UI changes.
### Release note
```release-note
feat(hami): add HAMi GPU virtualization as an optional system package for fractional GPU sharing. Enables memory and compute isolation for NVIDIA GPUs across tenant workloads. Integrates with GPU Operator and can be enabled per-cluster via the hami.enabled toggle. Note: compute isolation requires glibc < 2.34 in workload containers.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added HAMi GPU virtualization middleware as an optional cluster addon
to enable fractional GPU sharing.
* HAMi addon includes an enable toggle (default: disabled) and a
customizable Helm values override; when enabled it enforces GPU Operator
presence and adjusts GPU Operator values accordingly.
* New HelmRelease integration for deploying HAMi with dependency
ordering and conditional rendering.
* **Documentation**
* Added HAMi setup guide, compatibility notes, and updated cluster addon
configuration reference.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
In environments where control-plane nodes are not available (e.g., AWS
managed nodes, Cozy-in-Cozy tenant Kubernetes clusters), the DaemonSet
cannot schedule any pods because no nodes match this selector.
## What this PR does
This PR adds ability to replace DaemonSet by Deployment with custom
number of replicas for HA purpose. Default values saves current
behavior.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Optional Deployment mode for lineage-controller-webhook with
configurable replicas
* PodDisruptionBudget support when Deployment mode is enabled
* **Updates**
* Improved pod scheduling: node affinity replaces nodeSelector, optional
tolerations, and pod anti-affinity when using Deployment
* Clarified behavior for local API endpoint scheduling (only applies
when pod lands on nodes running an API server)
* **Documentation**
* Added README with usage examples and parameter details
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The HelmRelease at packages/apps/kubernetes/templates/helmreleases/hami.yaml
references chartRef.name 'cozystack-kubernetes-application-kubevirt-kubernetes-hami',
but that component was missing from the kubernetes-application PackageSource.
The HelmRelease would sit in Stalled: ArtifactNotFound at install time.
Add kubernetes-hami next to kubernetes-gpu-operator under variant kubevirt,
mirroring the existing pattern for every other tenant-cluster addon. The
standalone cozystack.hami PackageSource is retained — same shape as
gpu-operator, which is registered both standalone (for bundles/iaas) and
as a kubernetes-application component (for tenant-cluster HelmReleases).
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Collapse the chart to a single Deployment shape that mirrors
cozystack-api: 2 replicas, soft (preferred) nodeAffinity to
node-role.kubernetes.io/control-plane via Exists, permissive
tolerations, soft podAntiAffinity on hostname, an unconditional
PodDisruptionBudget with maxUnavailable: 1, and a Service with
spec.trafficDistribution: PreferClose. The same shape works on
Talos / kubeadm / k3s and on managed Kubernetes / Cozy-in-Cozy
tenant clusters without per-distro overrides — fixing #2417 by
making the soft control-plane affinity gracefully fall back to
worker scheduling when no control-plane nodes are visible.
Drop the DaemonSet path entirely. The previous PR's deployment.enabled
toggle, the workload-kind switch in templates/workload.yaml, the
fail-guard for localK8sAPIEndpoint+nodeAffinity=[], the conditional
PDB, and the values-shape knobs for nodeAffinity and tolerations all
go away as a consequence. Override the standard Deployment fields
through the usual component-values mechanism if a non-default topology
is ever needed.
Mark localK8sAPIEndpoint.enabled deprecated and flip the default to
false. The flag injects KUBERNETES_SERVICE_HOST=status.hostIP, which
is only valid when the pod is actually scheduled on an apiserver-
bearing node. With the new soft control-plane affinity, the pod can
land off-control-plane and crash-loop. The latency motivation for the
flag is real but pending separate webhook performance work; once
addressed, the flag can be removed.
Revert all changes to packages/core/platform/images/migrations/migrations/20.
The earlier ds/...-or-deploy/... fallback was over-engineered: per
run-migrations.sh, migration 20 fires only when CURRENT < 20 (i.e.
direct upgrades from pre-0.37 to 1.3+), and that path is unsupported
anyway. The original ds/... rollout-status line is dead code on every
supported install and upgrade path.
Add helm-unittest coverage for: the default Deployment shape (replicas,
soft nodeAffinity, soft podAntiAffinity, tolerations); no env vars at
the default localK8sAPIEndpoint.enabled=false; PDB rendered
unconditionally with maxUnavailable: 1, including at replicas=1 (the
no-op case); replicas value drives spec.replicas; and that
localK8sAPIEndpoint.enabled=true does inject the env vars. The Service
test continues to assert trafficDistribution: PreferClose with no
internalTrafficPolicy.
Add a slim README documenting the topology, parameters, and the
deprecation note for localK8sAPIEndpoint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Add PackageSource for HAMi with dependency on gpu-operator.
Include HAMi in the iaas bundle as an optional package, allowing
host-level deployment alongside the existing kubernetes addon path.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Service.spec.externalIPs is deprecated upstream in Kubernetes v1.36
(KEP-5707, kubernetes#137293). The AllowServiceExternalIPs feature gate
is expected to default to off around v1.40 and the implementation to
be removed around v1.43. For bare-metal installs that rely on
externalIPs today, cozystack needs a migration path.
This change adds an opt-in 'loadBalancer' exposure mode for the
ingress-nginx Service:
- New platform value 'publishing.exposure' (enum: externalIPs |
loadBalancer, default externalIPs). Plumbed through cozystack-values
into each tenant's ingress HelmRelease via the new 'expose-mode' key.
- Unknown values and loadBalancer with an empty externalIPs list fail
the chart render with explicit error messages, rather than silently
producing a broken Service.
- When exposure=loadBalancer and the current namespace matches
publishing.ingressName, the Service becomes type: LoadBalancer with
externalTrafficPolicy: Local.
- A new template renders a CiliumLoadBalancerIPPool whose blocks come
from publishing.externalIPs (IPv4 addresses get /32, IPv6 addresses
get /128) and whose serviceSelector uses Cilium's synthetic
io.kubernetes.service.namespace key combined with the standard
app.kubernetes.io/name: ingress-nginx label. No custom label is
written to the Service itself, avoiding cross-tenant collisions from
user-defined labels.
Default behaviour is unchanged: without opting in, the Service is
still ClusterIP + spec.externalIPs as today.
Scope: only ingress-nginx is migrated by this setting. Other cozystack
components that still write Service.spec.externalIPs directly (notably
the vpn app) must be migrated separately before the v1.40 feature gate
flip.
Tests: packages/extra/ingress/tests/exposure_test.yaml adds 13
helm-unittest cases covering both modes, IPv4/IPv6, empty-token
filtering, unknown-mode rejection, and the non-matching-namespace
fallback.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vm-default-images] Added package that brings set of images that can be used clusterwide
[vm-disk] Updated source "image" for prettier dropdown selection
[vm-disk] Added new source for vm-disk called disk - to use as source vm-disk from same namespace.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Clone VM disks by specifying an existing vm-disk as the source.
* Global default image collection and chart to publish pre-provisioned
images.
* **UI**
* Forms provide selectable lists for default images and existing VM
disks.
* **Migration**
* Migration to rename existing image DataVolumes to the new
default-images naming and bumped migration version.
* **Documentation**
* VM disk docs and README updated to reflect image sourcing and cloning.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The package imports ~320Gi of golden-image PVCs (ubuntu-noble, fedora,
debian, centos, etc.) as soon as it's installed. On small test and dev
clusters that's enough to consume the entire replicated storage pool,
after which no tenant PVCs — including the ones E2E itself provisions —
can be bound. It's also unreasonable to force that cost on every iaas
user: many deployments don't need prebuilt images at all, and the ones
that do often want to curate their own subset.
Switch the bundle entry from 'package.default' to 'package.optional.default',
matching the treatment already applied to gpu-operator directly below it.
Users who want the golden images can opt in via:
bundles:
enabledPackages:
- cozystack.vm-default-images
The package, its Source, and migration 38 all stay in place — nothing
else changes for users who explicitly enable it. Users who previously
relied on the bundle auto-installing it will need to add the package
to enabledPackages on upgrade; this is called out in the release note.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Adds a new Cozystack system package `linstor-gui` that ships [LINBIT's
LINSTOR web UI](https://github.com/LINBIT/linstor-gui) (GPL-3.0) so
operators can manage nodes, resources, volumes and snapshots from a
browser instead of the `linstor` CLI.
Changes:
- `packages/system/linstor-gui/` — umbrella chart with:
- `images/linstor-gui/Dockerfile` builds the image from the upstream
`pkg.linbit.com` tarball (v2.3.0) on top of
`nginxinc/nginx-unprivileged:1.29-alpine`, mirroring the build pattern
already used for `piraeus-server` and `linstor-csi`.
- `templates/configmap-nginx.yaml` — chart-supplied `nginx.conf` that
proxies `/v1` and `/metrics` to
`linstor-controller.cozy-linstor.svc:3371` over **mTLS**, using the
existing `linstor-client-tls` secret created by the `linstor` package.
- `templates/deployment.yaml` — read-only rootfs, non-root (UID 101),
`runAsNonRoot`, `RuntimeDefault` seccomp, no SA token mounted,
`reloader.stakater.com/auto` on the Deployment.
- `templates/service.yaml` — **ClusterIP only**. No Ingress is shipped
because the LINSTOR controller API is a privileged cluster-wide storage
surface and auth depends on the deployment's OIDC setup; operators wire
up ingress + auth explicitly.
- `tests/deployment_test.yaml` — `helm-unittest` covering Service shape,
TLS secret mount, securityContext, and nginx proxy+mTLS config.
- `packages/core/platform/sources/linstor-gui.yaml` — new
`PackageSource` depending on `cozystack.linstor`.
- `packages/core/platform/templates/bundles/system.yaml` — registers
`linstor-gui` as an **optional** system package (controlled via
`bundles.enabledPackages`), alongside `velero`, `telepresence`, etc.
### Why standalone (not a sidecar in the controller pod)
LINBIT's upstream Dockerfile is designed for this standalone
reverse-proxy topology, and keeping the UI as its own Deployment means
it scales, upgrades, and restarts independently of `linstor-controller`.
The price is the small amount of mTLS config in `nginx.conf`, all of
which lives in the chart's ConfigMap.
### Release note
```release-note
[linstor-gui] Add an opt-in system package that deploys LINBIT's linstor-gui web UI alongside the LINSTOR controller. Enable via `bundles.enabledPackages: [cozystack.linstor-gui]`, then `kubectl -n cozy-linstor port-forward svc/linstor-gui 3373:80` to access.
```
## Test plan
- [ ] `make unit-tests` (requires `helm-unittest` plugin) passes on the
new `tests/deployment_test.yaml`
- [ ] `make image` in `packages/system/linstor-gui/` builds and pushes
the image
- [ ] Install the package on a dev cluster with LINSTOR already
deployed; confirm the pod comes up
- [ ] `kubectl -n cozy-linstor port-forward svc/linstor-gui 3373:80` →
browser loads the UI at `localhost:3373`
- [ ] UI lists LINSTOR nodes, storage pools and resources correctly
(proves mTLS proxy to `/v1` works)
- [ ] Disable via `bundles.enabledPackages` removal; verify the
`Package` resource is cleaned up
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* LINSTOR GUI added as an optional system package deployable to the
cozy-linstor namespace.
* Web UI serves on port 3373, proxies LINSTOR controller over mTLS, and
exposes a /healthz endpoint.
* Image build and release targets added to produce multi-arch container
images.
* **Documentation**
* New README with deployment guidance, connection examples, and
configurable options (endpoint, client secret, image, replicas).
* **Tests**
* Helm chart tests validating service, deployment, config and mTLS
behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The PackageSource for vm-default-images existed but was never included
in any bundle, so it would never be installed. Add it to iaas.yaml
immediately after kubevirt-cdi (which it depends on) so it is deployed
together with the rest of the KubeVirt IaaS stack.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The vm-disk package previously referenced golden-image DataVolumes with
the prefix "vm-image-<name>" in the cozy-public namespace. The new
vm-default-images package creates them as "vm-default-images-<name>",
so any existing vm-disk that sources an image by name would fail to
find its PVC after upgrade.
Migration 38 renames every DataVolume in cozy-public whose name starts
with "vm-image-" to "vm-default-images-" (dropping the old object and
re-creating with the new name) so live vm-disk resources keep resolving
their source PVC transparently.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Makes `cozystack-scheduler` LINSTOR storage-aware by configuring it to
call the existing `linstor-scheduler-extender` as a scheduler extender.
This fixes#2328 (phase 1): pods with both a `SchedulingClass` and
LINSTOR PVCs now get storage-locality-aware placement instead of
bypassing LINSTOR's filter/prioritize logic.
Changes:
- **linstor-scheduler package**: expose the extender sidecar (port 8099)
via a new ClusterIP Service; patch the vendored deployment to add a
`app.kubernetes.io/component: scheduler` label for clean Service
targeting
- **cozystack-scheduler package**: bump to v0.3.0 which adds
configurable `extenders` support; set the LINSTOR extender URL in
wrapper values
- **tests**: helm-unittest for extender rendering in the ConfigMap
### Release note
```release-note
[scheduler] cozystack-scheduler now calls the LINSTOR scheduler extender for storage-aware pod placement. Pods assigned to a SchedulingClass that also use LINSTOR-backed PVCs will prefer nodes with local volume replicas.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added scheduler extender support: exposes an extender Service and adds
pod metadata to enable extender routing.
* **Chores**
* Bumped scheduler chart and image versions to 0.3.0.
* Added default extender configuration and a sample remote extender
entry.
* Updated build scripts to apply an additional patch and added a test
target.
* **Tests**
* Added Helm tests validating rendered ConfigMap with and without
extender configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Expose the existing linstor-scheduler-extender sidecar as a ClusterIP
Service and configure cozystack-scheduler to call it during the
scheduling cycle. Pods with both a SchedulingClass and LINSTOR PVCs
now get storage-locality-aware placement.
- Add extender Service (port 8099) for linstor-scheduler
- Patch vendored deployment to label pods for Service selector
- Bump cozystack-scheduler to v0.3.0 (configurable extenders)
- Add "linstor" PackageSource variant with extender values
- Default variant ships without extender for non-LINSTOR clusters
- Select linstor variant in system bundle
- Add helm-unittest tests for both packages
Ref: #2328
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Ships LINBIT's LINSTOR web UI (GPL-3.0) as an opt-in system package under
packages/system/linstor-gui so operators can manage LINSTOR nodes, resources,
and volumes from a browser instead of the linstor CLI.
The UI is built from the upstream pkg.linbit.com tarball onto an
nginx-unprivileged base image. A chart-supplied nginx.conf proxies /v1 and
/metrics to the LINSTOR controller REST API over mTLS using the existing
linstor-client-tls secret created by the linstor package.
Only a ClusterIP Service is created; no Ingress is shipped, because LINSTOR's
controller API is a privileged cluster-wide storage surface and auth depends
on the deployment's OIDC setup. Operators opt in via bundles.enabledPackages
and wire up ingress + auth themselves (port-forward works out of the box).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
E2E revealed that the kube-apiserver rejects a DaemonSet carrying a
per-container AppArmor annotation for a container that is not present
in the pod spec:
DaemonSet.apps 'cilium' is invalid:
spec.template.annotations[container.apparmor.security.beta.kubernetes.io/mount-cgroup]:
Invalid value: 'mount-cgroup': container not found
On Talos (isp-full bundle) values-talos.yaml sets
cgroup.autoMount.enabled=false, so the upstream chart does not render
the mount-cgroup init container, and our unconditional podAnnotations
entry for it becomes an invalid reference.
Move the podAnnotations block from the shared values.yaml into a new
values-apparmor.yaml and include it only in the cilium-generic and
kubeovn-cilium-generic PackageSource variants, where
cgroup.autoMount.enabled is explicitly true. Talos variants
(cilium, cilium-kilo, kubeovn-cilium) get no AppArmor annotations,
which is safe because the Talos kernel does not load the AppArmor
LSM anyway.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add migration 37 to backfill spec.version=v17 for existing PostgreSQL
resources without a version set.
Hardcode PostgreSQL 17.7 image in monitoring databases (Grafana and Alerta)
to ensure compatibility with monitoring queries that expect PostgreSQL 17
features (pg_stat_checkpointer, updated pg_stat_bgwriter).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Fixes a regression introduced in the bundle restructure (2d022e38) where
`cpuAllocationRatio`, `memoryAllocationRatio`, and
`ephemeralStorageAllocationRatio` from `platform/values.yaml` became
no-ops — never propagated to child packages.
The old monolithic bundles read these values from the cozystack
ConfigMap via `lookup` and passed them explicitly. After migration to
the Package CRD model, the bridge was lost: cozy-lib expects the ratios
in `_cluster` config (via `cozystack-values` Secret), but platform never
wrote them there, so the hardcoded defaults (10, 1, 40) were always used
regardless of what the user configured.
This commit restores the propagation by:
- Adding `cpu-allocation-ratio`, `memory-allocation-ratio`, and
`ephemeral-storage-allocation-ratio` to the `_cluster` section in the
`cozystack-values` Secret, so cozy-lib in all managed applications picks
them up
- Passing `cpuAllocationRatio` to the kubevirt Package component values,
so the KubeVirt CR gets the configured value
### Release note
```release-note
[platform] Fix propagation of resource allocation ratios (cpu, memory, ephemeral-storage) to managed applications and KubeVirt, which were silently ignored since the bundle restructure.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added new resource allocation configuration fields to platform
templates for flexible resource management.
* Enhanced conditional handling of resource settings within container
infrastructure components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Renames the misleading `subnets` field to `networks` in VMInstance. The
old field
name implied the VM was creating subnets, when in reality it attaches
the VM to
existing VPC network attachments (NetworkAttachmentDefinitions). This
change:
- Adds a new `networks` field as the primary way to attach VMs to VPC
networks
- Keeps `subnets` as deprecated with backward-compatible fallback via
Helm `default`
- Fails with a clear error if both `networks` and `subnets` are set
simultaneously
- Adds an API-backed dropdown selector for `networks[].name` listing
available
NetworkAttachmentDefinitions in the namespace
- Hides the deprecated `subnets` field from the dashboard UI
### Release note
```release-note
[vm-instance] Rename `subnets` field to `networks` in VMInstance for clarity.
The old `subnets` field is deprecated but still supported for backward compatibility.
A dropdown selector for available networks has been added to the dashboard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added native networks field for VM attachments and a UI dropdown to
select network attachments; UI hides deprecated subnet fields for VM
instances.
* **Deprecations**
* subnets is deprecated; use networks moving forward (subnets retained
for backward compatibility).
* **Chores / Migration**
* Added a migration to copy existing subnets into networks and bumped
platform migration target version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Adds annotation `helm.sh/resource-policy: keep` to all packages to
prevent its automatic deletion when:
1, Package added to `disabledPackages` parameter in
`cozystack.cozystack-platform` package
2. Package removed form `enabledPackages` parameter in
`cozystack.cozystack-platform` package
This change bring back [docs explained
behavior](https://cozystack.io/docs/v1/operations/configuration/components/)
- platform administrator should manually delete package if needed.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Prevent installed packages deletion
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Helm templates to configure resource-policy annotations on
package resources, ensuring proper resource lifecycle management and
retention policies during cluster deployments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
After migration 36 copies subnets to networks, both fields are populated.
The fail guard would break Helm reconciliation. Instead, networks simply
takes priority via the existing default fallback.
Also handle missing VMInstance CRD gracefully in migration 36.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Existing VMInstance resources store network attachments in the deprecated
spec.subnets field. This migration copies subnets to the new spec.networks
field so the dashboard UI correctly displays attached networks. The old
subnets field is kept intact because migrations run before the new CRD is
applied.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Since the bundle restructure in 2d022e38, the cpuAllocationRatio,
memoryAllocationRatio and ephemeralStorageAllocationRatio values from
platform values.yaml were no longer propagated to child packages.
The old monolithic bundles read these values from the cozystack
ConfigMap via lookup and passed them explicitly. After migration to
the Package CRD model, the bridge was lost: cozy-lib expects the
ratios in _cluster config (via cozystack-values Secret), but platform
never wrote them there, so the hardcoded defaults were always used.
This commit restores the propagation by:
- Adding cpu/memory/ephemeral-storage allocation ratios to the
_cluster section in the cozystack-values Secret, so cozy-lib in
all managed applications picks them up.
- Passing cpuAllocationRatio to the kubevirt Package component, so
the KubeVirt CR gets the configured value.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Install the cozystack-scheduler package and the SchedulingClass CRD by
default.
### Release-note
```release-note
[cozystack-scheduler] Enable the cozystack-scheduler and SchedulingClass
CRD by default.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Removes the `apiServerEndpoint` fallback for `MASTER_NODES` in the
`isp-full-generic` variant.
When `MASTER_NODES` is not explicitly set (the default), the template
used to parse a single IP from `apiServerEndpoint` and pass it to
kube-ovn. This prevented kube-ovn from discovering all control-plane
nodes, breaking OVN RAFT consensus on multi-master clusters
(`ovn-central` crashed with `host ip X not in env NODE_IPS`).
Now kube-ovn uses its built-in Helm `lookup` to find all control-plane
nodes by the `MASTER_NODES_LABEL`
(`node-role.kubernetes.io/control-plane=true`), correctly discovering
all masters.
Explicit `MASTER_NODES` override via platform values is preserved.
The `isp-full` (Talos) variant is not affected — it uses a separate code
path.
Fixes#2242
### Release note
```release-note
[kube-ovn] Fix multi-master support for generic Kubernetes clusters by letting kube-ovn auto-discover control-plane nodes by label instead of falling back to a single API server IP
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced support for automatic control plane node discovery using
Kubernetes node labels, providing greater flexibility and scalability
options for cluster configurations.
* **Bug Fixes**
* Removed unreliable automatic node configuration fallback logic to
improve overall deployment consistency and predictability. Master nodes
now require explicit configuration or automatic discovery through
Kubernetes node labels.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When only leaf packages change but the platform chart remains unchanged,
the Flux ArtifactGenerator produces an ExternalArtifact with the same
content digest. This prevents the platform HelmRelease from reconciling,
so lookup() in repository.yaml never runs and cozystack-packages
OCIRepository does not pick up the new OCI digest.
Write a content-based hash of the working tree state to
core/platform/.build-revision before pushing the OCI artifact, ensuring
the platform chart content always changes on every build.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Remove the apiServerEndpoint fallback for MASTER_NODES in the
isp-full-generic variant. When MASTER_NODES is not explicitly set,
kube-ovn now uses its built-in Helm lookup to find all control-plane
nodes by the MASTER_NODES_LABEL, correctly discovering all masters
instead of just the single API server IP.
The previous fallback always produced a single IP (the API endpoint),
which broke OVN RAFT consensus on multi-master clusters because
ovn-central was configured with only one node in NODE_IPS.
Explicit MASTER_NODES override is preserved for users who need it.
Fixes: #2242
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds `authentication.oidc.keycloakInternalUrl` platform value that
allows oauth2-proxy
in the dashboard to route backend calls (token exchange, JWKS, userinfo,
logout) through
an internal cluster URL while keeping browser redirects on the external
Keycloak URL.
When set, oauth2-proxy uses `--skip-oidc-discovery` and explicit
endpoint URLs pointing
to the internal Keycloak service. This avoids external DNS lookups and
TLS overhead for
pod-to-pod communication.
Fully backward-compatible: when the value is empty (default), behavior
is unchanged.
### Release note
```release-note
[dashboard] Added `authentication.oidc.keycloakInternalUrl` platform value to route oauth2-proxy backend requests through internal Keycloak service URL, bypassing external DNS and TLS.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configuration support for an internal KeyCloak URL, enabling
backend authentication requests to be routed through an alternative
endpoint while maintaining existing external URLs for browser
interactions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[backups] Added fix to roles and changed backupstrategy-controller location
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated backup controller namespace configuration.
* Enhanced backup controller permissions for leader election and event
recording capabilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When set, oauth2-proxy skips OIDC discovery and routes all backend calls
(token exchange, JWKS, userinfo, logout) through the internal cluster URL
while keeping browser redirects on the external URL. This avoids external
DNS lookups and TLS overhead for pod-to-pod communication with Keycloak.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Adds the cozystack-scheduler as an optional system package, vendored
from https://github.com/cozystack/cozystack-scheduler. The scheduler
extends the default kube-scheduler with SchedulingClass-aware affinity
plugins, allowing platform operators to define cluster-wide scheduling
constraints via a SchedulingClass CRD. Pods opt in via the
`scheduler.cozystack.io/scheduling-class` annotation.
The package includes:
- Helm chart with RBAC, ConfigMap, Deployment, and CRD
- PackageSource definition for the cozystack package system
- Optional inclusion in the platform system bundle
### Release note
```release-note
[cozystack-scheduler] Add cozystack-scheduler as an optional system
package. The custom scheduler supports SchedulingClass CRDs for
cluster-wide node affinity, pod affinity, and topology spread constraints.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added cozystack-scheduler component as an optional system package.
* Introduced SchedulingClass custom resource for advanced scheduling
configurations.
* Scheduler supports node affinity, pod affinity, pod anti-affinity, and
topology spread constraints.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Add external-dns as a standalone self-managed application in
`packages/extra/external-dns/`, allowing tenants to deploy and configure
their own DNS management directly from the dashboard.
## Motivation
Tenants need the ability to manage their own DNS domains with their own
provider. Following the [developers
guide](https://github.com/cozystack/website/pull/413), this is
implemented as an extra package (like `ingress` and `seaweedfs`) using
the HelmRelease-based pattern, rather than embedding it in the tenant
chart.
This enables multi-tenant scenarios where:
- Tenant A uses Cloudflare for `domain-a.com`
- Tenant B uses AWS Route53 for `domain-b.com`
- Each tenant deploys and manages external-dns independently from the
dashboard
## Changes
- **New package**: `packages/extra/external-dns/` — standalone
HelmRelease-based application
- **New PackageSource**:
`packages/core/platform/sources/external-dns-application.yaml` —
references `system/external-dns` and `extra/external-dns` components
- **Cleaned tenant chart**: removed the previously embedded
`externalDns` block from `packages/apps/tenant/`
## Features
- Support for 9 DNS providers: cloudflare, aws, azure, google,
digitalocean, linode, ovh, exoscale, godaddy
- Per-provider credential configuration with full JSON schema validation
- Domain filtering via `domainFilters`
- Configurable sync policy (`sync` or `upsert-only`)
- Namespaced operation (`namespaced: true`) for tenant isolation
- Unique `txtOwnerId` per namespace to prevent DNS record conflicts
- Resource sizing via presets or explicit CPU/memory
## Usage Example
Deploy from the dashboard, or via values:
```yaml
# Cloudflare
provider: cloudflare
domainFilters:
- example.com
cloudflare:
apiToken: "your-cloudflare-api-token"
```
```yaml
# AWS Route53
provider: aws
domainFilters:
- example.org
aws:
accessKeyId: "AKIAXXXXXXXX"
secretAccessKey: "your-secret-key"
region: "us-east-1"
```
## Test plan
- [ ] `helm template external-dns packages/extra/external-dns/ --set
provider=cloudflare --set cloudflare.apiToken=test` renders correctly
- [ ] `helm template external-dns packages/extra/external-dns/` fails
(provider required)
- [ ] `helm template wrong-name packages/extra/external-dns/ --set
provider=cloudflare` fails (release name check)
- [ ] Deploy external-dns from tenant dashboard
- [ ] Verify HelmRelease is created in tenant namespace with namespaced
RBAC
- [ ] Create an Ingress and verify DNS record is created
- [ ] Verify no conflict with global external-dns instance
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an External DNS package for automatic DNS record management.
* Support for 9 DNS providers: Cloudflare, AWS, Azure, Google,
DigitalOcean, Linode, OVH, Exoscale, GoDaddy.
* Helm-based deployment with namespaced/system variants and release
configuration options.
* Configurable synchronization policies, domain filtering, provider
credentials, extra args, and resource presets.
* **Documentation**
* New README and schema-driven values documentation for installation and
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Migrates VictoriaLogs from the deprecated single-node `VLogs` CR to
`VLCluster` (cluster mode) with vlinsert/vlselect/vlstorage components
for reliability and horizontal scalability.
**Operator upgrade:**
- Upgrades victoria-metrics-operator from v0.55.0 to v0.68.1 to add
VLCluster CRD support
**VLCluster deployment:**
- Replaces `VLogs` (v1beta1) with `VLCluster` (v1) — 2 replicas per
component, consistent with VMCluster
- Adds VPA for all VLCluster components (vlinsert, vlselect, vlstorage)
- Updates WorkloadMonitors for the three-component architecture
**Endpoint updates:**
- Fluent-bit outputs: `vlogs-generic:9428` → `vlinsert-generic:9481`
- Grafana datasource: `vlogs-{name}:9428` → `vlselect-{name}:9471`
- ExternalName service: `vlogs-generic` → `vlinsert-generic`
**Migration (35 → 36):**
- Adds `helm.sh/resource-policy: keep` annotation to existing VLogs
resources so they are preserved during upgrade
- Users need to verify the new VLCluster is working, then optionally
migrate historical data and manually delete old VLogs resources
### Release note
```release-note
[monitoring] Migrate VictoriaLogs from single-node VLogs to VLCluster (cluster mode). Old VLogs resources are preserved with `helm.sh/resource-policy: keep` annotation. After upgrade, verify the new cluster is working, then optionally migrate historical data and delete old VLogs resources manually.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* CRD upgrade workflow with configurable hooks, Job, and ServiceAccount
support
* Monitoring storage split into vlinsert / vlselect / vlstorage with
corresponding VPAs
* Service traffic distribution option and optional shareProcessNamespace
toggle
* **Updates**
* VictoriaMetrics Operator bumped to v0.68.1
* Fluent Bit and Grafana endpoints/ports updated to new monitoring
targets
* Global extra-labels support for resources
* Migration target advanced to version 36
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds the cozystack-scheduler as an optional system package, vendored from
https://github.com/cozystack/cozystack-scheduler. The scheduler extends
the default kube-scheduler with SchedulingClass-aware affinity plugins,
allowing platform operators to define cluster-wide scheduling constraints
via a SchedulingClass CRD. Pods opt in via the
`scheduler.cozystack.io/scheduling-class` annotation.
The package includes:
- Helm chart with RBAC, ConfigMap, Deployment, and CRD
- PackageSource definition for the cozystack package system
- Optional inclusion in the platform system bundle
### Release note
```release-note
[cozystack-scheduler] Add cozystack-scheduler as an optional system
package. The custom scheduler supports SchedulingClass CRDs for
cluster-wide node affinity, pod affinity, and topology spread constraints.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## Summary
- Migration 34 fails with `error: the server doesn't have a resource
type "rabbitmqs"` when `rabbitmqs.apps.cozystack.io` CRD does not exist
on the cluster
- This happens when RabbitMQ was never installed — the CRD is not
present, `kubectl get` fails, and `set -euo pipefail` terminates the
migration job
- Added a CRD existence check before listing resources; if CRD is
absent, the migration stamps the version and exits cleanly
## Test plan
- [ ] Deploy a cluster without RabbitMQ installed and run migration 34 —
should skip gracefully
- [ ] Deploy a cluster with RabbitMQ instances without `spec.version`
set — should patch them to `v3.13`
- [ ] Deploy a cluster with RabbitMQ instances already having
`spec.version` — should skip patching
Replace deprecated single-node VLogs CR with VLCluster (cluster mode)
for reliability and horizontal scalability.
Changes:
- Replace VLogs (v1beta1) with VLCluster (v1) using vlinsert/vlselect/vlstorage
- Update fluent-bit outputs to vlinsert-generic:9481
- Update Grafana datasource to vlselect:9471
- Update ExternalName service from vlogs-generic to vlinsert-generic
- Add VPA for all VLCluster components
- Update WorkloadMonitors for three-component architecture
- Add migration 35 to preserve old VLogs resources with keep annotation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Migration 34 fails when rabbitmqs.apps.cozystack.io CRD does not exist,
which happens when RabbitMQ was never installed on the cluster. Add a
check for CRD presence before attempting to list resources.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
