## What this PR does
Documentation: https://github.com/cozystack/website/pull/517
Integrates [HAMi](https://github.com/Project-HAMi/HAMi) v2.8.1 (CNCF
Sandbox) into Cozystack as a system-level package for fractional GPU
sharing in tenant Kubernetes clusters.
The integration covers three layers:
- **System chart** (`packages/system/hami/`): Vendors upstream HAMi Helm
chart with device plugin, scheduler extender, mutating webhook, and
RuntimeClass configuration. The broken DRA subchart was removed — it
renders resources even when disabled and references unpublished images.
- **Kubernetes app addon** (`packages/apps/kubernetes/`): HAMi exposed
as an optional toggle (`hami.enabled`). Automatically disables GPU
Operator's native device plugin when active to avoid conflicts. Enforces
hard dependency on GPU Operator.
- **Platform registration** (`packages/core/platform/`): HAMi declared
as PackageSource with gpu-operator dependency, included in the iaas
bundle.
**Known limitation**: HAMi-core relies on a private glibc symbol
(`_dl_sym`) removed in glibc 2.34, which breaks compute isolation on
modern container images (Ubuntu 22.04+, PyTorch/TensorFlow official
images). Alpine/musl is entirely incompatible. See the package README
for details and upstream issue references.
### Screenshots
N/A — no UI changes.
### Release note
```release-note
feat(hami): add HAMi GPU virtualization as an optional system package for fractional GPU sharing. Enables memory and compute isolation for NVIDIA GPUs across tenant workloads. Integrates with GPU Operator and can be enabled per-cluster via the hami.enabled toggle. Note: compute isolation requires glibc < 2.34 in workload containers.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added HAMi GPU virtualization middleware as an optional cluster addon
to enable fractional GPU sharing.
* HAMi addon includes an enable toggle (default: disabled) and a
customizable Helm values override; when enabled it enforces GPU Operator
presence and adjusts GPU Operator values accordingly.
* New HelmRelease integration for deploying HAMi with dependency
ordering and conditional rendering.
* **Documentation**
* Added HAMi setup guide, compatibility notes, and updated cluster addon
configuration reference.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
In environments where control-plane nodes are not available (e.g., AWS
managed nodes, Cozy-in-Cozy tenant Kubernetes clusters), the DaemonSet
cannot schedule any pods because no nodes match this selector.
## What this PR does
This PR adds ability to replace DaemonSet by Deployment with custom
number of replicas for HA purpose. Default values saves current
behavior.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Optional Deployment mode for lineage-controller-webhook with
configurable replicas
* PodDisruptionBudget support when Deployment mode is enabled
* **Updates**
* Improved pod scheduling: node affinity replaces nodeSelector, optional
tolerations, and pod anti-affinity when using Deployment
* Clarified behavior for local API endpoint scheduling (only applies
when pod lands on nodes running an API server)
* **Documentation**
* Added README with usage examples and parameter details
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The `update:` recipe now reproduces the top-level vendoring overrides
(remove broken hami-dra subchart, clear Chart.yaml dependencies, strip
dra/hami-dra/podSecurityPolicy from upstream values.yaml) after
`helm pull`, so they no longer silently disappear on the next bump.
Template-level patches (DRA guards in scheduler/*, indent fix in
device-plugin/monitorservice.yaml) are documented with rationale and
commit references — they remain a manual step because automating them
would be fragile across upstream template restructures.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The HelmRelease at packages/apps/kubernetes/templates/helmreleases/hami.yaml
references chartRef.name 'cozystack-kubernetes-application-kubevirt-kubernetes-hami',
but that component was missing from the kubernetes-application PackageSource.
The HelmRelease would sit in Stalled: ArtifactNotFound at install time.
Add kubernetes-hami next to kubernetes-gpu-operator under variant kubevirt,
mirroring the existing pattern for every other tenant-cluster addon. The
standalone cozystack.hami PackageSource is retained — same shape as
gpu-operator, which is registered both standalone (for bundles/iaas) and
as a kubernetes-application component (for tenant-cluster HelmReleases).
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Match the pattern used by every other tenant-cluster addon HelmRelease
(gpu-operator, cilium, fluxcd, ...): only render when the upstream etcd
namespace is ready, not just when the addon is enabled. Add values-ci.yaml
to the HAMi test suite so the new gate has the fixture it needs.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The HAMi-driven default of disabling gpu-operator's device plugin is
applied via valuesOverride and can be re-enabled by users for advanced
topologies (mixed HAMi / vanilla NVIDIA device plugin pools). Document
this explicitly so the README aligns with the existing template merge
order.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
HAMi#173 was closed as "not planned" and only suggests a typo fix
(< 2.3.0 → < 2.30); it does not establish the actual 2.34 boundary.
HAMi-core#174 (symbol-level cause) and HAMi#1190 (empirical per-glibc
behavior) cover the same ground accurately.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The auto-generated changelog only listed #2459/#2467 (velero-configmap
Role move). Five additional PRs were backported and merged into
release-1.3 between then and the v1.3.1 tag (2026-04-28):
- #2471/#2524 - fix(api): IDOR in TenantNamespace Get/Watch
- #2496/#2505 - feat(linstor): linstor-csi v1.10.6 (Protocol-C dual-attach)
- #2462/#2511 - fix(etcd): remove destructive post-upgrade hook
- #2421/#2491 - fix(kamaji): memory limits + startup probe
- #2498/#2518 - build(linstor): wire linstor-gui into root build target
Update the release date to match the actual tag (2026-04-28), rewrite
the intro paragraph, and add @kvaps to contributors.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The AI-generated v1.3.1 changelog (#2480) was generated from
git log v1.3.0..main rather than git log v1.3.0..v1.3.1, because
the workflow checked out main while the v1.3.1 tag points to
release-1.3. As a result the changelog included:
- 8 PRs that were merged to main but never shipped in v1.3.1
- 6 backport PRs that were merged to release-1.3 *after* v1.3.1
was tagged
- Both originals and their backports as separate duplicate entries
- A 2024 PR (#435) that has nothing to do with this range
- Generic "Documentation updates" placeholders for website entries
- Title duplicated as both the bold label and the description
(`* **fix(...): X**: fix(...): X (...)`)
- The cozystack-ci bot listed as a human contributor
The actual v1.3.1 release range (v1.3.0..v1.3.1) contains exactly
one user-facing change:
41bcb0be [Backport release-1.3] fix(backups): move
velero-configmap Role to velero chart (#2467)
which is the backport of #2459 (myasnikovdaniil) shipped via
#2467 (IvanHunters). This commit replaces the contents of
docs/changelogs/v1.3.1.md with that one entry, the matching
two-person contributors list, and the standard footer.
The workflow + docs fixes that prevent this regression for future
patch releases will land in a separate PR against main.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Adds GPU observability to Cozystack: five Grafana dashboards, a set of
VictoriaMetrics recording rules with a throttle-regression alert, and
reference manifests for wiring the NVIDIA GPU Operator and DCGM Exporter
on Talos when running GPU workloads directly in pods.
### Components
1. **Dashboards** — `dashboards/gpu/*.json`, registered in
`packages/system/monitoring/dashboards-infra.list`. All follow the
project convention: the Prometheus data source is selected through the
`$ds_prometheus` template variable.
- `gpu-performance` — per-GPU utilization (NVML, tensor pipe, graphics
engine, memory copy), VRAM, power, temperature, and health (XID errors,
power/thermal throttling). All metrics are shown per-GPU, filterable by
the `$Hostname` selector.
- `gpu-fleet` — cluster-wide inventory, capacity, utilization, and
per-node power draw as percentage of TDP.
- `gpu-tenants` — per-namespace allocation and 24-hour GPU-hour/kWh
accounting; utilization and power are shown per-node (DCGM cannot
attribute hardware metrics to namespaces).
- `gpu-quotas` — allocated vs. requested GPUs, pending pods, and a
per-namespace requests/limits table. All counts derive from the
`namespace:gpu_count:allocated` recording rule, which excludes
`Failed`/`Succeeded` pod carryover.
- `gpu-efficiency` — cluster and per-GPU efficiency: tensor saturation,
util-per-watt, and per-GPU power/thermal throttle fractions.
2. **Recording rules and alert** —
`packages/system/monitoring-agents/alerts/gpu-recording.rules.yaml`.
Four rule groups:
- `gpu.recording.cluster.1m` — cluster-wide totals consumed by overview
panels.
- `gpu.recording.node.1m` — per-node hardware aggregates (grouped by
`Hostname`) plus per-namespace allocation counts from kube resource
requests; filters `cozy-*`/`kube-*`.
- `gpu.recording.efficiency.1m` — per-GPU tensor saturation and
util-per-watt, plus per-GPU power/thermal throttle fractions bounded via
`clamp_max(..., 1)`.
- `gpu.recording.throttle.validation.5m` — houses the
`GPUThrottleFractionOverOne` warning alert that fires if the pre-clamp
throttle fraction exceeds 1.0. This guards against DCGM counter-unit
drift between GPU families; the `/1e9` divisor is verified on NVIDIA A10
with DCGM 3.x.
Rules are safe on clusters without DCGM — they evaluate to empty series
when no matching metrics are scraped.
3. **Reference manifests** under
`packages/system/gpu-operator/examples/`. These are **not** templates;
they document one working configuration:
- `values-native-talos.yaml` — Cozystack Package values for the
native-pod scenario.
- `dcgm-custom-metrics.yaml` — ConfigMap extending the DCGM CSV with
profiling, ECC, throttling, and energy counters used by the dashboards
and recording rules.
- `nvidia-driver-compat.yaml` — DaemonSet that stages
`libnvidia-ml.so.1` and `nvidia-smi` from the Talos glibc tree into a
location the GPU Operator validator inspects. Workaround for
[NVIDIA/gpu-operator#1687](https://github.com/NVIDIA/gpu-operator/issues/1687).
Requires `pod-security.kubernetes.io/enforce: privileged` on the target
namespace when the cluster enforces baseline/restricted
PodSecurityStandards.
- `README.md` — explains the two deployment paths (sandbox vs. native),
which DCGM metrics each dashboard depends on, and how the pieces fit
together.
4. **Cross-validation test** — `hack/check-gpu-recording-rules.bats`
enforces that every recording-rule reference in tracked dashboards
resolves, and that every DCGM metric referenced by the rules is declared
in either the upstream default CSV snapshot
(`hack/dcgm-default-counters.csv`) or the custom CSV shipped under
`examples/`.
### Why reference manifests, not templates
The out-of-the-box `packages/system/gpu-operator/values-talos.yaml`
targets **sandbox workloads** — GPUs passed through to KubeVirt VMs via
VFIO, driver disabled on the host, device plugin disabled. That is the
sensible default for Cozystack today.
Running GPU workloads directly in pods on Talos is also possible but
requires a different configuration that makes assumptions not every user
shares:
- the NVIDIA Talos system extension must be installed on GPU nodes,
- the operator's own driver and toolkit components must be disabled
because the extension provides them,
- the validator's hardcoded paths need a workaround until
NVIDIA/gpu-operator#1687 lands upstream.
Shipping those as active templates would silently impose those
assumptions. Shipping them as reference files next to the package lets
operators opt in with eyes open and understand the moving parts before
applying them.
### Scope note
The dashboards and recording rules are useful independently of the
reference manifests — any cluster that scrapes DCGM Exporter (however it
is installed) will populate them. The reference manifests are
self-contained and do not change any default behavior of the
`gpu-operator` package.
### Screenshots
<img width="2824" height="1055" alt="2026-04-27_12-36-52"
src="https://github.com/user-attachments/assets/71f85de7-1c80-4817-b36c-aeb1c2bdd39a"
/>
<img width="2913" height="1255" alt="2026-04-27_12-36-39"
src="https://github.com/user-attachments/assets/7a54daca-0727-4510-938c-080d31bdc229"
/>
<img width="2932" height="1251" alt="2026-04-27_12-36-30"
src="https://github.com/user-attachments/assets/f6c6c215-3d05-400c-b21c-29f267c93f23"
/>
<img width="2864" height="1222" alt="2026-04-27_12-36-00"
src="https://github.com/user-attachments/assets/e9d34bab-ade5-434b-92c3-f36712cf68d9"
/>
<img width="2958" height="1226" alt="image"
src="https://github.com/user-attachments/assets/0047a5f8-12a5-4cda-8676-041de139cb76"
/>
### Release note
```release-note
feat(monitoring): add GPU observability — five Grafana dashboards (performance, fleet, tenants, quotas, efficiency), VictoriaMetrics recording rules with a throttle-regression alert, and reference manifests under packages/system/gpu-operator/examples/ for running GPU workloads natively in pods on Talos.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added five GPU dashboards for performance, efficiency, fleet overview,
quotas/allocation, and tenant usage (utilization, power, throttling,
allocation, billing).
* Added cluster recording rules and a throttling validation alert; new
Prometheus-derived metrics and per-GPU rankings/timeseries.
* **Documentation**
* Added GPU Operator examples and README documenting native Talos setup
and required DCGM counters.
* **Tests**
* Added validation tests ensuring dashboard ↔ recording-rule and DCGM
metric consistency.
* **Chores**
* Registered new GPU dashboards in infra listings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Tests need _namespace.etcd from values-ci.yaml after the merge
introduced an etcd-namespace guard on the gpu-operator HelmRelease
condition.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## Summary
Fixed two IDOR (Insecure Direct Object Reference) vulnerabilities in the
TenantNamespace API handlers that allowed authenticated users to access
metadata of tenant namespaces without proper authorization checks.
## Changes
### New optimized function: hasAccessToNamespace()
- Lists RoleBindings **only in the target namespace** instead of all
cluster RoleBindings
- Used by Get() and Watch() for single-namespace access checks
- Order of magnitude faster than the previous approach
### Get() handler
- Uses `hasAccessToNamespace()` for efficient authorization
- Returns `NotFound` instead of `Forbidden` to prevent tenant
enumeration
- Now correctly enforces RoleBinding-based access control
### Watch() handler
- Uses `hasAccessToNamespace()` for per-event authorization
- Logs authorization errors with klog for security audit
- Events for unauthorized namespaces are silently filtered out
- Ensures users only receive watch events for namespaces they have
access to
### Additional fixes
- Fixed ServiceAccount subject handling when namespace is empty
(defaults to RoleBinding namespace)
- Added proper error logging in Watch handler
## Performance Impact
| Handler | Before | After |
|---------|--------|-------|
| List() | List all RoleBindings × 1 | No change ✅ |
| Get() | List all RoleBindings × 1 | List RoleBindings in 1 namespace 🚀
|
| Watch() | List all RoleBindings × N events | List RoleBindings in 1
namespace × N events 🚀 |
**For Watch with 100 events:**
- Before: 100 × (all cluster RoleBindings) = catastrophic
- After: 100 × (1-5 RoleBindings in namespace) = fast + cached by
controller-runtime
## Security Impact
**Before**: Any authenticated user could:
- Read metadata (labels, annotations, creation time) of any `tenant-*`
namespace via `Get()`
- Stream all tenant namespace events via `Watch()`, including
creation/modification/deletion
**After**: Users can only access tenant namespaces they have explicit
RoleBindings for, matching the behavior of the `List()` handler.
## Testing
Manually verified:
- Users can only `get` their own tenant namespaces
- Users can only `watch` events for their own tenant namespaces
- Unauthorized access returns `NotFound` (not `Forbidden`) to prevent
enumeration
- `List()` behavior remains unchanged and consistent with `Get()` and
`Watch()`
- Authorization errors are logged for security audit
## Checklist
- [x] Code follows project style and conventions
- [x] Security vulnerability is fully mitigated
- [x] Authorization logic is consistent across List/Get/Watch handlers
- [x] Performance optimized per feedback from @timofei.larkin
- [x] Error logging added for security audit
- [ ] Unit tests added (can be done in follow-up PR)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enforced per-namespace access control for Get and Watch; inaccessible
namespaces return Forbidden or NotFound as appropriate
* Forwarded field/label selectors to upstream watches and added
defensive filtering to skip inaccessible events (logged)
* Improved ServiceAccount subject namespace fallback and
privileged-group bypass
* **Tests**
* Added security tests covering RoleBinding subjects, groups, privileged
bypasses, ServiceAccounts, and access-denied behaviors
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Add `serverName` field to bootstrap configuration to support backup
recovery when the Barman server name in `backup.info` differs from the
Kubernetes cluster name.
This fixes "no target backup found" errors during CloudNativePG recovery
operations.
## Problem
CloudNativePG forms the backup path as `destinationPath + "postgres-" +
oldName`, but searches for backups using the `server_name` field from
`backup.info`. When these values don't match (e.g., cluster name is
`grafana` but `server_name` is `cloud`), recovery fails with "no target
backup found".
## Solution
- Add optional `serverName` parameter to `bootstrap` configuration
- When specified, CloudNativePG uses this value to search for backups in
S3
- Falls back to `oldName` when `serverName` is not provided (backwards
compatible)
## Changes
- Add `ServerName` field to PostgreSQL CRD type definition
- Add conditional `serverName` to Cluster `externalClusters` template
- Update `values.yaml` and README.md with `serverName` documentation
- Regenerate `values.schema.json` and `postgres.yaml` CRD
## Test plan
- [ ] Deploy postgres cluster with backups enabled
- [ ] Create backup
- [ ] Create new postgres instance with `bootstrap.enabled=true`,
`bootstrap.oldName=<original-name>`, and
`bootstrap.serverName=<value-from-backup.info>`
- [ ] Verify recovery completes successfully
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an optional bootstrap.serverName setting to specify the Barman
server name from the original cluster’s backup.info (used when that
server name differs from the Kubernetes cluster name).
* **Documentation**
* Clarified that bootstrap.oldName must match the serverName value
recorded in backup.info; updated docs and examples to reflect the new
bootstrap.serverName option.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds an opt-in `publishing.exposure=loadBalancer` mode for the
ingress-nginx Service as a migration path away from
`Service.spec.externalIPs`, which is deprecated upstream in Kubernetes
v1.36
([KEP-5707](https://github.com/kubernetes/enhancements/issues/5707),
[kubernetes#137293](https://github.com/kubernetes/kubernetes/pull/137293)).
The `AllowServiceExternalIPs` feature gate is expected to default to off
around v1.40 and the implementation to be removed around v1.43.
Stacked on top of #2464 (cilium v1.19.3 bump) — depends on it for the
`CiliumLoadBalancerIPPool` at `cilium.io/v2`.
### Behavior
- New platform value `publishing.exposure` — enum `externalIPs |
loadBalancer`, default `externalIPs` (current behavior unchanged on
upgrade).
- Plumbed through `cozystack-values` into each tenant's ingress
HelmRelease via the new `expose-mode` key.
- When `exposure=loadBalancer` and the current namespace matches
`publishing.ingressName`, the Service becomes `type: LoadBalancer` with
`externalTrafficPolicy: Local` and a `CiliumLoadBalancerIPPool`
announces the addresses from `publishing.externalIPs` via Cilium LB
IPAM.
- The pool uses a namespace-only `serviceSelector`
(`io.kubernetes.service.namespace: <ns>`) — any LoadBalancer Service in
the tenant namespace draws from it. See "Pool ownership" below.
- IPv4 addresses get `/32` CIDRs, IPv6 addresses get `/128`. Mixed
families supported. Pre-CIDR entries (`192.0.2.10/32`) are accepted
without double-suffixing.
- Unknown values, `loadBalancer` with empty externalIPs, and stray empty
entries from `publishing.externalIPs` are rejected at render time with
explicit error messages.
### Pool ownership
The `CiliumLoadBalancerIPPool` is rendered from
`packages/apps/tenant/templates/cilium-lb-pool.yaml`, not from the
ingress chart. The tenant chart is the per-tenant owner of cross-cutting
resources (Namespace, `cozystack-values` Secret, HelmReleases for
ingress and gateway), so one pool per tenant lives there.
Cilium LB IPAM rejects overlapping CIDRs across pools regardless of
`serviceSelector` — the last-added pool gets `cilium.io/PoolConflict`
and stops allocating. Keeping the pool in the ingress chart would
collide with the Gateway-API PR (#2470), which materialises its own
LoadBalancer Service from the same `publishing.externalIPs` range when
`tenant.spec.gateway=true`. Moving the pool to the tenant chart with a
namespace-only selector lets a single pool back both services
(ingress-nginx today, a Cilium Gateway Service once #2470 lands).
Only the ingress-loadBalancer signal is wired here
(`_cluster.expose-mode=loadBalancer` + `.Values.ingress=true` +
publishing tenant). #2470 rebases on top of this PR, drops its own
`packages/extra/gateway/templates/cilium-lb-pool.yaml`, and adds an OR
branch for `.Values.gateway` in the tenant template.
### Scope
Only the ingress-nginx Service is migrated by this setting. Other
cozystack components that still write `Service.spec.externalIPs`
directly (notably the `vpn` app at
`packages/apps/vpn/templates/service.yaml`) need separate follow-up
before Kubernetes v1.40.
### Tests
- `packages/extra/ingress/tests/exposure_test.yaml` — 10 helm-unittest
cases on the ingress-nginx Service: type / externalTrafficPolicy /
externalIPs assertion for both modes, unknown-mode rejection
(case-sensitive enum), empty-IPs failure, empty-entry filtering,
non-publishing-tenant fallback.
- `packages/apps/tenant/tests/exposure_test.yaml` — 9 cases on the pool
itself: IPv4, IPv6, mixed, pre-CIDR input, empty-entry filtering,
`ingress=false` in publishing tenant (no pool), non-publishing tenant
(no pool), empty externalIPs (no pool).
- Both suites are auto-discovered by `hack/helm-unit-tests.sh` via the
`test:` target in each package's Makefile.
### Caveats (copied from the inline `values.yaml` comment)
- `loadBalancer` mode uses `externalTrafficPolicy: Local`. The external
IP must already be routed to a node that hosts an ingress pod (floating
IP / upstream router / podAntiAffinity).
- Cilium does not announce the IP on its own unless L2 announcements or
BGP are enabled in the Cilium values (disabled by default in cozystack).
- Switching the value on a running cluster causes the ingress-nginx
Service to be recreated (`upgrade.force: true` on the HelmRelease +
Service kind change); expect a brief interruption of ingress traffic.
### Release note
```release-note
feat(ingress): add opt-in publishing.exposure=loadBalancer mode that uses type: LoadBalancer + CiliumLoadBalancerIPPool instead of the deprecated Service.spec.externalIPs. Default (externalIPs) preserves existing behavior.
```
## What this PR does
Closes a bootstrap race where the `cnpg-webhook-service` Service gets
its EndpointSlice populated and the data plane (kube-proxy / Cilium)
programmed a second or two after `helm install --wait` on
`cozy-postgres-operator` declares the HelmRelease Ready. Any HelmRelease
that `dependsOn: postgres-operator` and creates a
`postgresql.cnpg.io/v1` resource in that window (cozy-keycloak, tenant
Postgres apps) has kube-apiserver's call to `mcluster.cnpg.io` fail with
```
Internal error occurred: failed calling webhook "mcluster.cnpg.io":
Post "https://cnpg-webhook-service.cozy-postgres-operator.svc:443/...":
dial tcp <svc-ip>:443: connect: connection refused
```
which fails the downstream release's install.
The fix is a post-install / post-upgrade Helm hook (ServiceAccount +
ClusterRole + ClusterRoleBinding + Job) that probes `/readyz` on the
webhook through the apiserver service proxy. Apiserver's service proxy
uses the same endpoint-resolution and apiserver-initiated pod dial as
the admission webhook path, so once `/readyz` answers through the proxy,
the subsequent admission call will also succeed. Helm `--wait` blocks
the install from completing until the Job exits 0, so the HelmRelease
Ready condition does not lie anymore.
Hardened per review:
- RBAC scoped to a single `services/proxy` resourceName
(`https:cnpg-webhook-service:webhook-server`) — the exact string the
apiserver URL path parser expects for the Service proxy subresource.
- A drift-guard helm-unittest test renders the vendored cnpg Service
template and fails if its `metadata.name` / `ports[0].name` diverge from
the literals in the hook, so a future `make update` that renames the
service forces this template to be updated in the same change.
- Image digest-pinned (`clastix/kubectl:v1.32@sha256:…`) with a
`renovate:` annotation; `imagePullPolicy: IfNotPresent` when
digest-pinned, `Always` when tag-only.
- Job runs under PSA-restricted-compatible securityContext (non-root,
seccomp RuntimeDefault, readOnlyRootFilesystem, drop ALL caps).
- `activeDeadlineSeconds` is derived from `maxAttempts × sleepSeconds +
60s` so a values override raising retries does not get silently cut by a
fixed deadline.
- `backoffLimit: 2` (configurable) so a transient pod-level failure
(image pull rate limit, OOM, CNI hiccup) does not fail the whole
HelmRelease.
- On timeout the Job prints the last `kubectl get --raw` stderr so the
operator can distinguish DNS / refused / 401 / TLS from the Job logs.
21 helm-unittest assertions cover ordering, RBAC/URL parity, subchart
drift, image policy, retry-loop bounds, securityContext, and the
deadline-scaling invariant. Wired via `make test` in
`packages/system/postgres-operator/Makefile`.
First surfaced on #2470 E2E run 24862782568 where
`cozy-keycloak/keycloak` failed with the exact signature above. Not tied
to that PR's branch — this is independently applicable to `main`.
### Release note
```release-note
fix(postgres-operator): add a post-install readiness gate that blocks the HelmRelease from reporting Ready until the cnpg admission webhook actually serves through the cluster Service, preventing "dial tcp <svc-ip>:443: connect: connection refused" on the first HelmRelease that depends on postgres-operator (keycloak, tenant Postgres apps).
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added webhook readiness validation during install/upgrade to block
completion until the admission webhook is reachable.
* New configuration options for the readiness probe image and
retry/timeouts.
* **Tests**
* Introduced a comprehensive test suite validating rendered manifests,
probe behavior, RBAC, image rendering, and retry/timeout logic.
* **Chores**
* Added a test entry point to the project Makefile to run chart/unit
tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Collapse the chart to a single Deployment shape that mirrors
cozystack-api: 2 replicas, soft (preferred) nodeAffinity to
node-role.kubernetes.io/control-plane via Exists, permissive
tolerations, soft podAntiAffinity on hostname, an unconditional
PodDisruptionBudget with maxUnavailable: 1, and a Service with
spec.trafficDistribution: PreferClose. The same shape works on
Talos / kubeadm / k3s and on managed Kubernetes / Cozy-in-Cozy
tenant clusters without per-distro overrides — fixing #2417 by
making the soft control-plane affinity gracefully fall back to
worker scheduling when no control-plane nodes are visible.
Drop the DaemonSet path entirely. The previous PR's deployment.enabled
toggle, the workload-kind switch in templates/workload.yaml, the
fail-guard for localK8sAPIEndpoint+nodeAffinity=[], the conditional
PDB, and the values-shape knobs for nodeAffinity and tolerations all
go away as a consequence. Override the standard Deployment fields
through the usual component-values mechanism if a non-default topology
is ever needed.
Mark localK8sAPIEndpoint.enabled deprecated and flip the default to
false. The flag injects KUBERNETES_SERVICE_HOST=status.hostIP, which
is only valid when the pod is actually scheduled on an apiserver-
bearing node. With the new soft control-plane affinity, the pod can
land off-control-plane and crash-loop. The latency motivation for the
flag is real but pending separate webhook performance work; once
addressed, the flag can be removed.
Revert all changes to packages/core/platform/images/migrations/migrations/20.
The earlier ds/...-or-deploy/... fallback was over-engineered: per
run-migrations.sh, migration 20 fires only when CURRENT < 20 (i.e.
direct upgrades from pre-0.37 to 1.3+), and that path is unsupported
anyway. The original ds/... rollout-status line is dead code on every
supported install and upgrade path.
Add helm-unittest coverage for: the default Deployment shape (replicas,
soft nodeAffinity, soft podAntiAffinity, tolerations); no env vars at
the default localK8sAPIEndpoint.enabled=false; PDB rendered
unconditionally with maxUnavailable: 1, including at replicas=1 (the
no-op case); replicas value drives spec.replicas; and that
localK8sAPIEndpoint.enabled=true does inject the env vars. The Service
test continues to assert trafficDistribution: PreferClose with no
internalTrafficPolicy.
Add a slim README documenting the topology, parameters, and the
deprecation note for localK8sAPIEndpoint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
- Hoist user identity extraction out of the Watch goroutine; reuse a
cached username and groups map across events instead of re-fetching
them per event. Watch now returns Unauthorized up front when no user
is present in the context, rather than failing silently per event.
- Switch the per-event access-check error log to structured klog.ErrorS
to comply with the project Go style guide.
- Strengthen TestGet_WithAccess to assert the concrete *TenantNamespace
type plus Name, Kind, and APIVersion, so type or metadata regressions
fail fast.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
The `linstor-gui` package (added in #2382) was never wired into the root
`Makefile`'s `build:` target, so the image is not built or published by
CI. `ghcr.io/cozystack/cozystack/linstor-gui` returns `NAME_UNKNOWN`
from the GHCR API, and `values.yaml` still pins `tag: 2.3.0` without a
digest because the per-package Makefile that rewrites the tag after a
successful push has never run in CI.
This PR adds the missing line so the next build publishes the image and
digest-pins `values.yaml` automatically.
### Release note
```release-note
build(linstor): include linstor-gui in root image build target so the image is built and published by CI (the chart previously referenced an image that did not exist in the registry)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Integrated the linstor-gui container image build into the main image
build workflow so the GUI image is produced as part of standard builds.
* Streamlined the GUI image build configuration to use consolidated
build arguments, improving consistency and maintainability of automated
image builds and metadata updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Bumps the vendored cozy-proxy chart and image tag from v0.2.0 to v0.3.0
by running `make update` in `packages/system/cozy-proxy`.
cozy-proxy v0.3.0 release:
https://github.com/cozystack/cozy-proxy/releases/tag/v0.3.0
## Why
cozy-proxy v0.3.0 is the companion of #2501. The chart fix in #2501
finally honors the documented PortList semantics (only `externalPorts`
reachable, ICMP preserved by default), but only takes effect once
cozy-proxy v0.3.0 is in place — v0.2.0 has no port-aware logic, so the
chart fix silently no-ops on it. This PR ships that runtime side.
## Upgrade impact
For cozystack users this is a **bug fix, not a breaking change**.
cozy-proxy v0.3.0 introduces two contract changes upstream — (1)
label-only selector via `service.kubernetes.io/service-proxy-name:
cozy-proxy`, and (2) port-filter as the default ingress mode — both of
which are absorbed by the vm-instance chart (label landed in #2357,
`wholeIP` + `allowICMP` wired in #2501).
- **`externalMethod: WholeIP`** (default) — chart renders label +
`wholeIP: "true"` in both old and new versions. cozy-proxy keeps
passthrough. **No change.**
- **`externalMethod: PortList`** — previously the chart always rendered
`wholeIP: "true"` regardless of method, and cozy-proxy v0.2.0 had no
port-aware logic, so PortList was silently a no-op (all ports
reachable). After this bump + #2501, PortList behaves as documented:
only ports listed in `externalPorts` reach the VM. Users who configured
PortList but reached undeclared ports (relying on the silent no-op) will
see those ports become unreachable — that's the documented intent of
PortList finally taking effect. ICMP keeps working by default thanks to
`externalAllowICMP: true`.
- **DaemonSet rollover** — re-init of the nft table during cozy-proxy
pod restart is a brief blip on existing flows; same as any DaemonSet
upgrade.
Out-of-tree consumers using cozy-proxy directly (without the
`service-proxy-name` label, or relying on the absent-annotation
passthrough default) need to migrate per the upstream v0.3.0 release
notes — not a cozystack concern.
## Order of operations
This PR is functionally complete on its own (image bump), but the
user-facing PortList fix it enables only happens when #2501 also lands.
Recommended merge order:
1. #2501 first (chart adapts to the new contract)
2. This PR second (cozy-proxy DaemonSet picks up the new image)
The reverse order is also safe — cozy-proxy v0.3.0 with the unmodified
vm-instance chart still produces correct results for `externalMethod:
WholeIP` (most common case) and keeps PortList silently broken until
#2501 lands.
## Test plan
- [ ] `helm template packages/system/cozy-proxy` — image tag `v0.3.0`
rendered
- [ ] `make unit-tests` passes
- [ ] After #2501 lands, end-to-end: `externalMethod: PortList` with
`externalPorts: [22]` filters all other ports, ping works
- [ ] WholeIP-method VM Service unchanged after upgrade
## Companion PRs
- #2501 — vm-instance chart wires `wholeIP` / `allowICMP` per
`externalMethod`
- cozystack/cozy-proxy#13 (merged) — label-only selector + port-filter
default
- cozy-proxy v0.3.0 release notes —
https://github.com/cozystack/cozy-proxy/releases/tag/v0.3.0
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Released version 0.3.0 with updated Helm chart metadata and container
image configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes the `post-upgrade` Helm hook in `packages/extra/etcd` that was
`kubectl delete`ing the etcd TLS chain (`etcd-ca-tls`,
`etcd-peer-ca-tls`,
`etcd-client-tls`, `etcd-peer-tls`, `etcd-server-tls`) and then deleting
etcd pods on every chart upgrade.
The hook was gated by a semver compare of a stored
`etcd-deployed-version`
ConfigMap against `2.6.1`. That gate was written when chart versions
looked
like `2.6.0`, `2.6.1`, etc. Since commit `f871fbdb` ("Remove
versions_map
logic") all chart versions are stamped as `0.0.0+<git-hash>`, which per
semver is always `< 2.6.1`, so the gate always evaluates to "regenerate
certs" and the destructive hook fires on every upgrade.
On clusters running Kamaji-managed tenant control planes the consequence
is
severe: wiping the etcd CA triggers cert-manager to re-issue it with a
brand-new CA, but Kamaji's `datastore-certificate` Secrets still carry
the
old CA bundle mounted into tenant `kube-apiserver` pods. Those
apiservers
hit `x509: certificate signed by unknown authority` against
`etcd.<ns>.svc:2379` and go into CrashLoopBackOff until each tenant
`DataStore` is individually force-reconciled and the Deployments rolled.
Commit `47d81f70` ("Disabled private key rotation in CA certs") already
fixed the underlying `rotationPolicy: Always` issue the hook was written
to
paper over in March 2025, so the migration has no remaining use.
Changes:
- Delete `templates/hook/{job,role,rolebinding,serviceaccount}.yaml`
- Delete `templates/version.yaml` (only the hook read
`etcd-deployed-version`)
- Add `tests/no-post-upgrade-hook_test.yaml` as a regression guard — if
`templates/hook/job.yaml` or `templates/version.yaml` is ever brought
back,
`make unit-tests` fails
### Release note
```release-note
fix(etcd): remove destructive post-upgrade cert-regeneration hook. Previously the etcd chart ran a `post-upgrade` hook on every Helm upgrade that deleted etcd TLS Secrets and pods, causing cert-manager to re-issue the etcd CA. On clusters with Kamaji-managed tenant control planes this put every tenant `kube-apiserver` into CrashLoopBackOff until each DataStore was manually re-reconciled. The hook was a one-shot `2.6.0 -> 2.6.1` migration that became a permanent footgun once chart versioning moved to `0.0.0+<git-hash>` and the underlying `rotationPolicy: Always` issue was fixed in commit `47d81f70`. This is behavior removal, not cosmetic cleanup.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added Helm chart test suite verifying absence of post-upgrade hook Job
and version ConfigMap.
* **Chores**
* Removed post-upgrade hook Job and associated RBAC resources (Role,
RoleBinding, ServiceAccount).
* Removed version ConfigMap.
* Added `test` target to Makefile for running Helm chart unit tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review feedback from sircthulhu and CodeRabbit:
1. Return Forbidden instead of NotFound for unauthorized access
- Get() now returns 403 Forbidden to follow standard K8s RBAC behavior
- Previously returned 404 NotFound for security-by-obscurity
- Updated test expectations to match new behavior
2. Propagate field and label selectors in Watch handler
- Pass opts.FieldSelector and opts.LabelSelector to upstream Watch
- Add defensive filtering before authorization to prevent RBAC bypass
- Fixes potential issue with resourceNames restrictions
3. Refactor subject-matching logic to eliminate duplication
- Extract matchesSubject() helper for Group/User/ServiceAccount checks
- Remove duplicated code from filterAccessible and hasAccessToNamespace
- Consolidates ServiceAccount namespace fallback logic
All tests pass successfully.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
The `vm-instance` chart now drives the cozy-proxy `wholeIP` and
`allowICMP` annotations explicitly so that `externalMethod: PortList`
actually filters ingress traffic to declared ports while keeping
ping/PMTU functional.
- Render `networking.cozystack.io/wholeIP: "false"` when
`externalMethod: PortList` (was always `"true"`, which silently disabled
the PortList semantics in cozy-proxy).
- Add `externalAllowICMP` value (default `true`) propagated as
`networking.cozystack.io/allowICMP` when `externalMethod: PortList`.
Without this, cozy-proxy drops ICMP in port-filter mode (ping/PMTU
broken). Operators can set `externalAllowICMP: false` to opt out.
The changelog entry is intentionally **not** part of this PR — it will
be added in a dedicated `docs: add changelog for vX.Y.Z` commit at
release time, per project convention.
## Why
`externalMethod: PortList` is documented as filtering ingress traffic to
declared ports but has been non-functional on Cozystack v1.3.0 —
verified empirically on a 3-node Talos lab. Root cause was twofold:
chart always set `wholeIP: "true"`, and cozy-proxy v0.2.0 had no
port-aware logic. The cozy-proxy side was fixed in
cozystack/cozy-proxy#11 (merged) and cozystack/cozy-proxy#12 (allowICMP
follow-up); this PR completes the user-visible fix on the chart side.
## Companion PRs
- cozystack/cozy-proxy#11 (merged) — per-service ingress port filtering
- cozystack/cozy-proxy#12 (merged) — `allowICMP` annotation for
port-filter mode
## Test plan
- [x] Built cozy-proxy with the companion fix locally, deployed on a
3-node Talos lab (Cozystack v1.3.0)
- [x] `wholeIP: "false"` Service with `spec.ports: [22]`: only port 22
reachable from outside; ports 80/443/8080/9999 filtered
- [x] WholeIP-annotated Service unchanged: all listening ports reachable
- [x] Egress IP preservation works in both modes (TCP curl + UDP DNS)
- [x] `nft list table ip cozy_proxy` confirms expected ruleset
- [x] `helm template` renders the expected annotation matrix: `PortList`
default → `wholeIP=false, allowICMP=true`; `PortList` opt-out →
`allowICMP=false`; `WholeIP` → only `wholeIP=true`
- [x] `make unit-tests` passes locally
- [ ] CI unit tests
- [ ] CI E2E
## Backport
Suggesting `backport-v1.3` once merged.
## Release note
```release-note
[vm-instance] Make `externalMethod: PortList` actually filter ingress traffic to ports listed in `externalPorts`. New `externalAllowICMP` knob (default true) propagates the cozy-proxy `allowICMP` annotation to keep ping/PMTU functional in port-filter mode. Combined with cozy-proxy v0.3.0+, only listed ports plus ICMP are reachable from the VM's LoadBalancer IP.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added `externalAllowICMP` configuration option to control ICMP traffic
acceptance for VM external access in PortList mode (enabled by default).
* **Documentation**
* Updated parameter documentation to include the new ICMP traffic
control setting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Pulls in the per-port filtering and allowICMP support that the companion
vm-instance chart fix in #2501 relies on. cozy-proxy v0.3.0 also tightens
the selector to the standard service.kubernetes.io/service-proxy-name=cozy-proxy
label and switches the default ingress mode to port-filter; both are
already covered by the vm-instance chart (label landed in #2357,
wholeIP/allowICMP wired explicitly in #2501), so VM workloads upgrade
transparently.
Out-of-tree consumers using cozy-proxy annotations directly (without the
label, or relying on the absent-annotation passthrough default) are
called out in the upstream v0.3.0 release notes:
https://github.com/cozystack/cozy-proxy/releases/tag/v0.3.0
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Address review feedback from coderabbitai on packages/system/monitoring-agents/alerts/gpu-recording.rules.yaml:119
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Address review feedback from coderabbitai on packages/system/gpu-operator/examples/nvidia-driver-compat.yaml:77
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
- Adds a pre-commit hook and a dedicated GitHub Actions workflow that
run `make generate` at the repo root to catch drift in generated API
code (CRDs, DeepCopy, clients, RBAC) before it lands on main.
- Pre-commit hook is scoped to paths that actually affect codegen
(`api/`, `pkg/apis/`, `hack/update-codegen.sh`,
`hack/boilerplate.go.txt`) so unrelated commits are not slowed down.
- CI workflow (`.github/workflows/codegen-drift.yml`) installs Go from
`go.mod`, runs `make generate`, and fails on drift with an error
pointing contributors to the local fix.
- Includes one drift fix the check surfaced: `RestoreJobSpec.Options`
(added in #2437) had no `DeepCopyInto` handling — regenerated.
### Release note
```release-note
ci(api): add pre-commit hook and GitHub Actions workflow that verify generated API code (CRDs, deepcopy, clients, RBAC) is in sync with `make generate`.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a CI workflow that checks generated code during PRs and fails
the build if generated artifacts diverge.
* Added a pre-commit hook that runs generation checks locally to prevent
committing outdated generated files.
* Fixed deep-copy behavior for backup restore job specs so nested
options are correctly duplicated.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Add `externalAllowICMP` value (default true) propagated as
`networking.cozystack.io/allowICMP` annotation on the rendered Service
when `externalMethod: PortList`. The cozy-proxy companion (released as
part of cozystack/cozy-proxy#11 + #12) drops ICMP by default in
port-filter mode, which breaks ping and PMTU discovery; defaulting the
chart to "true" preserves user expectations while still allowing
operators to opt out by setting `externalAllowICMP: false`.
- Remove the v1.3.1.md changelog entry. Project convention is to add
changelogs in a dedicated "docs: add changelog for vX.Y.Z" commit at
release time, not as part of feature/fix PRs.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
## What this PR does
Bumps `linstor-csi` from v1.10.5 to v1.10.6 and ships an out-of-tree
patch that fixes live migration of KubeVirt VMs whose volumes sit on a
DRBD Protocol-A/B resource (e.g. a `replicated-async` StorageClass).
### Problem
DRBD requires Protocol C whenever `allow-two-primaries=yes` is enabled.
Operators commonly opt into Protocol A on a per-resource-group basis for
async / WAN replication, which silently breaks every subsequent live
migration of consumers of those volumes: `drbdadm adjust` rejects the
second-attach with `Protocol C required` (errno 139), KubeVirt's
evacuation loop retries indefinitely, and the VM stays pinned to the
source node.
### Change
- `LINSTOR_CSI_VERSION` 1.10.5 → 1.10.6 (Makefile + Dockerfile default).
- New patch `002-protocol-c-override-for-dual-attach.diff`: when
`Attach` installs `allow-two-primaries=yes` on the resource-definition
during a second attach, it also installs `DrbdOptions/Net/protocol=C` as
an override on the resource-definition. The override applies to every
connection (including diskless TieBreaker peers, where a per-pair
override would still leave one connection broken). It is tagged with
`Aux/csi-protocol-override=yes` so `Detach` removes only the override
this driver installed, leaving any operator-set `Protocol` property on
the resource-definition untouched.
- Existing patch `001-relocate-after-clone-restore.diff` regenerated
against v1.10.6 (context shift only, no logic change — the old patch
hunks no longer aligned cleanly).
### Verification
- `make image-linstor-csi` builds successfully on linux/amd64 with both
patches applied.
- End-to-end test on dev5 cluster (KubeVirt v1.6.3, 3-node Talos):
created a Protocol-A resource-group + StorageClass, provisioned a VM on
top, and triggered live migration. Migration succeeds in a single Attach
with the override installed during dual-attach and removed by Detach.
Reproducer (without the patch) is the well-known evacuation loop with
`(node) Failed to adjust DRBD resource ... Protocol C required`.
### Upstream
Upstreamed as draft PR piraeusdatastore/linstor-csi#435.
### Compatibility
- No behaviour change for resources already using Protocol C (the common
case).
- No behaviour change for resources never attached with
allow-two-primaries.
- Idempotent: re-running `Attach` is a no-op once the override is
installed.
- Operator-set Protocol overrides on the resource-definition are
preserved (gated by the Aux marker).
### Release note
```release-note
fix(linstor): live migration of KubeVirt VMs on Protocol-A/B (async) DRBD volumes no longer fails with "Protocol C required" — linstor-csi now installs a Protocol=C override on the resource-definition during dual-attach and reverts it on detach.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Automatic replica relocation after volume clone and snapshot restore
to improve placement and load distribution.
* Conditional DRBD protocol override to enable/clean up dual-attach
(two-primaries) scenarios more reliably.
* **Chores**
* Updated LINSTOR CSI default to v1.10.6.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Render `networking.cozystack.io/wholeIP: "false"` on the Service when
`externalMethod: PortList` is configured (was always `"true"` before).
Combined with cozy-proxy v0.3.0+ which adds per-port filtering for
"false"-annotated services, this makes `externalMethod: PortList`
behave as documented: only ports listed in `externalPorts` are
reachable from the LoadBalancer IP.
Backward-compatible: existing services with `externalMethod: WholeIP`
continue to set `wholeIP: "true"` and behave identically. cozy-proxy
versions older than v0.3.0 ignore Services with `wholeIP: "false"`,
which means PortList Services on older cozy-proxy will lose their
egress IP preservation — but that path was already non-functional
ingress-wise, so this is not a regression for users actually relying
on PortList.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Bump linstor-csi to v1.10.6 and add an out-of-tree patch that overrides
DrbdOptions/Net/protocol to C on the resource-definition whenever a
volume is attached to a second node with allow-two-primaries (the
live-migration flow), and reverts it on detach.
Without the patch, any KubeVirt live migration of a VM whose volume
sits on a Protocol-A/B resource (for example a "replicated-async"
StorageClass) ends up in a permanent evacuation loop: drbdadm adjust
on the satellite rejects allow-two-primaries with "Protocol C
required" (errno 139), the migration fails, and KubeVirt retries
until manual intervention.
The override is set at the resource-definition level so it covers
every connection, including diskless TieBreaker peers, and is tagged
with an Aux marker so Detach reverts only the override we installed.
The 001 relocate-after-clone patch was regenerated against v1.10.6
(context shift only, no logic change).
Verified end-to-end on dev5: live migration of a KubeVirt VM whose
PVC sits on a Protocol-A resource-group now succeeds in a single
Attach.
Patch is upstreamed as draft PR piraeusdatastore/linstor-csi#435.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Closes#2412. On a cold tenant-Kubernetes bootstrap, the parent
HelmRelease raced the admin-kubeconfig Secret that Kamaji provisions
asynchronously. Three CP-side Deployments (cluster-autoscaler, kccm,
kcsi-controller) mounted that Secret as a hard volume, flux
helm-controller's default wait budget was too short for Kamaji cold
start, and `install.remediation { retries: -1 }` then uninstalled the
Cluster CR and restarted the cycle forever.
Implements a defense-in-depth fix:
- `optional: true` on the admin-kubeconfig Secret volume in all three
Deployments so kubelet no longer FailedMounts while Kamaji is still
bootstrapping.
- A shared `wait-for-kubeconfig` init container (in
`templates/_helpers.tpl`) that polls for `super-admin.svc` with a 10m
deadline, strictly below the HelmRelease Install.Timeout so a broken
tenant falls into CrashLoopBackOff visibly instead of hanging forever.
- Per-Application HelmRelease Install/Upgrade timeout, driven by a new
`release.cozystack.io/helm-install-timeout` annotation on
ApplicationDefinition. Kubernetes-rd sets it to `15m`; other kinds leave
it unset and keep flux defaults, so their failed installs remediate on
the normal cadence. Parser rejects ns/us/µs (accepted by
`time.ParseDuration`, rejected by Flux's CRD pattern) at startup.
- Soft-skip when `_namespace.etcd` is empty: the CP-side Deployments,
the Cluster/KamajiControlPlane/KubevirtCluster/WorkloadMonitor CRs, and
every child HelmRelease that references admin-kubeconfig now render only
when an etcd DataStore exists for this tenant. An `awaiting-etcd`
ConfigMap is emitted as a user-visible status beacon so `helm install`
still succeeds and flux retries on its 5m interval until the Tenant
chart catches up.
- e2e remediation guard built on `.status.history[].status` (the
Snapshot shape), not on `.status.installFailures` - `ClearFailures()`
zeroes the latter on every successful reconciliation, which made the
previous guard vacuous.
Tests:
- Go unit tests for the annotation parser (accepted/rejected units) and
the HR builder (table-driven across kinds).
- helm unittest for the per-template structure (optional volume, init
container, dataStoreName, awaiting-etcd beacon).
- bats unit tests for the shell guard (every combination of
empty/zero/positive history entries, plus pinned HR v2 shape).
- Chart-wide bats invariants: every Deployment mounting admin-kubeconfig
has the guards; zero such Deployments and zero HelmReleases render when
etcd is empty.
All wired into the existing `make unit-tests` target (`go-unit-tests`
added alongside `helm-unit-tests` and `bats-unit-tests`).
Option 2 from the ticket (separate HelmRelease with `dependsOn`) was
intentionally not taken: the combination above closes the same race
without restructuring the chart's HelmRelease topology.
### Release note
```release-note
fix(kubernetes): close admin-kubeconfig race on tenant Kubernetes bootstrap. The parent HelmRelease no longer enters an uninstall/retry cycle when Kamaji control-plane cold start exceeds flux's default wait budget. A Kubernetes tenant created before the parent Tenant application has etcd enabled now renders only an awaiting-etcd beacon ConfigMap and waits quietly for the DataStore to appear, instead of producing half-installed Deployments that CrashLoopBackOff forever.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Per-application Helm install/upgrade timeout via metadata annotation.
* Init-container guards that wait for admin kubeconfig before workloads
start.
* Chart resources now render conditionally based on etcd presence.
* **Tests**
* Helm-template tests for admin-kubeconfig invariants and
remediation-cycle detection.
* New Go unit tests and CI Helm/unittest coverage plus test value files.
* **Chores**
* Added BusyBox image pin and new Makefile test targets (including Go
unit-tests).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The per-package Makefile added in #2382 hardcoded buildx flags
(--provenance, --builder, --platform=linux/amd64,linux/arm64,
--push, --load, --label) instead of using the shared $(BUILDX_ARGS)
macro from hack/common-envs.mk.
This broke CI: the runner's default docker driver does not support
multi-platform builds, and the hardcoded multi-arch platform list
crashed `make build` with "Multi-platform build is not supported for
the docker driver."
Replace the hardcoded flags with $(BUILDX_ARGS) to match every other
package (e.g. linstor, dashboard, cilium). $(BUILDX_ARGS) injects
--push, --load, --label, --provenance=false, and only sets --builder
or --platform when the operator explicitly exports BUILDER/PLATFORM.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The linstor-gui package was added in #2382 with its own per-package
Makefile and Dockerfile, but the root Makefile's `build:` target was
not updated to invoke it. As a result `ghcr.io/cozystack/cozystack/
linstor-gui` has never been published (registry returns NAME_UNKNOWN)
and the chart's `image.tag` was never digest-pinned. Any cluster
deploying the chart hits ImagePullBackOff.
Wire the package into the root build alongside the other system
images. The next CI build will publish the image and the per-package
Makefile will rewrite values.yaml `image.repository`/`image.tag` to a
digest-pinned reference automatically.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The cozystack.io/status-beacon: "true" annotation had no consumer in
the chart, no documented contract, and no convention defined for other
charts to follow. It would have become accidental precedent for
contributors copying the pattern without understanding it.
The ConfigMap itself is self-explanatory: the name <release>-awaiting-etcd,
data.status: "awaiting-etcd", and the human-readable message in
data.message all surface the same operator signal via kubectl get cm.
Drop the annotation; keep the ConfigMap.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Operators in air-gapped or rate-limited environments cannot reach
docker.io and the bundled busybox digest pin gives them no escape
hatch. Add an optional images.waitForKubeconfig chart value that, when
set, replaces the helper's image reference with any registry path
kubelet can pull. Empty value falls back to images/busybox.tag, so the
prior digest-pinned default is preserved.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Wrap the values: block in a conditional so the key is omitted entirely
when no defaults or overrides produce content. Previously the template
always emitted values: null, triggering unnecessary FluxCD reconciliation.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Use nindent instead of indent with leading whitespace to prevent
broken YAML rendering when devicePlugin.service.labels is set.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
OnDelete requires manual pod deletion to apply updates. RollingUpdate
with maxUnavailable constraint matches upstream default and is consistent
with all other system packages.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Adopt CNCF/Kubernetes label conventions for issues and PRs and add
automated labeling.
**Canonical label file**: `.github/labels.yml`. Synced into the
repository by `.github/workflows/labels.yaml` (EndBug/label-sync@v2) on
push to `main`, weekly cron, and manual dispatch. UI-only label edits
are overwritten — propose changes via PR to this file.
### Label namespaces
Following the [Kubernetes label
scheme](https://github.com/kubernetes/test-infra/blob/master/label_sync/labels.md):
- `kind/*` — issue or PR type (bug, feature, documentation, support,
cleanup, regression, flake, failing-test, api-change, breaking-change)
- `priority/*` — urgency (critical-urgent, important-soon,
important-longterm, backlog)
- `triage/*` — review state (needs-triage, accepted, needs-information,
not-reproducible, duplicate, unresolved)
- `lifecycle/*` — issue/PR lifecycle (active, frozen, stale, rotten)
- `area/*` — subsystem; 15 seeded plus `area/uncategorized` fallback.
Extensible — propose a new area when no existing one fits.
- `do-not-merge/*` — PR merge blockers (work-in-progress, hold)
Cozystack-specific labels preserved: `epic`, `community`, `help wanted`,
`good first issue`, `quality-of-life`, `upstream-issue`, `backport`,
`backport-previous`, `release`, `automated`, `debug`, `sponsored`,
`lgtm`, `ok-to-test`, `security/*`, `size:*`.
### `area/*` set
15 areas seeded by activity in open issues and PRs: `area/ai`,
`area/api`, `area/build`, `area/ci`, `area/dashboard`, `area/database`,
`area/extra`, `area/kubernetes`, `area/monitoring`, `area/networking`,
`area/platform`, `area/release`, `area/storage`, `area/testing`,
`area/virtualization`. Plus `area/uncategorized` as the auto-labeler
fallback.
### Migration safety
Existing labels are renamed via `aliases:` in `labels.yml`. GitHub
preserves the label ID, so all currently tagged issues and PRs keep
their tags under the new name without losing references:
| Old | New |
|---|---|
| `bug` | `kind/bug` |
| `enhancement` | `kind/feature` |
| `documentation` | `kind/documentation` |
| `question` | `kind/support` |
| `frozen` | `lifecycle/frozen` |
| `stale` | `lifecycle/stale` |
| `duplicate` | `triage/duplicate` |
| `do-not-merge` | `do-not-merge/work-in-progress` |
| `do not merge` | `do-not-merge/work-in-progress` |
`delete-other-labels: false` on the initial rollout. Generic
GitHub-default labels (`wontfix`, `invalid`) are preserved untouched and
will be removed in a follow-up cleanup PR. EndBug processes aliases
sequentially, so the second of the two `do-not-merge*` aliases hits a
name collision and logs a warning — the legacy label survives that one
sync and is cleaned up in the same follow-up.
### PR auto-labeling
`.github/workflows/pr-labeler.yaml` parses each PR title on `opened`,
`edited`, `reopened`, and `synchronize` and applies labels additively
(never removes):
- **type → `kind/*`**: feat, fix, docs, chore, refactor (others get no
kind)
- **scope → `area/*`**: scope mapping covers all current cozystack
components (full table in `docs/agents/contributing.md`)
- **`!` after type or `BREAKING CHANGE:` footer**: applies
`kind/breaking-change`
- **`[Backport release-1.x]` prefix**: stripped before parsing;
`area/release` and `backport` labels added
- **Composite scope** (`feat(platform, system, apps): …`): each part
mapped independently
- **Bracket fallback** (`[scope] description`): maps `area/*` but cannot
infer `kind/*`
- **Unmapped scope or non-conventional title**: applies
`area/uncategorized` for human review
### Schema validation
`.github/workflows/labels.yaml` runs a `validate` job on every PR
touching `labels.yml` or its workflow. Asserts:
- description ≤ 100 chars (GitHub REST API limit)
- color is 6-char hex without leading `#`
- unique top-level names
- aliases do not collide with top-level names
Sync runs only on push to main, weekly cron, and manual dispatch; PR
runs validate-only.
### Hardcoded label/title references updated
- `.github/ISSUE_TEMPLATE/bug_report.md`: `labels: 'bug'` → `labels:
'kind/bug'`
- `.github/workflows/tags.yaml`:
- changelog PR labels `['documentation', 'automated']` →
`['kind/documentation', 'automated']`
- release PR title `Release v${version}` → `chore(release): cut
v${version}` (so the auto-labeler applies `kind/cleanup` +
`area/release`)
- changelog PR title `docs: add changelog for v${version}` →
`docs(release): add changelog for v${version}` (so the auto-labeler
applies `kind/documentation` + `area/release`)
### Documentation
- `AGENTS.md`: Activation entry pointing agents to `labels.yml` and the
PR title auto-labeling rules. States explicitly that `area/*` accuracy
outweighs reuse — propose a new area when none fits, do not shoehorn
into a wrong one.
- `docs/agents/contributing.md`: PR Title Auto-Labeling section with
type→kind and scope→area tables.
### Out of scope (follow-up PRs)
- Removal of redundant labels (`wontfix`, `invalid`, plus the surviving
legacy `do not merge` if alias-rename collision keeps it)
- Org-wide sync from `cozystack/.github/labels.yml`
- Dosu bot configuration update for `lifecycle/stale` (requires
dashboard access)
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Automated PR labeling from Conventional Commits (type → kind/*, scope
→ area/*), plus a comprehensive namespaced label taxonomy.
* **Chores**
* Workflows to validate, sync, and auto-apply labels (including
scheduled/manual runs and validation checks).
* Added repository-wide label configuration and normalized bug label
metadata to namespaced form.
* Updated release PR titling/labeling conventions.
* **Documentation**
* Contributor and agent guidance on PR title conventions, label
mappings, and triage procedures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review feedback from myasnikovdaniil on .github/workflows/pr-labeler.yaml:160,173:
emit core.warning when a Conventional Commits type has no kind/*
mapping or a scope has no area/* mapping. Without the warning, typos
(e.g., "hotfix" instead of "fix") and recurring new scopes silently
fall through to area/uncategorized, masking that the mapping has
drifted.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from myasnikovdaniil on .github/workflows/pr-labeler.yaml:155:
Conventional Commits 1.0 spec item 16 treats BREAKING CHANGE: and
BREAKING-CHANGE: as synonymous footers. The hyphen form was silently
ignored before, so PRs that use it would miss kind/breaking-change.
https://www.conventionalcommits.org/en/v1.0.0/#specification
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from myasnikovdaniil on .github/workflows/pr-labeler.yaml:129:
defensive (pr.labels || []) avoids TypeError if the webhook payload
arrives without the labels field on edge cases like stripped edited
events.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
DCGM exporter metrics carry the exporter's own namespace
(cozy-gpu-operator), not the workload namespace. Recording rules that
filtered namespace!~"cozy-.*" silently dropped all DCGM series,
producing empty dashboard panels.
Replace namespace-level hardware aggregations with node-level
equivalents (grouped by Hostname), keep namespace-level allocation
rules that use kube_pod_container_resource_requests (which carries the
real workload namespace), and rename pod-level efficiency rules to
gpu-level since DCGM cannot attribute hardware metrics to individual
pods.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Add an opt-in `upgradeCRDs` field to `ComponentInstall` in
`PackageSource`. The field maps directly to
`HelmRelease.Spec.Upgrade.CRDs` so a component author can declare how
Flux should handle CRDs from the chart's `crds/` directory when the
release is upgraded.
The helm-controller default on upgrade is `Skip`, which means CRDs added
by a chart bump never reach clusters that already have the release
installed — they must be applied manually with `kubectl apply --filename
charts/.../crds/`. This surfaces on every upgrade of an operator whose
CRD set expands between versions (etcd-operator, cnpg, kubevirt, kamaji,
etc.).
Setting `upgradeCRDs: CreateReplace` lets Flux apply new CRDs
declaratively with the chart.
Values are restricted to `Skip`, `Create`, `CreateReplace` via a
kubebuilder enum marker. Empty / unset preserves the existing
helm-controller default, so every current `PackageSource` keeps working
unchanged.
Migration is out of scope here — follow-ups will opt individual packages
in case-by-case.
### Relation to existing CRD management approach
The project convention (per #377) is to extract CRDs into a dedicated
Helm chart that reconciles ahead of the operator chart. This PR does not
replace that pattern — it complements it for charts that keep CRDs
inline under `charts/<name>/crds/` where extraction isn't practical
(vendored upstream charts with tightly coupled CRDs). Packages that
already split CRDs out can leave `upgradeCRDs` unset and keep using
their existing separate chart.
### Release note
```release-note
feat(operator): add opt-in `upgradeCRDs` field to `PackageSource` component `install` block to control how CRDs from the chart's `crds/` directory are applied on HelmRelease upgrades (`Skip` by default; use `CreateReplace` for operators whose CRD set expands between versions).
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Configurable CRD upgrade policy for component installs: Skip, Create,
CreateReplace — controls CRD handling during package upgrades and
preserves controller default when unset.
* **Documentation**
* Guidance on CRD upgrade semantics, advice to use CreateReplace for
specific operators, and warning about potential data-loss risks;
clarified contributor scope examples and PR template guidance.
* **Tests**
* Added tests validating CRD policy parsing and presence of the CRD
policy enum in the published schema.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Match the wording adopted in docs/agents/contributing.md so that human
contributors and AI agents see the same guidance in both places.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The Scopes section was read as an exhaustive enumeration, which led to
review feedback flagging any scope outside the list as invalid. The
intent has always been that contributors pick the most specific scope
for the change and extend the list when a genuinely new area appears.
Reword the section accordingly and add operator as an example scope.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Describe when to set upgradeCRDs: CreateReplace (operators that evolve
their CRD set additively between versions) and the data-loss risk of
enabling it on operators that drop fields.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add an opt-in UpgradeCRDs field to ComponentInstall that maps to
HelmRelease.Spec.Upgrade.CRDs, allowing a PackageSource component to
declare how Flux should handle CRDs from the chart's crds/ directory
on upgrade.
The helm-controller default on upgrade is Skip, which means new CRDs
added between chart versions never reach existing clusters and must be
applied manually. Setting upgradeCRDs: CreateReplace makes Flux apply
new CRDs declaratively with the chart.
Allowed values are restricted to Skip, Create, CreateReplace via a
kubebuilder enum marker. Empty / unset preserves the existing Flux
default, so all existing PackageSource resources keep working.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on docs/agents/contributing.md:73:
add space before closing pipe in the type to kind mapping table for
consistency with other rows.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on AGENTS.md:33:
expand bare labels.yml and pr-labeler.yaml to .github/labels.yml and
.github/workflows/pr-labeler.yaml for consistency with surrounding refs.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on docs/agents/contributing.md:88:
… replaced with ... for compatibility across editors and tools.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on .github/labels.yml:242:
all hex color values use lowercase characters for consistency.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add .github/labels.yml as the canonical label set, synced into the
repository by .github/workflows/labels.yaml using EndBug/label-sync.
Conventions follow the Kubernetes scheme:
https://github.com/kubernetes/test-infra/blob/master/label_sync/labels.md
Six namespaced groups: kind/, priority/, triage/, lifecycle/, area/,
do-not-merge/. Cozystack-specific labels preserved (epic, community,
security/*, size:*).
Migration via aliases keeps references on existing issues and PRs:
- bug -> kind/bug
- enhancement -> kind/feature
- documentation -> kind/documentation
- question -> kind/support
- frozen -> lifecycle/frozen
- stale -> lifecycle/stale
- do-not-merge -> do-not-merge/work-in-progress
delete-other-labels is false on the initial rollout; redundant labels
("do not merge", duplicate, invalid, wontfix) stay until a follow-up
PR removes them after stabilisation.
The labels workflow has a validate job (python3 schema check) that
runs on PR. Sync runs only on push to main, weekly cron, and manual
dispatch. Schema invariants:
- description <= 100 chars (GitHub REST API limit)
- color is 6-char hex without leading #
- unique top-level names
- aliases do not collide with top-level names
PR auto-labeling (.github/workflows/pr-labeler.yaml):
- Parses PR title as Conventional Commits header (type, scope, !).
- type -> kind/* (feat -> kind/feature, fix -> kind/bug, docs ->
kind/documentation, chore/refactor -> kind/cleanup; style, perf,
test, build, ci, revert -> no kind label).
- scope -> area/* via embedded mapping; composite scopes split on
comma. Bracket-style fallback ([scope] description) maps area/*
but cannot infer kind/*.
- '[Backport release-1.x]' prefix is stripped; area/release and
backport labels are added.
- '!' after type or 'BREAKING CHANGE:' footer in body adds
kind/breaking-change.
- Unmapped scope or non-conventional title adds area/uncategorized
to flag for human review.
- Additive only — never removes existing labels.
Hardcoded label references updated:
- .github/ISSUE_TEMPLATE/bug_report.md (bug -> kind/bug)
- .github/workflows/tags.yaml (documentation -> kind/documentation)
AGENTS.md gains an Activation entry pointing agents to labels.yml
as the source of truth and to contributing.md for the title
auto-labeling table.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Empty commit to retrigger Build that lost the OCI registry tag race
on the previous attempt.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Empty commit to retrigger Build job that failed on Docker Hub anonymous
pull rate limit (429 Too Many Requests) for golang:1.26.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## Summary
- Increase kamaji controller memory limit from 500Mi to 512Mi
- Increase kamaji controller memory request from 100Mi to 256Mi
- Add startup probe with 60-second timeout (12 attempts × 5s periods)
- Increase readiness/liveness probe initialDelaySeconds from 5s/15s to
30s
## Problem
The kamaji controller was experiencing frequent CrashLoopBackOff due to
OOMKilled errors. Analysis showed:
- Container was being killed with exit code 137 (OOMKilled) after ~20-25
seconds of runtime
- Memory limit of 500Mi was insufficient for controller initialization
- Readiness probe was failing because it started too early (5s
initialDelay), before the controller finished leader election (~17s)
## Solution
**Memory increase:**
- Limit: 500Mi → 512Mi (based on production testing)
- Request: 100Mi → 256Mi (ensures adequate reservation)
**Startup probe:**
- Added to give controller up to 60 seconds to initialize without being
killed by liveness probe
- 12 attempts × 5s period = 60s maximum startup time
**Probe delays:**
- ReadinessProbe: 5s → 30s initialDelay (controller needs ~17s to
acquire leader lease)
- LivenessProbe: 15s → 30s initialDelay (aligned with readiness)
## Testing
Verified in production cluster:
- Controller runs stable with 0 restarts
- No more OOMKilled events
- Successfully creates kubeconfig secrets for tenant clusters
## Related Issues
Fixes tenant cluster components stuck in ContainerCreating due to
missing kubeconfig secrets (caused by crashing kamaji controller).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Introduced automated health checks using HTTP-based probes to monitor
service status during startup, continuous operation, and readiness to
handle traffic.
* Adjusted container memory resource allocation for enhanced stability
and performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The pool is now rendered from packages/apps/tenant/templates/cilium-lb-pool.yaml instead of packages/extra/ingress/templates/cilium-lb-pool.yaml. Cilium LB IPAM forbids overlapping CIDRs across pools regardless of serviceSelector, so both the ingress-loadBalancer path and the upcoming per-tenant Gateway in #2470 cannot each own their own pool on the same publishing.externalIPs range. The tenant chart is the natural per-tenant owner — it already creates the Namespace, the cozystack-values Secret, and the HelmReleases for both ingress and gateway.
The new pool uses a namespace-only serviceSelector (io.kubernetes.service.namespace: <ns>), which matches any LoadBalancer Service in the tenant namespace. The metadata.name changed from <trim>-ingress to <trim>-exposure to reflect that the pool is not ingress-specific.
Only the ingress-loadBalancer signal is wired in this commit (_cluster.expose-mode=loadBalancer plus .Values.ingress=true on the publishing tenant). The gateway branch is added in #2470 on top of this commit — it rebases, drops its own packages/extra/gateway/templates/cilium-lb-pool.yaml, and adds an OR branch for .Values.gateway in the tenant template.
Pool-rendering unit tests moved from packages/extra/ingress/tests/ to packages/apps/tenant/tests/. The ingress chart tests keep the Service-level asserts. packages/apps/tenant/Makefile gains a test target so hack/helm-unit-tests.sh picks up the new suite.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Only emit values key in hami.yaml when valuesOverride has content,
matching gpu-operator pattern. Add test verifying empty valuesOverride
does not produce spec.values. Fix trailing whitespace and missing
newlines in vendored chart templates.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Unconditionally emit values in hami.yaml matching the project pattern.
Remove duplicate test case and add coverage for omitted valuesOverride key.
Delete dead PSP template and RBAC rules (policy/v1beta1 removed in K8s
1.25). Override kube-scheduler image registry to registry.k8s.io to avoid
Chinese registry for international users.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Remove broken hami-dra subchart dependency from vendored chart
(Chart.yaml, Chart.lock, values.yaml) and strip DRA condition guards
from all templates since the subchart was already deleted. Override
devicePlugin updateStrategy to OnDelete to prevent destructive rolling
updates of GPU workloads. Align gpu-operator template with project
pattern (unconditional values emission). Add nodeConfiguration format
documentation and test for conflicting valuesOverride scenario.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Address review feedback from coderabbitai on
packages/system/postgres-operator/values.yaml:30.
The old comment's 5-minute ImagePullBackOff scenario conflicted with
the 180s activeDeadlineSeconds that the default maxAttempts/sleepSeconds
resolve to, so the numbers could not both be taken at face value.
Rewrite the comment to state the actual deadline math and frame the two
gates as an AND with activeDeadlineSeconds being the shorter one under
defaults, so readers understand why backoffLimit has little headroom
without an accompanying maxAttempts/sleepSeconds bump.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on
packages/system/postgres-operator/templates/webhook-ready-hook.yaml:115.
The wait container had no resource requests or limits, so schedulers
treated it as BestEffort and downstream quota enforcement had no signal.
Set small requests (10m CPU, 32Mi memory) and conservative limits
(100m CPU, 64Mi memory) matching the actual footprint of the kubectl
polling loop.
Add a matching unittest assertion so the values stay in sync if anyone
touches the template.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on
packages/system/postgres-operator/templates/webhook-ready-hook.yaml:83.
The Job targets the cnpg-webhook-service/services/proxy subresource, which
is namespaced and lives in the release namespace. A namespaced Role and
RoleBinding grant the exact permission needed without creating global
RBAC for a namespaced probe, which is the principle of least privilege.
Also update the kind assertions in tests/webhook-ready-hook_test.yaml so
the unittest suite tracks the new Role/RoleBinding objects.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Bumps the vendored `robotlb` chart to the latest upstream build.
Chart version remains `0.1.3`; the bundled `appVersion` moves from
`0.0.5` to `0.0.6`.
The new `robotlb` release adds RBAC permissions for
`discovery.k8s.io/endpointslices` (`get`, `list`, `watch`), which are
required to manage services backed by `EndpointSlice` — notably
KubeVirt-exposed workloads that do not publish classic `Endpoints`.
Notes:
- Upstream also replaced `replicas: {{ .Values.replicas }}` with a
hardcoded `replicas: 1` in `templates/deployment.yaml`. The
effective replica count is unchanged (we already set `1`), but the
value is no longer overridable via chart values. A minor cosmetic
reformat was applied to `templates/role.yaml`.
Closes#2256
### Release note
```release-note
chore(hetzner-robotlb): update robotlb to 0.0.6 — adds RBAC for EndpointSlices so services backed by EndpointSlice (e.g. KubeVirt) are supported.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Extended service account permissions to access Kubernetes endpoint
slices from the discovery API.
* **Bug Fixes**
* Deployment replica configuration now fixed to single instance.
* **Style**
* Improved YAML formatting in role template declarations.
* **Chores**
* Updated application version metadata to 0.0.6.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Refreshes the vendored Cilium chart in `packages/system/cilium` from
v1.19.1 to v1.19.3 via `make update`. Chart templates, values, CRDs and
the Cilium image reference are regenerated from upstream.
### Motivation
- **v1.19.2** ships a critical fix for cert-manager HTTP-01 Gateway API
challenges on hostnames that have both HTTP and HTTPS listeners
([cilium#44492](https://github.com/cilium/cilium/pull/44492), backport
[#44517](https://github.com/cilium/cilium/pull/44517)). Without this
fix, cert-manager cannot issue certificates via Gateway API when a
redirect HTTP listener and a TLS HTTPS listener share a hostname.
- **v1.19.3** is the latest stable patch release in the v1.19.x line (15
Apr 2026).
- This bump is a prerequisite for upcoming Gateway API work tracked
separately.
### Upstream changes pulled in
- Cilium Envoy bootstrap config, operator clusterrole, config template,
`values.schema.json` and the cilium-agent DaemonSet refreshed from
upstream.
- New `templates/ztunnel/` directory (DaemonSet, Secret, ServiceAccount)
added by upstream — not enabled by default in Cozystack values.
### Release note
```release-note
chore(cilium): bump to v1.19.3 (cert-manager HTTP-01 fix via cilium#44492)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added ztunnel encryption support with configurable deployment settings
* Added ConfigDriftDetection for monitoring ConfigMap changes
* Added endpoint policy update timeout configuration
* Added load balancer service topology support
* Extended Envoy circuit breaker configuration with connection and
request limits
* **Updates**
* Upgraded Cilium to v1.19.3
* Updated container images (Envoy, certgen, Hubble relay, clustermesh)
* Enhanced Cilium operator RBAC capabilities for managing ServiceImport
finalizers
* **Removals**
* Removed BIG TCP tunnel configuration option
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cnpg admission webhook's controller pod passes readinessProbe as soon
as the local HTTPS server binds to :9443. The HelmRelease marks itself
Ready right after that via helm install --wait. But the Service
cnpg-webhook-service needs its EndpointSlice populated and the data plane
(kube-proxy / cilium) programmed before kube-apiserver can reach the
webhook through the Service ClusterIP. That gap is short but not zero,
and any HelmRelease that depends on postgres-operator (cozy-keycloak,
tenant Postgres apps) can fire its own install inside the window and hit
Internal error occurred: failed calling webhook "mcluster.cnpg.io":
failed to call webhook: Post "https://cnpg-webhook-service.cozy-postgres-operator.svc:443/...":
dial tcp <svc-ip>:443: connect: connection refused
which fails the install of the downstream release. Seen on cozystack/cozystack#2470
E2E run 24862782568.
Add a post-install,post-upgrade Helm hook Job that blocks the release
from reporting Ready until the webhook answers /readyz through the
apiserver service proxy. Apiserver proxy routes the call over the same
Service IP → EndpointSlice → pod path the admission webhook uses, so
once it responds, the webhook admission path is also working.
RBAC is minimal: a dedicated ServiceAccount with a ClusterRole that only
grants get on services/proxy scoped to https:cnpg-webhook-service:webhook-server.
The Job times out after 120s with 60 attempts at 2s intervals — longer
than any data-plane programming delay seen on E2E, but bounded.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on
packages/extra/ingress/templates/cilium-lb-pool.yaml:19: if the operator
passes an externalIP already in CIDR form (192.0.2.10/32 or 2001:db8::1/128),
the template appended a second /32 or /128 suffix producing an invalid
CiliumLoadBalancerIPPool block. Guard the suffix append on the absence of
"/" in the input.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add PackageSource for HAMi with dependency on gpu-operator.
Include HAMi in the iaas bundle as an optional package, allowing
host-level deployment alongside the existing kubernetes addon path.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The hami-dra subchart renders resources even when dra.enabled=false
because Helm dependency conditions don't work for vendored subcharts.
The subchart also contains duplicate YAML label keys
(app.kubernetes.io/component, app.kubernetes.io/name) and references
unpublished container images (v0.0.1-dev). DRA support can be
re-added when the upstream project stabilizes it.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- Add generated-output dirs (pkg/generated, internal/crdinstall/manifests,
packages/system/*/definitions) and Makefile to the workflow paths: filter
so PRs that modify only generated artifacts still trigger the drift check.
- Mirror the same paths in the root pre-commit hook's files: regex so
manual edits to generated files or changes to the root generate target
re-run make generate through pre-commit.
- Switch drift detection in the workflow from `git diff --exit-code` to
`git status --porcelain` so new untracked files produced by make generate
(e.g. generated YAML/Go for a new API type) also fail the job; dump
`git diff --color=always` on failure for easier debugging.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Added comprehensive unit tests for authorization logic:
- hasAccessToNamespace() tests:
- User subject access (positive and negative cases)
- Group subject access
- ServiceAccount subject access
- ServiceAccount with empty namespace (defaults to RoleBinding ns)
- Privileged groups (system:masters, cozystack-cluster-admin)
- Get() handler tests:
- Returns namespace when user has access
- Returns NotFound when user lacks access (not Forbidden)
- Returns NotFound for non-tenant namespaces
All tests verify that authorization correctly enforces RoleBinding-based
access control and prevents IDOR vulnerabilities.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Adds explicit guidance to `docs/agents/contributing.md` about running
`make generate` in any touched package before committing. Pre-commit CI
runs `make generate` in every package and fails with exit code 123 on
any uncommitted generator output (regenerated `README.md`, reordered
`values.schema.json`, refreshed
`packages/system/<name>-rd/cozyrds/<name>.yaml`).
Recent PRs have tripped on this during review cycles. Documenting it in
the contributing checklist saves a round-trip.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Require regenerating and committing generated artifacts when
designated source files are edited.
* Added a section listing generated artifacts per package and concrete
regeneration/staging steps.
* Documented CI enforcement that detects unstaged generator output and
blocks PRs.
* Added guidance for rerunning generation after amended commits, plus
updated commit-scope guidance (now “not exhaustive”) and included
“agents” in allowed scopes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Address review feedback from gemini-code-assist on docs/agents/contributing.md:11:
Scope linters kept flagging valid scopes like 'agents' as unknown because
the list read as exhaustive. Annotate it as examples (not exhaustive) and
add 'agents' to the Other group so both humans and review bots stop
tripping on scopes that are already in regular use across the repo
history.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on docs/agents/contributing.md:30:
Reword "likely to need regenerated" (regional construction) to
"likely needs to be regenerated" for standard technical prose.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from coderabbitai on docs/agents/contributing.md:26:
Replace the hard-coded packages/extra/<name> path in the example with
packages/<apps-or-extra>/<name> so the example matches the preceding
text that describes both apps and extra packages.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Service.spec.externalIPs is deprecated upstream in Kubernetes v1.36
(KEP-5707, kubernetes#137293). The AllowServiceExternalIPs feature gate
is expected to default to off around v1.40 and the implementation to
be removed around v1.43. For bare-metal installs that rely on
externalIPs today, cozystack needs a migration path.
This change adds an opt-in 'loadBalancer' exposure mode for the
ingress-nginx Service:
- New platform value 'publishing.exposure' (enum: externalIPs |
loadBalancer, default externalIPs). Plumbed through cozystack-values
into each tenant's ingress HelmRelease via the new 'expose-mode' key.
- Unknown values and loadBalancer with an empty externalIPs list fail
the chart render with explicit error messages, rather than silently
producing a broken Service.
- When exposure=loadBalancer and the current namespace matches
publishing.ingressName, the Service becomes type: LoadBalancer with
externalTrafficPolicy: Local.
- A new template renders a CiliumLoadBalancerIPPool whose blocks come
from publishing.externalIPs (IPv4 addresses get /32, IPv6 addresses
get /128) and whose serviceSelector uses Cilium's synthetic
io.kubernetes.service.namespace key combined with the standard
app.kubernetes.io/name: ingress-nginx label. No custom label is
written to the Service itself, avoiding cross-tenant collisions from
user-defined labels.
Default behaviour is unchanged: without opting in, the Service is
still ClusterIP + spec.externalIPs as today.
Scope: only ingress-nginx is migrated by this setting. Other cozystack
components that still write Service.spec.externalIPs directly (notably
the vpn app) must be migrated separately before the v1.40 feature gate
flip.
Tests: packages/extra/ingress/tests/exposure_test.yaml adds 13
helm-unittest cases covering both modes, IPv4/IPv6, empty-token
filtering, unknown-mode rejection, and the non-matching-namespace
fallback.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The previous image digest in values.yaml pointed at a single-arch
linux/arm64 manifest because 'make image' was run from an arm64 host
with the default buildx platform. Cozystack targets amd64 (Talos build
output, E2E runners, most real-world clusters) and also arm64 for
hybrid fleets, so Helm install would fail on amd64 nodes with 'no
matching manifest for linux/amd64 in the manifest list entries'
whenever somebody installed directly from this commit between merge
and the next release-tag CI rebuild.
Fix: rebuilt the image locally with
PLATFORM='linux/amd64,linux/arm64' make image from a buildx
docker-container driver, pushed the multi-arch manifest, and
refreshed values.yaml with:
- digest of the new multi-arch manifest list (verified via
'docker manifest inspect': amd64 sha256:e1977323..., arm64
sha256:8f5ab529...).
- tag bumped from 'latest' (emitted by the common-envs.mk settag
macro on a non-tagged checkout) to '1.19.3', matching the
established convention in every other packages/system/*/values.yaml
so reviewers and incident response have a human-readable version
anchor independent of digest chasing.
The Makefile is left untouched so the CI builder (which only uses the
default docker driver) keeps building single-arch for whatever
architecture it runs on; multi-arch is a responsibility of the
release-tag pipeline or an explicit local rebuild.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Built ghcr.io/cozystack/cozystack/cilium from the refreshed upstream
v1.19.3 base image and updated values.yaml with the new digest.
Previously values.yaml still pointed at the v1.19.1 cozystack rebuild
by digest while Chart.yaml and the Dockerfile were on v1.19.3 — with
chart default useDigest=true that would have silently pulled v1.19.1
until the next release-tag rebuild.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Vendored chart refreshed via make update in packages/system/cilium.
Motivation: v1.19.2 fixes a cert-manager HTTP-01 bug on hostnames with
both HTTP and HTTPS listeners (cilium#44492, backport PR #44517). This
is a prerequisite for upcoming Gateway API work.
v1.19.3 is the latest stable release in the v1.19.x line (15 Apr 2026).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Pre-commit CI runs make generate in every package and fails with exit
123 on any uncommitted generator output. Add explicit guidance so
agents stage regenerated README.md, values.schema.json and
packages/system/<name>-rd artifacts alongside the hand edits.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Pulls the latest robotlb chart (0.1.3) which ships robotlb 0.0.6.
The new appVersion adds RBAC permissions for discovery.k8s.io/endpointslices
needed to support EndpointSlice-based services such as KubeVirt.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Fixes the failures that blocked the v1.3.0 release pipeline (workflow
run [24765377017]) and closes the gap in `docs/agents/changelog.md` that
made them possible.
### Bug 1 — pathspec error in `Create changelog branch and commit`
After the Copilot step, the workflow runs `git checkout -b
"$CHANGELOG_BRANCH" origin/main`. Copilot CLI was invoked with
`--allow-all-tools` and had committed the generated changelog onto HEAD
of `main`, so the reset to `origin/main` deleted the tracked file, and
`git add "$CHANGELOG_FILE"` then failed with `fatal: pathspec ... did
not match any files`.
Fix (commit 2):
- Snapshot the file across the branch switch so the checkout cannot drop
it.
- `set -euo pipefail` so any failure surfaces loudly.
- Drop the dead "no changes to commit" soft-branch — the earlier
`check_changelog` step gates this step on the file being absent from
`origin/main`, so `git commit` must produce a diff; if it doesn't (e.g.
empty file), fail loud instead of pushing an empty branch.
- Fail-early file-existence check with `::error::` annotation, and
`VERSION` moved to step `env:`.
### Bug 2 — no timeout on `Generate changelog using AI`
The re-run hung in Copilot for 10+ minutes with zero log output. With no
`timeout-minutes`, a hung Copilot would hold a self-hosted runner for 6
hours (job default).
Fix (commit 2): `timeout-minutes: 30` (the prior successful run took ~26
minutes).
### Root-cause fix — scope the agent prompt
`docs/agents/changelog.md` is the actual prompt driving Copilot. It
ended with "Save the changelog" and gave no boundary, so an agent with
`--allow-all-tools` could reasonably interpret "done" as "commit, push,
open a PR".
Fix (commit 1):
- Add a "Scope and boundaries" section at the top stating the single
deliverable is `docs/changelogs/v<version>.md` and enumerating forbidden
operations (git commit / push / branch / tag / reset / merge / rebase,
PR creation, GitHub API writes, modifying any file other than the
changelog).
- Add an explicit "then exit" at the end of Step 9 with a back-reference
to the scope section.
- Scoped "unless the caller explicitly instructs otherwise" so
interactive IDE use stays flexible.
With the rules in the doc, CI and interactive callers share the same
boundary, and the workflow prompt becomes a one-line invocation
(`--prompt "Generate the release changelog for tag v${VERSION}. Follow
the instructions in @docs/agents/changelog.md exactly, including the
'Scope and boundaries' section at the top. ..."`).
### Files touched
- `docs/agents/changelog.md` — +16 / -0 (scope section + exit rule)
- `.github/workflows/tags.yaml` — +36 / -31 (timeout + rewritten commit
step + simplified prompt)
Other jobs in the workflow (`prepare-release`, `update-website-docs`)
are untouched.
### Release note
```release-note
NONE
```
[24765377017]:
https://github.com/cozystack/cozystack/actions/runs/24765377017
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved release automation reliability with stricter verification,
enforced error handling, and a timeout for changelog generation
* Ensured automated commits/pushes occur only after successful output
validation to prevent accidental repository mutations
* **Documentation**
* Clarified agent instructions to produce a single changelog file and
terminate, and to restrict the agent to read-only inspection during
generation
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
hack/update-codegen.sh sources kube_codegen.sh from the Go module
cache at ~/go/pkg/mod/k8s.io/code-generator@vX.Y.Z/, but the module
is not declared in go.mod so a fresh runner has nothing to source
from. Add a workflow step that parses the pinned version out of the
script and pulls the module into the cache before running
make generate.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Post-release cleanup of `docs/changelogs/v1.3.0.md` so the notes match
what users actually experience in v1.3.0. No code changes.
- **Rewrite the postgres major-features entry** so author
(`@myasnikovdaniil`), PR (`#2369`), and description all line up with the
`17.7-standard-trixie` pin + migration-37 `imageName` rewrite that
actually shipped. The previous entry credited `#2304` with a description
matching a superseded `spec.version=v17` backfill approach.
- **Remove the duplicate `#2364` postgres bug-fix entry** — the same
work is now folded into the single major-features entry above, with
backport references to `#2309` (v1.2.1) and `#2364` (v1.2.2).
- **Remove the `[linstor-gui] Restrict to cozystack-cluster-admin group`
security entry.** The vulnerable state never shipped in a tagged
release, so there is nothing user-facing to announce. The
`cozystack-cluster-admin`-group restriction is already described in the
linstor-gui Feature Highlights section as part of the feature's day-one
shipping behavior.
### Release note
```release-note
[]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated v1.3.0 changelog with clarified PostgreSQL system version
pinning details and removed redundant entries for improved documentation
clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The etcd chart shipped a `post-upgrade` Helm hook that `kubectl delete`d the
etcd TLS chain (`etcd-{ca,peer-ca,client,peer,server}-tls`) and then deleted
etcd pods on every chart upgrade, gated by a semver compare of an
`etcd-deployed-version` ConfigMap against `2.6.1`.
The hook was added as a one-shot migration for the chart `2.6.0 -> 2.6.1`
transition. Since commit f871fbdb ("Remove versions_map logic") all chart
versions are stamped as `0.0.0+<git-hash>`, which per semver is always
`< 2.6.1`. The gate therefore always resolves to "update certs", firing the
destructive hook on every etcd upgrade. On clusters running Kamaji-managed
tenant control planes this wipes the etcd CA, cert-manager re-issues it, and
tenant kube-apiservers hit `x509: certificate signed by unknown authority`
against `etcd.<ns>.svc:2379` until each tenant DataStore is manually
re-reconciled.
Commit 47d81f70 ("Disabled private key rotation in CA certs") already fixed
the underlying `rotationPolicy: Always` issue the migration was papering
over, so the hook has no remaining use. Remove the hook Job, its RBAC, the
version ConfigMap it read, and add a helm-unittest suite under
`packages/extra/etcd/tests/` that guards against re-introducing the hook
or the version ConfigMap.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Review feedback on PR #2460: the Generate changelog using AI step ran
Copilot with --allow-all-tools and GH_TOKEN set to the write-capable
installation token issued to the job (contents: write,
pull-requests: write on all cozystack/* repos). The scope rules in
docs/agents/changelog.md and the step prompt tell the agent not to
use those permissions, but nothing at the token layer prevented it.
Mint a second, read-only installation token from the same app
(same COZYSTACK_CI_APP_ID / COZYSTACK_CI_PRIVATE_KEY, scoped to
contents/pull-requests/metadata read) and pass that one to the AI
step instead. The write-capable token is still used by the checkout,
commit/push, and PR-creation steps that actually need it.
This is defense in depth: even if a future prompt change or agent
misbehavior ignored the scope rules, the token itself has no write
capability on any repository in the cozystack org. No new secret,
no new GitHub App install, no admin-side change — the RO token is
minted in the same workflow from the same app credentials.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Review feedback on PR #2460: the existing `[ -f ]` check catches
missing files but a zero-byte `docs/changelogs/v${VERSION}.md` would
still be staged and committed — `git add` + `git commit -s` on a new
empty file succeeds and produces a real commit, leaving the
downstream PR with no actual changelog content.
Add a `[ -s ]` guard after the existence check: if the Generate
changelog using AI step produces an empty file, emit a matching
`::error::` annotation and exit 1 before snapshotting.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Review feedback on PR #2460: the previous "do not write to refs, HEAD,
or remotes" wording contradicted the explicit allowance of `git fetch`
(which updates remote-tracking refs) and the mandatory cross-repo
checks in Step 6, which `cd` into `_repos/<repo>` and run
`git checkout`, `git pull`, etc.
Sharpen the scope paragraph:
- The git-write ban is now explicitly scoped to the cozystack working
tree — it bans writing to local branches, tags, or HEAD in that
repo, not "refs/HEAD/remotes" globally.
- `git fetch` is called out as expected.
- Local git operations inside disposable `_repos/` clones
(`git checkout`, `git pull`, etc.) are explicitly allowed, with
the remaining rules (no push, no PR creation, no API writes)
applying to any repository.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
The `backupstrategy-controller` chart declared a `Role` and
`RoleBinding` scoped to the `cozy-velero` namespace (for managing
`ResourceModifier` ConfigMaps consumed by Velero Restore). Because
`cozystack.velero` is an optional package, that namespace does not exist
in bundles that do not enable velero — and `backupstrategy-controller`
is a **default** package. Helm install aborted with:
```
namespaces "cozy-velero" not found
```
which blocked the entire
`cozy-backup-controller/backupstrategy-controller` HelmRelease on any
cluster where velero was not explicitly enabled (including the E2E
environment).
This PR moves that Role/RoleBinding into the velero chart
(`packages/system/velero/templates/backupstrategy-controller-rbac.yaml`),
so the permission grant only exists when velero is actually installed —
where it is useful. The RoleBinding subject points to the stable
`backupstrategy-controller` ServiceAccount in `cozy-backup-controller`.
### Release note
```release-note
fix(backups): moved the velero-namespaced ResourceModifier ConfigMap Role and RoleBinding from the backupstrategy-controller chart into the velero chart. This unblocks installs of backupstrategy-controller on bundles that do not enable velero (previously the HelmRelease failed with `namespaces "cozy-velero" not found`).
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Reorganized RBAC configuration for the backup strategy controller by
consolidating namespace-scoped role definitions in the Velero chart
template
* Updated role bindings and permissions structure across system packages
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The backupstrategy-controller chart declared a Role/RoleBinding in the
cozy-velero namespace for ResourceModifier ConfigMap management. Because
velero is an optional package, that namespace does not exist in bundles
without velero, so Helm install aborted with "namespaces \"cozy-velero\"
not found" and blocked the default install of backupstrategy-controller.
Move the Role and RoleBinding into the velero chart so they are created
only when velero is actually installed. The RoleBinding subject points
to the backupstrategy-controller ServiceAccount in its fixed namespace
(cozy-backup-controller).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Three changes to the generate-changelog job to fix the v1.3.0
release pipeline failure (run 24765377017) and make the job robust
to whatever state the Copilot step leaves behind.
1. Add `timeout-minutes: 30` to the Generate changelog using AI
step. On the v1.3.0 re-run the step hung silently for 10+
minutes; with no timeout a hung Copilot would hold a self-hosted
runner for up to 6 hours (job default). The previous successful
run took ~26 minutes, so 30 is a reasonable ceiling.
2. Replace the terse, ambiguous Copilot prompt with a one-liner
that invokes docs/agents/changelog.md directly. The "Scope and
boundaries" section added to that doc in the previous commit is
now the single source of truth for what the agent may and may
not do, so the workflow only needs to pass the version and
point at the relevant doc. VERSION is moved to step env: to
match GitHub's workflow-injection hardening guidance.
3. Rewrite the Create changelog branch and commit step:
- add `set -euo pipefail` so any failure is visible
- validate the file exists up front and fail loud with
`::error::` if not
- copy the file to a tempfile BEFORE `git checkout -b`, so the
checkout to `origin/main` cannot remove it (this is the fix
for the original pathspec error the v1.3.0 run hit)
- use `trap` to clean up the tempfile on any exit path
- move VERSION to env
- drop the dead "no changes to commit" branch: the check_changelog
step earlier in the job gates this step on the file being
absent from origin/main, so `git add` + `git commit` must
produce a diff. If they don't (e.g. Copilot emitted an empty
file), fail loud instead of pushing an empty branch.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The v1.3.0 release pipeline broke because Copilot, invoked by
.github/workflows/tags.yaml with --allow-all-tools, committed the
generated changelog onto HEAD of main on its own. The workflow's
next step — `git checkout -b ... origin/main` — then wiped the file,
and `git add` failed with a pathspec error.
The root cause is in this document. The checklist ends with "Save
the changelog", which an agent with broad tool access can reasonably
interpret as "also commit it, push it, and open a PR". There was no
explicit boundary.
Add a "Scope and boundaries" section at the top and an explicit
"then exit" at the end of Step 9:
- The single deliverable is docs/changelogs/v<version>.md.
- Forbidden by default: git commit / push / checkout (to switch
branches) / branch / tag / reset / merge / rebase; PR creation;
GitHub API writes (POST/PATCH/DELETE); modifying any file other
than the changelog.
- Read-only analysis (git log/show/fetch/diff, gh pr view, gh api
GET) remains expected.
- Auxiliary repo clones under _repos/ remain allowed for cross-repo
analysis per Step 6.
- Scoped "unless the caller explicitly instructs otherwise" so
interactive use with an IDE remains flexible.
With the rules in the doc, CI and interactive callers share the
same boundary; the workflow can invoke the doc with a one-line
prompt instead of re-stating the constraints every time.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Post-release cleanup of docs/changelogs/v1.3.0.md so the notes match
what users actually experience in the released v1.3.0:
- Rewrite the postgres major-features entry so author (myasnikovdaniil),
PR (#2369), and description all match the 17.7-standard-trixie pin +
migration-37 imageName rewrite that actually shipped. The previous
entry credited #2304 (superseded spec.version=v17 backfill approach).
- Remove the duplicate #2364 postgres bug-fix entry; the same work is
now folded into the single major-features entry above, with backport
references to #2309 (v1.2.1) and #2364 (v1.2.2).
- Remove the [linstor-gui] Restrict to cozystack-cluster-admin group
security entry. The vulnerable state never shipped in a tagged
release, so there is nothing user-facing to announce; the restriction
is already described in the linstor-gui Feature Highlights section
as part of the feature's day-one behavior.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
This PR adds the changelog for release `v1.3.0`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.3.0.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Published Cozystack v1.3.0 release notes featuring LINSTOR scheduler
extender for storage-aware pod placement, managed LINSTOR GUI web
console, VM Default Images catalog, expanded observability with Events
dashboard and S3 metering, cross-namespace VMInstance backup/restore,
and various platform enhancements and bug fixes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
HAMi-core relies on _dl_sym (private glibc symbol removed in 2.34)
for CUDA interception. This breaks compute isolation on modern
container images using Ubuntu 22.04+ and makes Alpine/musl completely
incompatible.
Include upstream issue references and a compatibility matrix so users
can make informed decisions about base image selection.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add HAMi HelmRelease to the kubernetes app as a toggleable addon.
When enabled, GPU Operator's built-in device plugin is automatically
disabled via a values merge to avoid conflicts with HAMi's own
device plugin.
The HelmRelease fails fast if GPU Operator is not enabled, since HAMi
depends on it for driver management and container toolkit.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add HAMi v2.8.1 (CNCF Sandbox) as a new system package for fractional
GPU sharing in tenant Kubernetes clusters. The chart enables workloads
to request specific amounts of GPU memory and compute cores instead of
claiming entire GPUs.
Vendored upstream chart includes device plugin, scheduler extender,
mutating webhook, and DRA subchart. Wrapper values configure nvidia
RuntimeClass and clear hardcoded node config from upstream defaults.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Harbor is a PaaS service (`category: PaaS`), not a tenant module. It is
not deployed automatically into tenant namespaces — there is no
corresponding manifest in `packages/apps/tenant/templates/`, unlike
actual tenant modules (monitoring, ingress, etcd, info, seaweedfs).
Two flags were incorrectly set on its `ApplicationDefinition`:
- `spec.dashboard.module: true` — caused Harbor to appear in the sidebar
"Modules" section and be excluded from its proper PaaS category.
- `spec.release.labels."internal.cozystack.io/tenantmodule": "true"` —
caused controllers (`pkg/registry/core/tenantmodule`, dashboard) to
treat the Harbor HelmRelease as a tenant module.
Both flags are removed so Harbor is listed correctly under PaaS and
handled as an ordinary managed application.
### Release note
```release-note
fix(harbor): remove incorrect tenant module flags from Harbor ApplicationDefinition so it appears under the PaaS category in the dashboard and is no longer treated as a tenant module
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Simplified internal application configuration settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Bumps `packages/system/kubeovn` to `cozystack/kubeovn-chart` tag
`v1.15.10-cozy.1`, which:
1. Updates upstream kube-ovn from v1.15.3 to v1.15.10 (latest patch in
the v1.15 series).
2. Carries a patch over `pkg/controller/pod.go` that preserves a VM
LSP's port-group memberships when kubernetes GCs a completed
virt-launcher pod while another virt-launcher pod of the same VM is
still running.
Without the patch, the destination pod of a successful live migration
loses its security groups, network policies and node-scoped routing
after kubernetes cleans up the source pod, and only recovers after a
`kube-ovn-controller` restart. The buggy code path is identical between
v1.15.3, v1.15.10, release-1.15 HEAD and master HEAD — confirmed by diff
and by reproduction on a cozystack cluster.
- Upstream issue: kubeovn/kube-ovn#6665
- Upstream fix PR: kubeovn/kube-ovn#6666
- Chart carry: cozystack/kubeovn-chart#4
### Release note
```release-note
fix(kube-ovn): bump kube-ovn to v1.15.10 and carry upstream fix that keeps VM LSP port-group memberships when kubernetes GCs a completed virt-launcher pod (post-migration regression)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* VpcEgressGateway CRD extended with optional resources
(claims/limits/requests) and bandwidth (ingress/egress) fields.
* Added a pre-upgrade hook to run compatibility steps before upgrades.
* CNI DaemonSet now optionally respects MTU when configured.
* **Bug Fixes**
* Updated Kube-OVN and NAT gateway container images to the latest
maintenance release.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes an RBAC failure in `kube-ovn-plunger` that prevented it from
reconciling the `ovn-central` Deployment:
```
deployments.apps is forbidden: User "system:serviceaccount:cozy-kubeovn:kube-ovn-plunger"
cannot list resource "deployments" in API group "apps" at the cluster scope
```
Two causes addressed:
- The controller-runtime cache issued cluster-wide list/watch for
Deployments and Pods by default. The manager cache is now scoped to the
kube-ovn namespace, so requests go to the namespaced endpoints covered
by the Role.
- The Role's deployments rule used `resourceNames`, which does not apply
to list/watch in Kubernetes RBAC. The restriction is removed; the Role
is still namespace-scoped via RoleBinding to the `cozy-kubeovn`
namespace.
`--kube-ovn-namespace` and `--ovn-central-name` are now passed to the
binary from chart values so the previously unused value is wired up.
### Release note
```release-note
fix(kube-ovn): fix kubeovn-plunger RBAC forbidden error on ovn-central Deployment reconcile
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable parameters for KubeOVN namespace and OVN central
name in deployment configuration.
* **Improvements**
* Enhanced cache configuration with namespace-scoped resource management
capabilities.
* Updated RBAC permissions to remove resource-specific constraints,
enabling broader deployment access.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Options field was added to RestoreJobSpec without re-running
'make generate', leaving zz_generated.deepcopy.go out of sync.
Regenerated to include the missing DeepCopyInto handling for the
runtime.RawExtension pointer.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Run root 'make generate' as a pre-commit hook and as a dedicated
CI workflow so missed codegen updates (CRDs, deepcopy, clients, RBAC)
are caught instead of merging stale generated files.
Pre-commit hook is scoped to files that actually affect codegen
(api/, pkg/apis/, hack/update-codegen.sh, hack/boilerplate.go.txt)
so unrelated commits are not slowed down. CI job sets up Go from
go.mod, runs make generate, and fails on drift with a pointer to
the local fix.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Harbor is a PaaS service, not a tenant module. It is not deployed
automatically into tenant namespaces (no manifest in
packages/apps/tenant/templates/). Remove the misplaced
`dashboard.module: true` flag and
`internal.cozystack.io/tenantmodule: "true"` release label so Harbor
appears under the PaaS category in the dashboard and is not treated as
a tenant module by the controllers.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Pulls cozystack/kubeovn-chart v1.15.10-cozy.1, which bumps upstream
kube-ovn from v1.15.3 to v1.15.10 and carries a patch over
pkg/controller/pod.go that preserves a VM LSP's port-group memberships
when kubernetes GCs a completed virt-launcher pod while another
virt-launcher pod of the same VM is still running.
Without the patch, the destination pod of a successful live migration
loses its security groups, network policies and node-scoped routing
after kubernetes cleans up the migration source pod, and only recovers
after a kube-ovn-controller restart.
Upstream issue: kubeovn/kube-ovn#6665
Upstream fix PR: kubeovn/kube-ovn#6666
Chart carry: cozystack/kubeovn-chart#4
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
- Introduce list, details, create, and sidebar views for `RestoreJob` in
the dashboard controller.
- Auto-generate stable component IDs for sidebar entries and detail
sub-blocks so upserts remain consistent across reconciles.
- Render a single "Same as backup" fallback for an omitted
`spec.targetApplicationRef` on the details view instead of concatenating
three missing-path fallbacks.
- Mark non-CRD-backed `stock-project-factory-*-details` sidebars
(`kube-*`, `plan`, `backupjob`, `backup`, `restorejob`) as static so
they pick up consistent managed-by labels.
### Release note
```release-note
[dashboard] Add RestoreJob views (list, details, create, sidebar link) to the dashboard controller.
```
## Test plan
- [x] Deploy controller to dev stand and open the restore pages in the
dashboard
- [x] Create a RestoreJob through the UI against an existing Backup;
reconcile succeeds and the details view renders
- [ ] CI green
## Previous PR
https://github.com/cozystack/cozystack/pull/2387
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced comprehensive RestoreJobs dashboard interface enabling
users to view and manage backup restoration operations.
* New RestoreJobs section added to the Backups menu with dedicated
details page and navigation.
* Dashboard displays restore job status, phase, backup references,
target applications, and operation timestamps for easy monitoring.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The kubeovn-plunger controller-runtime cache attempted cluster-wide
list/watch on Deployments and Pods, which the namespace-scoped Role
cannot satisfy. Additionally, the deployments rule relied on
resourceNames, which does not restrict list/watch verbs and left the
permission effectively unusable.
Scope the manager cache to the kube-ovn namespace so list/watch hit
the namespaced API, and drop resourceNames from the deployments rule.
Wire --kube-ovn-namespace and --ovn-central-name through the
Deployment so values are actually consumed.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
HTTP-01 ACME issuance in Cozystack has been relying on deprecated
cert-manager fields that happen to still work, but point at a
non-existent ingress class on default installs.
**The bug:**
- `ClusterIssuer`s (`letsencrypt-prod`, `letsencrypt-stage`) hardcode
`http01.ingress.class: nginx`, but Cozystack's nginx ingress classes are
named after tenants (`tenant-root`, `tenant-*`) — there is no class
called `nginx`.
- Every ingress that needs a cert overrides this with the deprecated
annotation `acme.cert-manager.io/http01-ingress-class: <class>`.
cert-manager still honors it, which is why issuance works at all.
- Mixing the modern `ingressClassName` field with the legacy
`http01-ingress-class` annotation produces `fields ingressClassName and
class cannot be set at the same time` (cert-manager#6651), so the
migration has to be atomic.
**The fix:**
- Parameterize the `ClusterIssuer` HTTP-01 solver from
`_cluster.expose-ingress` (default `tenant-root`) using the modern
`ingressClassName` field.
- Rename the override annotation on all ingresses to the modern
`acme.cert-manager.io/http01-ingress-ingressclassname`.
- No new features, no behavior change for operators who already set
`_cluster.expose-ingress` correctly — just removes the dependency on the
deprecated API and stops pointing at a phantom class.
Touched ingresses (9 templates): keycloak, dashboard, linstor-gui,
bucket, grafana, alerta, bootbox/matchbox, seaweedfs, harbor. Plus
`seaweedfs` values.yaml default annotation and the
`cert-manager-issuers` ClusterIssuer template.
Verified renders for `http01` + `tenant-root`, `http01` + custom class,
and `dns01` (correctly omits the http01 branch).
### Release note
```release-note
fix(platform): ACME HTTP-01 challenges now use the modern `ingressClassName` API and target the tenant ingress class from `_cluster.expose-ingress` (default `tenant-root`) instead of a hardcoded `nginx` class that did not exist on most installs. Ingress templates switched from the deprecated `acme.cert-manager.io/http01-ingress-class` annotation to `acme.cert-manager.io/http01-ingress-ingressclassname`.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated cert-manager ACME HTTP-01 solver annotations across ingress
configurations to use the newer `ingressclassname` format for improved
compatibility with current cert-manager versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Updates the `update-website-docs` job in `.github/workflows/tags.yaml`
to match the new docs-versioning contract introduced in
cozystack/website#495.
The website repo replaces the old "pre-create `vX.Y/` draft directory"
scheme with a permanent `content/en/docs/next/` trunk. Released version
directories are no longer implicitly pre-created — the upstream release
workflow owns the explicit decision of when to promote `next/` →
`vX.Y/`.
### Changes
- **New step `Determine if this release promotes next/`** — parses the
tag and only sets `promote=true` for final `vX.Y.Z` tags where
`content/en/docs/vX.Y/` does not already exist. Prereleases
(`vX.Y.Z-rc.N`, etc.) and patch releases skip promotion.
- **New step `Promote next/ to released version`** — gated on
`promote=true`; runs `make release-next RELEASE_TAG=<tag>`.
- **Step order**: `release-next` runs **before** `update-all`. This way
`update-all` routes into the freshly-created `vX.Y/` directory (per the
website Makefile's routing logic), and `next/` stays untouched as the
trunk for the following release.
- **`git add content hugo.yaml`** — `release-next` registers the new
version in `hugo.yaml` via `register_version.sh`, so the commit step
must stage it.
- Converted the existing `update-all` step to use `env:` vars for
consistency with the new step and workflow-injection hardening.
### Behaviour by tag
| Tag | `release-next`? | `update-all` targets |
|-----|-----------------|----------------------|
| `v1.3.0` (v1.3 does not exist) | yes — promotes next/ → v1.3/ | v1.3/
|
| `v1.3.1` (v1.3 exists) | no — skipped | v1.3/ |
| `v1.3.0-rc.1` | no — skipped (prerelease) | next/ |
| `v2.0.0` (v2.0 does not exist) | yes — promotes next/ → v2.0/ | v2.0/
|
### Release note
```release-note
Release tagging now promotes the website's `next/` docs trunk to `vX.Y/` when cutting a new minor or major release (requires cozystack/website#495 to be merged first).
```
### Dependency
Must not merge until cozystack/website#495 is merged on `main` of the
website repo, otherwise `make release-next` won't exist there and
new-minor release tags will fail.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated release automation to apply conditional logic for version
promotion based on tag patterns and documentation availability. Modified
documentation staging behavior in release workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Replace rename-only wording on spec.targetApplicationRef form labels so
users discover that the field also supports restoring into a different
application, not just renaming.
- Drop VMInstance-specific keys from the spec.options form label; the
restore options are driver-specific, so hardcoding one driver's keys is
misleading once other drivers land.
- Render a single "Same as backup" fallback for an omitted
spec.targetApplicationRef on the details page instead of the noisy
"-.-/-" produced by concatenating three missing path fallbacks.
- De-duplicate the time-block / time-icon / time-value component IDs
within the restorejob-details factory by prefixing them with
created-, started-at-, and completed-at-.
- Mark non-CRD-backed stock-project-factory-*-details sidebars (kube-*,
plan, backupjob, backup, restorejob) as static in
upsertMultipleSidebars so they pick up consistent managed-by labels
instead of being left unmanaged.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
ClusterIssuer solver referenced IngressClass "nginx" which does not
exist on cozystack clusters — real classes are named after tenant
namespaces (e.g. tenant-root). Cert issuance only worked because every
requesting Ingress overrode the ClusterIssuer via the legacy
acme.cert-manager.io/http01-ingress-class annotation.
Switch both sides to the modern cert-manager API (available since
cert-manager 1.12; cozystack ships 1.19.3):
- ClusterIssuer: http01.ingress.ingressClassName, value parameterized
from _cluster.expose-ingress (default "tenant-root")
- Ingress annotation: http01-ingress-ingressclassname
These must migrate together — mixing ingressClassName (ClusterIssuer)
with the old http01-ingress-class annotation triggers cert-manager's
"fields ingressClassName and class cannot be set at the same time"
validation and breaks issuance.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Update oldName and serverName field descriptions based on code review
feedback to avoid confusion about their actual roles:
- oldName: Remove misleading "(matches serverName in backup.info)"
text. This field represents the Kubernetes cluster resource name,
not the Barman server name.
- serverName: Provide clearer explanation that it's the S3 path prefix
(barmanObjectStore.serverName) used by the original cluster, and should
only be set when it differs from the Kubernetes resource name.
Updated in:
- values.yaml (source of truth for field documentation)
- types.go (Go API type comments)
- values.schema.json (JSON schema for validation)
- postgres.yaml (CRD with embedded OpenAPI schema)
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add serverName field to bootstrap configuration to explicitly specify
Barman server name from backup.info. This fixes "no target backup found"
errors when server_name in backup.info differs from Kubernetes cluster name.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Upgrades the vendored `victoria-metrics-operator` Helm chart from
`0.59.1` to `0.61.0` (operator appVersion `v0.68.1` → `v0.68.4`) by
re-running `make update` in
`packages/system/victoria-metrics-operator/`.
Picks up upstream bugfixes in the `v0.68.x` operator line:
- Correct `VMPodScrape` port for `VMAgent` / `VLAgent` (previously
misrouted, causing scrape failures)
- `StatefulSet` pod *deletion* (not eviction) when `maxUnavailable=100%`
— matches the minimum-downtime upgrade strategy
- `VMDistributed` PVC no longer owned by both the `StatefulSet` and the
top-level object
- Finalizer cleanup on core Kubernetes resources and correct finalizer
targeting for `VLAgent`/`VLogs`/`VMAgent`/`VMSingle`
- Ingest-only `VMSingle`/`VMAgent` no longer mount scrape config secrets
(avoids RBAC errors)
- `StatefulSet` recreation when immutable fields change
- Operator waits for PVC resize completion before continuing reconcile
Chart `0.60.0` is skipped deliberately: it switched the operator
Deployment's selector to `app.kubernetes.io/component`, which required
Deployment recreation on upgrade. `0.61.0` reverts that change, so
jumping straight from `0.59.1` to `0.61.0` avoids the disruption.
Local patch `patches/disable-ca-key-rotation.patch` still applies
cleanly against the new `templates/webhook.yaml`.
### Release note
```release-note
[monitoring] Upgrade victoria-metrics-operator chart to 0.61.0 (operator v0.68.4) for upstream STS/PVC bugfixes, correct VMPodScrape port, and finalizer cleanup.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for configuring Deployment rolling update strategy
* Added Deployment unhealthyPodEvictionPolicy for Pod Disruption Budgets
* Added webhook annotations configuration
* Extended CRD with statefulRollingUpdateStrategyBehavior and
vmauth.enabled settings
* **Bug Fixes**
* Reverted Deployment matchLabels change from release 0.60.0
* **Documentation**
* Added changelog links to documentation
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds @myasnikovdaniil to the default owners in `.github/CODEOWNERS` so
PRs automatically request review from me alongside the existing
maintainers.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated internal repository code ownership configuration to extend
responsibility coverage.
---
**Note:** This release contains only internal administrative updates
with no end-user-facing changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases the LINSTOR satellite startup probe `failureThreshold` from
the Kubernetes default of 3 (30 seconds) to 30 (300 seconds / 5
minutes).
The default startup probe is too aggressive for nodes with slow storage
initialization — satellites get killed and restarted before they have a
chance to come up. This adds an explicit `startupProbe` override in the
`LinstorSatelliteConfiguration` pod template, keeping the same
`tcpSocket` check on the `linstor` port but giving satellites
significantly more time to start.
### Release note
```release-note
fix(linstor): increase satellite startup probe failure threshold from 3 to 30 (5 minutes to start)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a startup health check for the LINSTOR satellite container to
improve pod reliability and ensure the service fully initializes before
accepting traffic.
* This change only introduces container startup probing behavior and
does not alter other runtime behavior or public interfaces.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The website repo is switching to a permanent `content/en/docs/next/`
trunk (cozystack/website#495). Released version directories are no
longer pre-created — the upstream release workflow is now responsible
for explicitly promoting `next/` → `vX.Y/` on new minor/major releases
via `make release-next`.
Update the `update-website-docs` job to match that contract:
- New `Determine if this release promotes next/` step parses the tag
and only enables promotion for final `vX.Y.Z` tags where
`content/en/docs/vX.Y/` does not already exist. Prereleases and
patch releases keep the old behaviour (target routing is handled
by the website Makefile on its own).
- New `Promote next/ to released version` step runs
`make release-next RELEASE_TAG=...` before `make update-all`, so
the subsequent `update-all` routes into the freshly-created
`vX.Y/` directory and `next/` stays untouched as the trunk for
the following release.
- `git add content hugo.yaml` to pick up the version registration
that `release-next` writes via `register_version.sh`.
- Convert the existing `update-all` step to use `env:` vars,
matching the new step and addressing the workflow-injection
hardening guidance.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Bumps the vendored victoria-metrics-operator chart from 0.59.1 to 0.61.0
(operator appVersion v0.68.1 to v0.68.4) via `make update`.
Picks up upstream bugfixes in the v0.68.x line: correct VMPodScrape port
for VMAgent/VLAgent, StatefulSet pod deletion when maxUnavailable=100%,
VMDistributed PVC ownership fix, finalizer cleanup, ingest-only mode not
mounting scrape config secrets, STS recreation on immutable field changes,
and PVC resize completion wait.
Chart 0.60.0 is skipped because it introduced a matchLabels change
requiring Deployment recreation that was reverted in 0.61.0.
Local patch disable-ca-key-rotation.patch reapplies cleanly.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* WorkloadMonitor labels with the workloads.cozystack.io/ prefix are now
propagated onto created Workloads; created Workloads always include the
reserved workloads.cozystack.io/monitor label and source-object labels
take precedence on conflicts.
* Helm app charts now add workloads.cozystack.io/resource-preset
metadata to WorkloadMonitor manifests.
* **Tests**
* Added tests covering label extraction, propagation, conflict
resolution, and backward compatibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Make the 5-minute timeout self-documenting by setting periodSeconds: 10
explicitly rather than relying on the Kubernetes default.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The default Kubernetes startup probe allows only 30 seconds (3 retries
× 10s) for LINSTOR satellites to become ready. This is insufficient on
nodes with slow storage initialization, causing unnecessary pod restarts.
Raise failureThreshold to 30, giving satellites up to 300 seconds (5
minutes) to complete startup.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- Increase memory limit from 500Mi to 512Mi
- Increase memory request from 100Mi to 256Mi
- Add startup probe with 60s timeout (12 attempts × 5s)
- Increase readiness/liveness initialDelaySeconds from 5/15 to 30s
This fixes OOMKilled crashes observed in production where kamaji
controller was being killed due to insufficient memory during startup.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The /1e9 divisor in gpu:{power,thermal}_throttle_fraction:rate5m was
derived empirically against DCGM 3.x on A10 — the counter documents
itself as microseconds but ticks in nanoseconds in practice. If a
future exporter release honors the documented units, pre-clamp values
would exceed 1.0 while clamp_max(..., 1) silently masks the drift,
plateauing every throttle fraction at 100% and making the panels
lie in unison.
Add a validation group that fires when the raw max/1e9 value exceeds
1.0 for 15m, so we notice and rescale to /1e6 before dashboards
silently mislead operators.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
pod:util_per_watt:avg5m divided two DCGM metrics without an explicit
on(...) clause, so the match used the intersection of their label
sets. If dcgm-exporter relabeling ever diverges between
DCGM_FI_DEV_GPU_UTIL and DCGM_FI_DEV_POWER_USAGE (e.g. a pod-mapping
label appears on one but not the other after a config change), the
entire result drops to empty silently. Pin the match to the labels we
group by so divergence becomes a missing side, not a missing rule.
Throttle fractions had a related shape problem: dcgm-exporter emits
one series per GPU for each pod-mapping combination. On a shared GPU
(restart races, MIG/MPS) the same physical counter appears under
multiple pod labels and downstream avg(...) panels get diluted by the
pod count. Fold duplicates with max by (Hostname, gpu, UUID) before
clamp_max so the fraction is tied to the physical GPU.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
kube-state-metrics keeps kube_pod_container_resource_requests series
for Failed/Succeeded pods until the apiserver garbage-collects them,
which could inflate :allocated beyond what tenants actually hold and
drive cluster:gpu_count:free negative.
Join the request metric against kube_pod_status_phase filtered to
Pending|Running — the canonical pattern from Kubernetes' own
container_resource recording rules — on both the cluster and namespace
aggregates. Add clamp_min(..., 0) on cluster:gpu_count:free as a
second line of defence against transient label drift between
kube-state-metrics and DCGM.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Drop the hardcoded metadata.namespace so the rule inherits the chart's
release namespace, and add an explicit empty params field on every
group for schema consistency. No behavior change.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The Pending GPU pods counter on gpu-quotas joined raw
kube_pod_container_resource_requests (per-container series) against
kube_pod_status_phase (per-pod series). Multi-container pods were
counted once per requesting container instead of once per pod, so the
widget over-reported whenever a Pending pod had more than one GPU
container. Collapse the requests to pod level before the join.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Pod-level panels on the efficiency dashboard and DCGM-level panels on
the performance dashboard ignored the $namespace template variable, so
changing it left the visualizations unchanged. Add the filter to each
query. Performance-side queries use the `$namespace|` empty-tolerant
form so host-level DCGM series without a namespace label remain
visible when a specific namespace is selected.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Explain why tolerations: [{operator: Exists}] is safe on the driver
compat DaemonSet: the nodeSelector already confines scheduling to GPU
nodes, so the blanket toleration only kicks in when those nodes carry
the dedicated=gpu / nvidia.com/gpu taints that the GPU Operator's
default policy and many deployments apply.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Clarify that the "Average utilization" panel on gpu-fleet reflects the
legacy NVML view (DCGM_FI_DEV_GPU_UTIL) rather than engine-active
profiling. For AI/LLM workloads the NVML number is optimistic; the
gpu-efficiency dashboard carries the profiling-based view.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Align pod:tensor_saturation:avg5m with namespace:tensor_active:avg and
DCGM's native 0..1 range by dropping the * 100 from the recording rule
and multiplying at display time in gpu-efficiency.json. Also scope
pod:util_per_watt:avg5m with avg by (Hostname, gpu, UUID, namespace,
pod) so the series mirrors pod:tensor_saturation's grouping and stays
usable in topk queries.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
gpu-fleet.json references DCGM_FI_DEV_POWER_MGMT_LIMIT for its
"TDP vs draw" panel, but the custom DCGM Exporter CSV did not declare
it, so the panel silently rendered "No data" on clusters using that
config. Declare the counter, fix the dashboards table in the
gpu-operator examples README, and add a bats test that cross-checks
every DCGM_FI_* reference in tracked dashboards and recording rules
against the union of the upstream default set (snapshotted under
hack/) and the project's custom CSV.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- gpu-efficiency: scope Tensor Saturation, Util-per-Watt and Power
Throttle stats to the $namespace selector. Cluster-wide means were
misleading when a user had narrowed the dashboard to specific
tenants — the headline numbers lied relative to the panels below.
- gpu-fleet: show per-node power draw as % of combined TDP cap
(DCGM_FI_DEV_POWER_MGMT_LIMIT) instead of raw watts. Thresholds
(60 / 80 %) generalize across GPU SKUs without per-model tuning.
- gpu-quotas: read cluster:gpu_count:allocated from the recording
rules instead of recomputing sum(kube_pod_container_resource_requests)
inline. Keeps the dashboard aligned with the canonical definition
in gpu-recording.rules.yaml so the two can't drift.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- namespace:gpu_count:sum — never consumed by any tracked dashboard;
the billable view is already covered by namespace:gpu_count:allocated,
and the admin view by cluster:gpu_count:allocated.
- namespace:energy_joules:sum — no panel integrates joules; kWh
readings on the tenant dashboard compute their own integrations
from namespace:power_watts:sum.
- pod:tensor_to_nvml_ratio:avg5m — interesting tenant signal in
theory, but not wired into any panel and carrying it just burns
cardinality on large fleets.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- gpu-fleet: cluster-wide admin view — inventory, capacity (total /
allocated / free), per-node utilization and power, throttling,
temperatures, XID errors.
- gpu-tenants: per-namespace view — live allocation, utilization,
tensor saturation, power, and 24h GPU-hours / kWh integrations for
billing inputs.
Register both under gpu/* in dashboards-infra.list so they ship as
GrafanaDashboard CRs and fall under the bats cross-check introduced
earlier on this branch.
Update examples/README to spell out which DCGM counters each of the
five gpu/* dashboards actually needs on top of the upstream default
CSV — gpu-performance needs profiling and throttling counters,
gpu-efficiency needs profiling, gpu-tenants needs only
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE for its tensor panel, and gpu-fleet
and gpu-quotas work on the default counter set alone.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Align namespace:gpu_count:allocated with every other namespace:* rule
by filtering out cozy-*/kube-*. All other per-namespace rules
(gpu_util, tensor_active, fb_used_bytes, power_watts) already exclude
system namespaces, so the label set produced by :allocated diverged
from them — any dashboard variable or join that reads across these
rules could end up with a different namespace list depending on which
rule supplied the :allocated column.
Trade-off: per-namespace GPU accounting for system workloads is no
longer available through this rule. If it's ever needed, add a
dedicated system:gpu_count:allocated rather than widening this one —
the "billable tenant view" invariant is what the filter is protecting.
Cluster-level cluster:gpu_count:allocated intentionally keeps system
pods so it stays aligned with cluster:gpu_count:total and
cluster:gpu_count:free remains meaningful. As a consequence,
sum(namespace:gpu_count:allocated) no longer equals
cluster:gpu_count:allocated; the delta is system-pod GPU usage, which
is fine for the cluster-admin view.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Catch dangling references at PR time: every recording-rule name used
inside a tracked GPU dashboard must exist in
packages/system/monitoring-agents/alerts/gpu-recording.rules.yaml. The
first iteration of gpu-efficiency.json shipped panels keyed on
pod:tensor_saturation:avg5m without the rule defined; the test fails
on exactly that class of bug.
Scoped to dashboards listed under gpu/* in dashboards-infra.list, so
untracked drafts stay out of scope until they are registered. Reverse
direction (rule defined but unused) is intentionally NOT enforced —
some rules exist for ad-hoc PromQL or upcoming dashboards.
Auto-discovered by make bats-unit-tests via hack/cozytest.sh.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The previous wording implied that the entire custom DCGM CSV was
required by the recording rules. In fact only the profiling counters
(DCGM_FI_PROF_*) need to be added on top of the upstream defaults —
everything else the rules consume is already in default-counters.csv.
Add a Verification status block flagging that the minimum-set claim is
derived from the DCGM Exporter version pinned in the currently shipped
gpu-operator package and must be re-checked when that package moves to
a newer release.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Restrict the nvidia-driver-compat DaemonSet to nodes labelled
nvidia.com/gpu.present=true (NFD/GPU Operator label). Without the
nodeSelector it was scheduling onto every node — control-plane and
CPU-only workers included — burning a privileged pod slot per host
for no benefit.
Add resource requests and limits to the init and pause containers so
the DaemonSet stays within control-plane budget on small clusters.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
- Drop gpu.recording.30s group: per-GPU 30s aggregates had no consumers
in tracked dashboards, only burned cardinality.
- Drop namespace:gpu_allocated_count:gauge: identical expression to
namespace:gpu_count:sum under a different name.
- Reground :allocated on kube_pod_container_resource_requests so it
reflects what tenants requested (Pending+Running) rather than what
DCGM currently sees. namespace:gpu_count:sum stays DCGM-based and
represents actually-running pods; the gap between the two is the
signal admins want.
- Add namespace:gpu_count:allocated as the per-namespace counterpart.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Before this change oauth2-proxy fronting linstor-gui only enforced that
the user could authenticate against the `cozy` Keycloak realm
(`--email-domain=*`, no group restriction). Any realm user could reach
the UI and, through it, the LINSTOR controller REST API — which the
gatekeeper proxies with a static mTLS client cert and no per-user RBAC.
Add `--allowed-group=cozystack-cluster-admin` and include `groups` in
the OIDC scope so the claim is present at validation time. The
`cozystack-cluster-admin` KeycloakRealmGroup and the `groups` client
scope are already provisioned by keycloak-configure, so no cluster-wide
changes are needed.
Assisted-By: Claude <noreply@anthropic.com>
(cherry picked from commit 9b54e46723)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Backport of #2415 to release-1.3.
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated README to document that LINSTOR GUI access is restricted to
members of the `cozystack-cluster-admin` Keycloak group. Non-members
receive a 403 error.
* **Chores**
* Implemented group membership validation for LINSTOR GUI access
control.
* Added validation tests for group membership enforcement.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Before this change oauth2-proxy fronting linstor-gui only enforced that
the user could authenticate against the `cozy` Keycloak realm
(`--email-domain=*`, no group restriction). Any realm user could reach
the UI and, through it, the LINSTOR controller REST API — which the
gatekeeper proxies with a static mTLS client cert and no per-user RBAC.
Add `--allowed-group=cozystack-cluster-admin` and include `groups` in
the OIDC scope so the claim is present at validation time. The
`cozystack-cluster-admin` KeycloakRealmGroup and the `groups` client
scope are already provisioned by keycloak-configure, so no cluster-wide
changes are needed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 9b54e46723)
Strip Grafana export boilerplate (__inputs, __elements, __requires,
default annotations, embedded datasource inputs) and tighten panel
layouts across the three GPU dashboards. All three continue to use
the $ds_prometheus template variable.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Revise gpu-performance and add two new dashboards, registered in
dashboards-infra.list:
- gpu-efficiency (GPU Efficiency Score) — utilization vs. capacity
and workload efficiency signals.
- gpu-quotas (GPU Quotas & Allocation) — per-namespace requested vs.
used GPUs for tenant capacity planning.
All three dashboards use the $ds_prometheus template variable, per
the project convention.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add reference manifests (not templates) under
packages/system/gpu-operator/examples/ documenting one working
configuration for running CUDA workloads directly in pods on a Talos
cluster, with DCGM metrics that drive the gpu/gpu-performance
dashboard.
- values-native-talos.yaml: Cozystack Package values that disable the
sandbox path, enable the device plugin, and wire DCGM to the custom
metrics ConfigMap.
- dcgm-custom-metrics.yaml: ConfigMap extending the default DCGM CSV
with profiling, ECC, throttling and energy counters used by the
dashboard and recording rules.
- nvidia-driver-compat.yaml: DaemonSet that stages libnvidia-ml.so.1
and nvidia-smi from the Talos glibc tree into a location the
gpu-operator validator inspects. Workaround for
NVIDIA/gpu-operator#1687.
- README.md: explains why these are shipped as references rather than
first-class templates (sandbox vs native is a deployment choice),
and how the pieces connect.
The out-of-the-box values-talos.yaml still targets the sandbox (VFIO
passthrough) scenario. Operators who want native pod GPU workloads can
start from these references.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add the gpu/gpu-performance dashboard and register it in the infra
dashboard list. The dashboard provides:
- Cluster overview: total/allocated GPUs, average utilization,
aggregate power draw.
- Utilization: GPU util (NVML), tensor pipe active (realistic load
for LLM/AI workloads), graphics engine active, memory copy util.
- Memory: VRAM used/free per GPU.
- Power and temperature per GPU.
- Health: XID errors, power and thermal throttling.
The dashboard relies on DCGM_FI_* metrics plus the cluster:gpu_* and
namespace:gpu_* recording rules added to monitoring-agents.
The JSON follows the cozystack convention — Prometheus data source is
selected via the $ds_prometheus template variable.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add VMRule with recording rules for DCGM metrics at three levels:
- gpu.recording.30s: per-GPU aggregates over 30s windows
- gpu.recording.cluster.1m: cluster-wide totals for overview panels
- gpu.recording.namespace.1m: per-namespace aggregates for tenant
reporting and GPU-hour calculations
The rules are safe to ship on clusters without DCGM — they evaluate to
empty series when no matching metrics are scraped.
Used by dashboards/gpu/gpu-performance.json.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
## What this PR does
Adds BucketClaim support to WorkloadMonitorReconciler, bringing S3
buckets in line with how Pods, PVCs, and Services are already tracked as
Workload resources.
- Add WorkloadMonitor CR to the bucket Helm chart with
`app.kubernetes.io/instance` label on BucketClaim for selector matching
- Add `reconcileBucketClaimForMonitor()` following the same pattern as
`reconcilePVCForMonitor()` — watches COSI `BucketClaim` objects and
creates corresponding `Workload` CRDs
- Query SeaweedFS bucket size metrics (logical + physical) from
VictoriaMetrics, with the monitoring endpoint resolved automatically
from the `namespace.cozystack.io/monitoring` namespace label
- COSI API types dependency (`container-object-storage-interface-api`)
for typed BucketClaim access
No configuration flags needed — the controller discovers the monitoring
stack for each tenant namespace automatically.
### Result
When a BucketClaim is matched by a WorkloadMonitor, the controller
creates a Workload with S3 storage metrics:
```yaml
apiVersion: cozystack.io/v1alpha1
kind: Workload
metadata:
name: bucket-bucket-test-billing
namespace: tenant-testing
labels:
workloads.cozystack.io/monitor: bucket-test-billing
ownerReferences:
- apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketClaim
name: bucket-test-billing
status:
kind: bucket
type: s3
operational: true
resources:
s3-buckets: "1"
s3-storage-bytes: "10485864" # 10 MB logical
s3-physical-storage-bytes: "20971728" # 20 MB physical (replication factor 2)
```
- `s3-storage-bytes` — logical size (what the user stored), from
`SeaweedFS_s3_bucket_size_bytes`
- `s3-physical-storage-bytes` — physical size (with replicas), from
`SeaweedFS_s3_bucket_physical_size_bytes`
- When monitoring is not configured for the namespace, only `s3-buckets:
1` is tracked
- Sizes refresh every 60 seconds via `RequeueAfter`
Tested on a live dev cluster with SeaweedFS deployed in `tenant-root`
and a bucket created in `tenant-testing`.
### Release note
```release-note
[apps] Add WorkloadMonitor to bucket application. BucketClaims are now tracked as Workload resources with S3 storage size metrics resolved automatically from the tenant monitoring stack.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Monitor COSI bucket claims: creates/updates Workload records
reflecting bucket readiness and requeues periodically when buckets
exist.
* Prometheus-backed bucket storage metrics, with SeaweedFS-aware sizing
when available.
* Helm chart additions: WorkloadMonitor resource and instance label on
BucketClaim templates.
* **Tests**
* Unit tests covering bucket-monitor flows, metric querying, URL
resolution, and requeue behavior.
* **Chores**
* Updated module dependencies to enable COSI integration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace http.DefaultClient with a package-level *http.Client with an
explicit 10-second timeout. Avoids sharing the process-wide default
transport with other libraries.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Pass bucket names from BucketClaim.Status.BucketName into the PromQL
query as a bucket=~"name1|name2" filter. This prevents O(N²) load
where N WorkloadMonitors each fetch all buckets globally.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Wrap resolvePrometheusURL and queryAllBucketMetrics in a
len(bucketClaimList.Items) > 0 guard. Avoids unnecessary namespace
GET and HTTP request to Prometheus on every reconcile of non-bucket
WorkloadMonitors (postgres, redis, kubernetes, etc.).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Replace 2×N per-bucket HTTP requests with a single query that fetches
all SeaweedFS bucket size metrics at once:
{__name__=~"SeaweedFS_s3_bucket_(size|physical_size)_bytes"}
Results are keyed by bucket name in memory, then looked up per
BucketClaim. This reduces HTTP round-trips from 2N+1 to 2 (one
namespace lookup + one Prometheus query) regardless of bucket count.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Move resolvePrometheusURL call before the BucketClaim loop and pass
the URL as parameter. Avoids redundant namespace lookups and reduces
reconcile time from 2×N+1 to 2×N HTTP calls (where N = BucketClaims).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Change queryPrometheusMetric to return (int64, bool) so callers can
emit s3-storage-bytes=0 for empty buckets while omitting the field
entirely when monitoring is not configured.
Also:
- Add RBAC marker for core/namespaces GET (used by resolvePrometheusURL)
- Log namespace read errors instead of silently returning empty URL
- Add TestQueryPrometheusMetricZeroValue test
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Bucket existence is already tracked via LifetimeHours. The s3-buckets
count resource produced a meaningless "s3-buckets-Hours" fallback type
in billing. Only storage size metrics (s3-storage-bytes,
s3-physical-storage-bytes) are now set on bucket Workloads.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Initialize workload.Labels map inside CreateOrUpdate mutate function
to prevent nil map panic when existing Workload has no labels
- Use strconv.ParseFloat instead of resource.ParseQuantity for
Prometheus metric values, which may use scientific notation
(e.g. "1.048576e+06") that ParseQuantity does not support
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Replace static --prometheus-url flag with dynamic resolution from
namespace.cozystack.io/monitoring label. Each tenant namespace knows
which tenant hosts its monitoring stack, so the controller constructs
the vmselect URL automatically. This correctly handles multi-tenant
setups where different tenants may use different monitoring instances.
- Remove PrometheusURL field from WorkloadMonitorReconciler struct
- Remove --prometheus-url flag and prometheusUrl chart value
- Add resolvePrometheusURL() that reads namespace label
- queryPrometheusMetric() now accepts prometheusBaseURL as parameter
- Add tests for resolvePrometheusURL with and without label
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Set replicas/minReplicas to 0 in bucket WorkloadMonitor (buckets
have no pods, minReplicas=1 would mark monitor as not operational)
- Remove dead IsNotFound check on List (List returns empty, not 404)
- Trim trailing slash from PrometheusURL before path append
- Add strings import for TrimRight
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Check resp.StatusCode before parsing Prometheus response
- Limit response body read to 1 MB via io.LimitReader
- Use strings.HasPrefix in test instead of fragile slice indexing
- Add TestQueryPrometheusMetricServerError for 500 responses
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Query SeaweedFS_s3_bucket_physical_size_bytes alongside the logical
size metric. Physical size includes all replicas and reflects actual
disk usage, while logical size reflects what the user stored.
Refactor queryBucketSizeBytes into generic queryPrometheusMetric
to reuse for both metrics.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Pass --prometheus-url to the controller container when configured in
values. This enables querying SeaweedFS bucket size metrics from a
Prometheus-compatible API for S3 bucket billing.
RBAC already covers BucketClaim access via existing wildcard rule
(apiGroups: ['*'], resources: ['*'], verbs: ["get", "list", "watch"]).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Query SeaweedFS_s3_bucket_size_bytes from a Prometheus-compatible API
to populate s3-storage-bytes resource on bucket Workloads. The
Prometheus URL is configurable via --prometheus-url flag. When set,
bucket WorkloadMonitors are requeued every 60s to keep sizes current.
When Prometheus is not configured, buckets still get tracked with
s3-buckets=1 for existence-based billing.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Add reconcileBucketClaimForMonitor() that watches COSI BucketClaim
objects and creates Workload CRDs with s3-buckets resource, following
the same pattern as PVC and Service reconcilers. This enables the
billing pipeline to discover and track S3 buckets per tenant.
Changes:
- Add COSI API types dependency (container-object-storage-interface-api)
- Register cosiv1alpha1 scheme in controller main
- Add BucketClaim watch in SetupWithManager
- Add BucketClaim list + reconcile in Reconcile loop
- Add RBAC annotation for objectstorage.k8s.io/bucketclaims
- Add unit tests for BucketClaim reconciliation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Add WorkloadMonitor CR to bucket Helm chart so that the billing
pipeline can discover and track S3 buckets. Add instance label
to BucketClaim metadata for WorkloadMonitor selector matching.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
## What this PR does
Tightens access control on the `linstor-gui` Ingress so it is reachable
only by members of the `cozystack-cluster-admin` Keycloak group (the
same group that grants cluster-admin RBAC on the host cluster).
Before this change, the oauth2-proxy gatekeeper in front of
`linstor-gui` enforced only that the user could authenticate against the
`cozy` realm (`--email-domain=*`, no group restriction). Any realm user
— including tenant-scoped accounts — could reach the UI and the
underlying LINSTOR controller REST API, which the gatekeeper proxies
with a static mTLS client cert and no per-user RBAC.
Changes:
- `packages/system/linstor-gui/templates/gatekeeper.yaml` — add
`--allowed-group=cozystack-cluster-admin` and include `groups` in
`--scope` so the claim is available for validation. Extend the
top-of-file rationale block to document the gate.
- `packages/system/linstor-gui/tests/ingress_auth_test.yaml` — assert
both new args on the rendered Deployment.
- `packages/system/linstor-gui/README.md` — document the group
restriction under "Option 1 — Keycloak-protected Ingress".
No Keycloak-side changes are required: the `cozystack-cluster-admin`
KeycloakRealmGroup and the `groups` client scope are already provisioned
by `keycloak-configure`.
Port-forward access (Option 2) is unchanged; it remains gated by kubectl
RBAC on `cozy-linstor`.
### Release note
```release-note
fix(linstor): restrict `linstor-gui` Ingress access to members of the `cozystack-cluster-admin` Keycloak group.
```
Before this change oauth2-proxy fronting linstor-gui only enforced that
the user could authenticate against the `cozy` Keycloak realm
(`--email-domain=*`, no group restriction). Any realm user could reach
the UI and, through it, the LINSTOR controller REST API — which the
gatekeeper proxies with a static mTLS client cert and no per-user RBAC.
Add `--allowed-group=cozystack-cluster-admin` and include `groups` in
the OIDC scope so the claim is present at validation time. The
`cozystack-cluster-admin` KeycloakRealmGroup and the `groups` client
scope are already provisioned by keycloak-configure, so no cluster-wide
changes are needed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The test body asserts .status.history[].status extraction, but the
test name still referenced the old installFailures counter (leftover
from when the guard used that field before switching to status.history
to avoid ClearFailures zeroing the counters on successful reconcile).
Address review feedback from coderabbitai on
hack/remediation-guard.bats:84: rename so grep for what the test
actually pins matches the test name.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Log .status.history regardless of content so a silently empty result
(Flux field rename) shows up in CI logs, and treat empty history on
a Ready HelmRelease as a distinct failure. A Ready HR by definition
has at least one snapshot; anything else is a shape-drift signal.
- Replace the unquoted heredoc in remediation-guard.sh with a printf |
grep pipeline. printf %s treats statuses as literal payload (no $
expansion surprises for future callers), grep --quiet --extended-regexp
returns exit status the caller can forward directly.
- Share the etcd-absent values file between both invariant tests
(packages/apps/kubernetes/tests/values-ci-no-etcd.yaml) instead of
duplicating the --set block.
- Fix typo "override applied" -> "override is applied" in the
Kubernetes ApplicationDefinition.
- Add a coupling comment in the ApplicationDefinition annotation that
points at the wait-for-kubeconfig init deadline in _helpers.tpl, so
a future operator raising the HR timeout updates the init deadline
too.
- Clarify the per-annotation timeout comment in rest.go so it stops
implying the feature is Kubernetes-only (it is not - only today's
one user is).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Pulls the release.cozystack.io/helm-install-timeout parsing out of
start.go into ParseHelmInstallTimeoutAnnotation in pkg/config. The
helper rejects units that time.ParseDuration accepts but Flux
helm-controller rejects (ns, us, µs): feeding one of those through
would cause the HelmRelease admission webhook to reject the object at
install time, giving a silent drop to flux defaults that is hard to
debug. Fail loudly at cozystack-api startup instead.
Adds a table-driven unit test covering: unset (empty), accepted units
ms/s/m/h, compound 2h30m, decimal 1.5m, and the rejected shapes
(bare digits, garbage, negative, ns/us/µs). The test lives in
pkg/config so it runs under the existing go-unit-tests make target.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
17 child HelmReleases (cilium, coredns, csi, cert-manager,
metrics-server, ...) referenced *-admin-kubeconfig via
kubeConfig.secretRef and rendered even when _namespace.etcd was empty.
On an etcd-less tenant each one sat in NotReady forever because the
admin-kubeconfig Secret only exists after a KamajiControlPlane
reconciles, and KamajiControlPlane now only renders when etcd is set.
The outcome contradicted the "beacon only" contract claimed in the
soft-skip commit.
Extend the existing addon guards to also require _namespace.etcd, and
wrap the four unconditional HelmReleases (csi, metrics-server,
prometheus-operator-crds, volumesnapshot-crd) plus the always-on
cilium/coredns HR resources in the same gate. Add an invariant bats
test that renders the whole chart with etcd empty and asserts zero
HelmReleases reference *-admin-kubeconfig.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The image-busybox Makefile target + images/busybox/Dockerfile wrapper
just rebuilt an upstream busybox digest as ghcr.io/cozystack/cozystack/
busybox. Payload is a one-shot sh loop run once per pod; the pinned
upstream digest is already immutable, so maintaining a private mirror
adds churn (rebuild on every release) for no real hardening benefit.
Delete the wrapper and reference docker.io/library/busybox:<digest>
directly from images/busybox.tag. Document the choice in _helpers.tpl.
Also drop the false coupling in the go table test: the "unrelated
kind without configured timeout" case used the real kind name Qdrant,
which tied the test to the Qdrant ApplicationDefinition for no reason.
Switch to a clearly fictional kind so a future Qdrant timeout override
does not break this assertion.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
CI runs make unit-tests on every PR, which already covers helm
unittests and bats (hack/admin-kubeconfig-invariant.bats and
hack/remediation-guard.bats are both picked up by the existing
hack/*.bats glob). What was missing was any go test invocation.
Add a go-unit-tests target scoped to pkg/registry, pkg/config, and
pkg/cmd/server - the cozystack-api surface this repo actually owns
and tests in-tree. Running go test ./... pulls in generated-code
round-trip suites whose behavior is governed by generator tool
versions outside this repo's control; those are better exercised
from their own generator workflows.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Flux helm-controller's ClearFailures() zeroes installFailures and
upgradeFailures on every successful reconciliation (see the upstream
HelmReleaseStatus method). The previous guard ran after the HelmRelease
was Ready, at which point the counters were always 0 - the assertion
was vacuous and would have passed against a reverted fix.
Switch to .status.history, which retains per-revision release
Snapshots that survive a subsequent successful reconciliation. A
remediation cycle leaves behind a Snapshot with status=uninstalled
(the install-remediation code path) or status=failed (Helm release
failure that remediation then uninstalled). Either one signals the
race actually fired.
Rewrite the bats unit tests to cover: empty history, deployed-only,
deployed+superseded (happy path - not detected), single failed,
single uninstalled, uninstalled-then-deployed, and deployed-then-failed
(all detected). The pinned-shape test feeds a realistic HR status
snippet through yq the same way run-kubernetes.sh does via kubectl
-o jsonpath.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The soft-skip wrap in cluster.yaml only silenced Cluster and
KamajiControlPlane rendering when _namespace.etcd is empty. The three
CP-side Deployments (cluster-autoscaler, kccm, kcsi-controller) still
rendered, their wait-for-kubeconfig init containers CrashLoopBackOff'd
forever (no KamajiControlPlane = no admin-kubeconfig Secret),
HelmRelease hit its 15m wait timeout and triggered the very install
remediation cycle the rest of this PR prevents. Self-contradiction.
Wrap each of the three Deployment templates in
{{- if .Values._namespace.etcd }}...{{- end }} so they render only
when there is a DataStore to back them. Add an invariant bats test
that renders the whole chart with etcd empty and asserts zero
Deployments reference *-admin-kubeconfig.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
run-kubernetes.sh extracts .status.installFailures and
.status.upgradeFailures via kubectl -o jsonpath. If a future flux
release renames the counters, kubectl returns empty, the guard reports
no cycle, and e2e silently misses real remediation loops.
Add a bats unit test that feeds a pinned HelmRelease v2 status snippet
through the same jsonpath and asserts the extraction still yields the
expected values. Also leave a pointer comment in run-kubernetes.sh so a
future flux bump surfaces the version coupling in review.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The per-template unittests in packages/apps/kubernetes/tests/ assert
that cluster-autoscaler, kccm, and the csi controller each mount the
admin-kubeconfig Secret optional and carry the wait-for-kubeconfig
init. That locks in today's three Deployments by name - a fourth
Deployment that mounts the same Secret but forgets the guard would
slip past them.
Add a bats-unit test that renders the entire chart, enumerates every
Deployment whose spec mounts a Secret ending in -admin-kubeconfig, and
asserts optional:true plus wait-for-kubeconfig init on all of them.
Verified by temporarily removing optional:true from csi/deploy.yaml:
the test correctly flagged invariant-kcsi-controller as an offender.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace the hardcoded r.kindName == "Kubernetes" switch in rest.go
with a config-driven path. The ApplicationDefinition CR now accepts a
release.cozystack.io/helm-install-timeout annotation that is parsed
at cozystack-api startup into config.ReleaseConfig.HelmInstallTimeout
and applied to both Install.Timeout and Upgrade.Timeout on the
rendered HelmRelease. Applications that leave the annotation unset
keep flux defaults so their failed installs remediate on the normal
cadence - only the Kubernetes kind carries the override and gets a
15m budget. New kinds with a similar race can opt in by setting the
same annotation; no rest.go patch needed.
Kubernetes-rd sets the annotation to 15m. Table-driven test in
rest_timeout_test.go covers three cases: Kubernetes with 15m, Qdrant
unset, and an arbitrary future kind with 20m - all of which pin the
Remediation.Retries == -1 contract as well.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The three control-plane-side Deployments (cluster-autoscaler, kccm,
kcsi-controller) carried three copies of the same 20-line init
container. That already drifted: the CSI copy used 4-space nesting
while the other two used 2-space. Any future update to the image,
the deadline, or the poll script had to land in three places or
silently diverge.
Extract the block into a new kubernetes.waitForAdminKubeconfig helper
in templates/_helpers.tpl and include it at each call site. Tighten
the deadline from 20m to 10m so it stays strictly below the 15m
HelmRelease Install.Timeout and the CrashLoopBackOff surfaces in
dashboards before flux remediation can fire. Also clarify the wait
message so operators debugging a stuck init container do not chase
Kamaji for what is actually kubelet's optional-Secret refresh cadence.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A hard helm fail in cluster.yaml made every cold bootstrap racy: if
the parent Tenant chart had not yet populated _namespace.etcd in
cozystack-values when the Kubernetes HelmRelease first reconciled, the
fail fired, install.remediation triggered, installFailures incremented
and the new e2e remediation-guard flagged it as a bug. That directly
contradicts the race the rest of this PR is trying to close.
Replace fail with a graceful skip: render only a status-beacon
ConfigMap (test-awaiting-etcd) when etcd is empty, wrap all
CAPI/Kamaji resources in {{ if $etcd }}. The HelmRelease installs
successfully and goes Ready; flux retries on its 5m interval and picks
up the DataStore as soon as the Tenant chart finishes reconciling.
Update the helm unittest: positive test still asserts dataStoreName on
KamajiControlPlane; the negative test now asserts exactly one
ConfigMap document with status=awaiting-etcd, no Cluster / KCP /
KubevirtCluster / WorkloadMonitor rendered.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Cap the wait-for-kubeconfig init container at 20m. If Kamaji genuinely
fails to produce the admin-kubeconfig Secret (misconfigured tenant,
etcd outage after the guard already passed, Kamaji crash-loop), the
pod now exits non-zero and goes CrashLoopBackOff so the failure is
visible in dashboards, instead of silently sleeping in Init forever
and leaving only the flux helm-wait timeout to surface the problem.
Reword the etcd DataStore guard to reference the parent Tenant
application's etcd flag (not .Values.etcd of the Kubernetes chart,
which is a different chart). Update the helm unittest errorPattern
accordingly.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds an assertion that cluster.yaml renders successfully when
_namespace.etcd is set and produces a KamajiControlPlane whose
dataStoreName equals the tenant's etcd DataStore name. Without this
positive case a future edit that inverts or removes the existing etcd
guard would pass the suite as long as the negative case still fails.
Also adds documentSelector: kind=Deployment to the kccm and csi
controller assertions so the jsonpath filter operates on a single
document, matching the cluster-autoscaler case and removing
reliance on helm-unittest filter-vs-single-value coercion.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds images/busybox/Dockerfile and an image-busybox Makefile target
that mirror the same pattern as the rest of this chart's images (the
Dockerfile pins the upstream busybox by digest; the Makefile target
builds and tags for ghcr.io/cozystack/cozystack/busybox the same way
cluster-autoscaler et al. are handled). Also wires it into the
umbrella image target so 'make image' rebuilds everything.
Until the first release build runs image-busybox and rewrites the
.tag to point at ghcr.io, the .tag keeps a fully-qualified
docker.io/library/busybox:1.37.0@sha256:... reference so pods do not
silently resolve the short name via the default registry and pulls
remain immutable by digest. The release workflow overwrites this
file the same way it does for the other images.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The previous change set Install.Timeout and Upgrade.Timeout to 15m
on every Application's parent HelmRelease, but the admin-kubeconfig
race documented in #2412 is specific to the Kubernetes kind: only its
parent chart creates CAPI/Kamaji resources whose admin-kubeconfig
Secret is asynchronously provisioned and mounted by Deployments in
the same chart. Other kinds (Qdrant, MongoDB, Postgres, ...) have no
such race and should not have their failed installs linger three times
longer before flux triggers remediation.
Gate the timeout on r.kindName == "Kubernetes". Rewrite
rest_timeout_test.go to cover both sides: Kubernetes must get a
>= 15m timeout, other kinds must keep the flux defaults. Both tests
also pin Install/Upgrade Remediation.Retries == -1 so a future edit
that removes unbounded remediation would show up as a red test.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Shell && and || have equal precedence and left-to-right associativity,
so the previous guard parsed as (((A && B) || C) && D) and silently
passed on the canonical failure mode: install_failures=1 with an empty
upgrade_failures.
Extract the check into helmrelease_has_remediation_cycle() in a
dedicated helper sourced from run-kubernetes.sh, and add unit tests
under hack/remediation-guard.bats that pin the expected behavior for
every combination of empty, zero, and positive counter values.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Before cleanup, inspect the parent HelmRelease installFailures and
upgradeFailures counters. A non-zero value means flux helm-controller
hit its wait timeout, ran install/upgrade remediation (uninstall),
and re-installed - the exact race condition this PR closes. Fail the
bats test in that case so the signal surfaces in CI instead of being
masked by a green retry.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When a Kubernetes tenant is created without a parent tenant that has
etcd enabled, .Values._namespace.etcd is empty and the rendered
KamajiControlPlane spec carries an empty dataStoreName. The Kamaji
admission webhook then rejects every TenantControlPlane create with
"tenant-root DataStore does not exist" and the control plane never
comes up.
Add a helm template-level guard that fails rendering with a
descriptive, actionable error message before the HelmRelease even
reaches the webhook. This also closes the narrow race where a
Kubernetes HelmRelease reconciles before the etcd HelmRelease has
created the DataStore CR - flux retries on its interval and picks up
the DataStore once it appears.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Three Deployments in the Kubernetes app chart mount the tenant
admin-kubeconfig Secret directly as a volume: cluster-autoscaler,
kccm, and the kcsi controller. That Secret is provisioned
asynchronously by Kamaji after control-plane bootstrap, so on a fresh
install the pods used to hit FailedMount and the parent HelmRelease
ran out of its wait budget.
Mark the Secret volume optional and add a wait-for-kubeconfig
initContainer that polls the mounted path until the Secret appears.
Kubelet remounts the optional Secret within its sync period once
Kamaji publishes it, the init container exits, and the main container
starts cleanly. The Deployment becomes Available without the helm-wait
ever seeing a FailedMount.
Pins a busybox image for the init container via
images/busybox.tag (same format as the other pinned tags in this
chart).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds failing helm unittest suite for packages/apps/kubernetes covering:
- cluster-autoscaler, kccm, and csi controller Deployments mount the
admin-kubeconfig Secret with optional: true
- each of those Deployments has a wait-for-kubeconfig initContainer
that mounts the same kubeconfig path
- cluster.yaml renders a helm fail with a descriptive message when the
tenant has no etcd DataStore (empty _namespace.etcd)
Also wires up a test target in the chart Makefile so helm-unit-tests.sh
picks it up.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Parent HelmRelease created by cozystack-api for Kubernetes tenants
contains CAPI/Kamaji resources (Cluster, KamajiControlPlane,
MachineDeployment) that asynchronously provision the
*-admin-kubeconfig Secret. Three Deployments in the same chart
(cluster-autoscaler, kccm, kcsi-controller) mount that Secret
directly, so the helm-wait cannot complete until control-plane
bootstrap finishes.
Default flux helm-controller timeout is too short for a cold-node
first-tenant bootstrap (image pull + etcd bootstrap + apiserver Ready
+ admin-kubeconfig generation routinely exceed it). On timeout,
install.remediation triggers uninstall, which removes the Cluster CR
and restarts the cycle indefinitely.
Bumping Install.Timeout and Upgrade.Timeout to 15m gives realistic
bootstrap headroom while remaining bounded.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Adds a failing unit test for convertApplicationToHelmRelease asserting
that Install.Timeout and Upgrade.Timeout are at least 15 minutes. The
default flux helm-controller timeout is too short to cover cold-start
Kamaji control-plane bootstrap (image pull + etcd bootstrap + apiserver
Ready + admin-kubeconfig Secret generation) and causes install
remediation loops.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds the v1.3.0-rc.1 release changelog covering all changes since
v1.2.0, and updates the changelog agent instructions to include two new
side repositories.
**Changelog** (`docs/changelogs/v1.3.0-rc.1.md`):
- Feature Highlights: storage-aware scheduling, LINSTOR GUI, VM Default
Images, WorkloadsReady conditions, cross-namespace VM backup restore
- Covers main repo, website, talm, ansible-cozystack, and
external-apps-example changes
- Backported fixes marked with *(backported to v1.2.x)* annotation
**Agent instructions** (`docs/agents/changelog.md`):
- Adds `external-apps-example` and `ansible-cozystack` to the optional
repositories list
### Screenshots
N/A — documentation only.
### Release note
```release-note
docs(changelog): add changelog for v1.3.0-rc.1
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated the README's introductory description to refine the platform
positioning and improve clarity on its core capabilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds two features to improve application observability in the dashboard,
plus a bug fix:
### 1. WorkloadsReady condition on Application status
- Queries WorkloadMonitor resources to determine if all application pods
are running
- Exposes `WorkloadsReady` as a separate condition alongside `Ready`
- `Ready` continues to reflect HelmRelease state only — no override — to
preserve backward compatibility with existing tooling (kubectl wait,
GitOps health checks) and avoid false-negative Ready=False during normal
startup windows
- Handles three states: operational, not operational, unknown (pending
reconciliation)
- Fails open on WorkloadMonitor query errors (prefers availability)
- Integrates with Application Watch to emit MODIFIED events on
WorkloadMonitor changes
- Registers cozystack.io/v1alpha1 types in API server scheme with
informer cache
### 2. Events tab in dashboard
- Shows Kubernetes Events scoped to the application's namespace
- Uses status.namespace for Tenant applications (consistent with
Resource Quotas tab)
- Includes both lastTimestamp and eventTime columns for Kubernetes
version compatibility
### 3. Bug fix: WorkloadMonitor Operational status persistence
- Operational field was written to stale `monitor` variable instead of
`fresh` inside RetryOnConflict, so it was never persisted to the cluster
Closes#2359Closes#2360
### Release note
```release-note
[dashboard] Added Events tab to application detail pages showing namespace-scoped Kubernetes Events
[application] Added WorkloadsReady condition exposing aggregated WorkloadMonitor status
[workloadmonitor] Fixed bug where Operational status was never persisted to the cluster
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vm-default-images] Added package that brings set of images that can be used clusterwide
[vm-disk] Updated source "image" for prettier dropdown selection
[vm-disk] Added new source for vm-disk called disk - to use as source vm-disk from same namespace.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Clone VM disks by specifying an existing vm-disk as the source.
* Global default image collection and chart to publish pre-provisioned
images.
* **UI**
* Forms provide selectable lists for default images and existing VM
disks.
* **Migration**
* Migration to rename existing image DataVolumes to the new
default-images naming and bumped migration version.
* **Documentation**
* VM disk docs and README updated to reflect image sourcing and cloning.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
core/v1 Events populate .lastTimestamp and .firstTimestamp but leave
.eventTime null; events.k8s.io/v1 Events do the opposite. The previous
column bound to .eventTime alone and rendered 'Invalid Date' for every
Helm-generated event.
Extend createTimestampColumn with an optional second jsonPath that is
encoded as a nested reqsJsonPath fallback in the template, and use
.eventTime → .firstTimestamp for Event Time and .lastTimestamp →
.eventTime for Last Seen so both APIs render correctly.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Add a "Screenshots" section to the pull request template.
PRs that include UI changes are now required to attach screenshots
or screen recordings demonstrating the visual impact of the changes.
### Release note
```release-note
docs(ci): add mandatory screenshots requirement for UI-related pull requests
```
Add a Screenshots section to the pull request template that requires
contributors to attach screenshots or screen recordings when their
changes affect the UI.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
## Maintainer Nomination: Mattia Eleuteri
Per the process in
[CONTRIBUTOR_LADDER.md](./CONTRIBUTOR_LADDER.md#maintainer), this PR
nominates **Mattia Eleuteri** (@mattia-eleuteri, Hidora) as a
Maintainer.
### Summary
Mattia has demonstrated sustained, high-quality contributions across
CSI/storage, networking, monitoring, and security. Key areas include:
- CSI fixes for migration, RWX NFS mounts, and multi-node volumes
- CiliumNetworkPolicy and VPC peering for multi-tenant environments
- Monitoring improvements (vmagent, infrastructure dashboards)
- Security scanning and hardening
- New platform packages (external-dns)
Full discussion and vote: #2343
### Process checklist
- [x] @mattia-eleuteri please comment confirming you agree to all
[Maintainer responsibilities](./CONTRIBUTOR_LADDER.md#maintainer)
- [x] Majority of current Maintainers (5 of 8) approve this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added Mattia Eleuteri (Hidora) to the project maintainers list,
assigned to CSI, Storage, Networking & Security.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Maintainer Nomination: Matthieu Robin
Per the process in
[CONTRIBUTOR_LADDER.md](./CONTRIBUTOR_LADDER.md#maintainer), this PR
nominates **Matthieu Robin** (@matthieu-robin, Hidora) as a Maintainer.
### Summary
Matthieu has contributed consistently across managed applications,
platform quality, and community engagement. Key areas include:
- Full managed OpenSearch service (operator, packaging, validation)
- Workload monitoring with instance profile labels
- Operational hardening (etcd-defrag resource limits)
- Kubernetes benchmarking and platform validation
- CozySummit speaker and Program Committee member
Full discussion and vote: #2344
### Process checklist
- [x] @matthieu-robin please comment confirming you agree to all
[Maintainer responsibilities](./CONTRIBUTOR_LADDER.md#maintainer)
- [x] Majority of current Maintainers (5 of 8) approve this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the project maintainers documentation to reflect the current
team composition and responsibility assignments. The update documents
team members responsible for overseeing managed applications, platform
quality initiatives, and benchmarking efforts to ensure comprehensive
coverage of critical project areas.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The package imports ~320Gi of golden-image PVCs (ubuntu-noble, fedora,
debian, centos, etc.) as soon as it's installed. On small test and dev
clusters that's enough to consume the entire replicated storage pool,
after which no tenant PVCs — including the ones E2E itself provisions —
can be bound. It's also unreasonable to force that cost on every iaas
user: many deployments don't need prebuilt images at all, and the ones
that do often want to curate their own subset.
Switch the bundle entry from 'package.default' to 'package.optional.default',
matching the treatment already applied to gpu-operator directly below it.
Users who want the golden images can opt in via:
bundles:
enabledPackages:
- cozystack.vm-default-images
The package, its Source, and migration 38 all stay in place — nothing
else changes for users who explicitly enable it. Users who previously
relied on the bundle auto-installing it will need to add the package
to enabledPackages on upgrade; this is called out in the release note.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Tenant applications live in a child namespace computed from the Tenant
name, not in the HelmRelease's namespace. Scope the WorkloadMonitor
watch cluster-wide when kindName is Tenant and reverse-map WM events
back to the owning HelmRelease via computeTenantNamespace. The
condition-enrichment path now looks up monitors in the computed child
namespace too, so WorkloadsReady is populated for Tenants.
Buffer WorkloadMonitor events that arrive before the initial-events-end
bookmark and replay them once the bookmark is emitted, to preserve the
watch-list contract while not dropping workload-state transitions that
happen during the snapshot window.
Pass the fresh WorkloadMonitor object from the watch event into the
conversion path so WorkloadsReady reflects the state that triggered
the event, even when the cache client is lagging behind the watch
client.
Derive WorkloadsReady.LastTransitionTime from a stable source
(max of HelmRelease creation and condition timestamps, plus monitor
timestamps) instead of metav1.Now(), so repeated conversions of the
same underlying state produce identical timestamps.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Inside the RetryOnConflict block, derive the operational status from
fresh.Spec.MinReplicas instead of the stale monitor.Spec.MinReplicas
so that concurrent spec updates observed by the retry are respected.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add two features to improve application observability in the dashboard:
1. WorkloadsReady condition on Application status
- Query WorkloadMonitor resources to determine if all pods are running
- Expose WorkloadsReady as a separate condition alongside Ready
- Ready continues to reflect HelmRelease state only (no override) to
preserve backward compatibility with tooling and avoid false-negatives
during normal startup windows when pods are still coming up
- Handle three states: operational, not operational, unknown (pending)
- Fail-open on WorkloadMonitor query errors
- Integrate with Application Watch to emit MODIFIED events on
WorkloadMonitor changes (with initial-events-end safety)
- Register cozystack.io/v1alpha1 types in API server scheme
2. Events tab in dashboard
- Show Kubernetes Events scoped to application namespace
- Use status.namespace for Tenant applications
- Include both lastTimestamp and eventTime columns for K8s compat
3. Bug fix: WorkloadMonitor Operational status persistence
- Operational was written to stale 'monitor' instead of 'fresh'
inside RetryOnConflict, so it was never persisted to the cluster
Closes#2359Closes#2360
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Two independent cleanups in agent-facing documentation:
1. **Refocus `docs/agents/contributing.md` on project-side
conventions.** The file previously mixed project rules (commit format,
PR structure) with personal workflow preferences (e.g. "do not commit
automatically, show the diff first", subjective guidance on how to
evaluate AI-bot review comments). Personal preferences belong in
individual contributors' own agent config (`~/CLAUDE.md` or equivalent),
not in a shared project doc that every agent reads. This PR keeps only
the project-side artifacts: what a commit/PR must look like when it
reaches the repository.
2. **Adopt Conventional Commits.** Recent commit history mixes
`[component]` prefix and `type(scope):` styles. This PR picks
Conventional Commits and aligns all agent-facing docs and the PR
template on it, so contributors and bots get one consistent answer.
Also documents the `Assisted-By:` trailer convention for AI-authored
commits.
Changes:
- `.github/PULL_REQUEST_TEMPLATE.md`: update guidance and release-note
example
- `docs/agents/contributing.md`: drop personal workflow guidance, link
to the PR template instead of duplicating it, switch examples to
Conventional Commits, document `Assisted-By:` trailer
- `docs/agents/overview.md`, `AGENTS.md`, `.gemini/styleguide.md`:
update commit format references
### Release note
```release-note
docs: adopt Conventional Commits (`type(scope): description`) for commit and PR titles
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Switched contribution & PR guidance to Conventional Commits
(`type(scope): description`) and added explicit allowed types, scope
examples, and breaking-change notation (`!` or `BREAKING CHANGE:`).
* Updated PR template, release-note expectations, validation messaging,
and contributor checklist to match the new convention.
* Added AI Agent Attribution trailers for assisted commits and
streamlined branch/PR workflow guidance (rebase/cleanup and PR creation
examples).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Stacked on top of #2382. Publishes the linstor-gui UI on
`https://linstor-gui.<root-host>` behind the cluster Keycloak realm
using the same oauth2-proxy gatekeeper pattern that the dashboard uses,
so the LINSTOR REST API is never exposed unauthenticated through
Ingress.
The Ingress is rendered only when **both** gates are satisfied:
- `_cluster.oidc-enabled == "true"`
- `linstor-gui` is listed in `_cluster.expose-services` (i.e. in
`publishing.exposedServices` in the core cozystack values)
If OIDC is disabled cluster-wide we intentionally do **not** ship an
unauthenticated ``token-proxy`` fallback — unlike dashboard, there is no
reason to front a raw storage-management API with k8s bearer tokens.
Operators can still reach the UI via \`kubectl port-forward\` to the
ClusterIP service.
## What's new
- \`templates/gatekeeper.yaml\` — oauth2-proxy Deployment (OIDC mode),
upstream \`linstor-gui.<ns>.svc:80\`
- \`templates/gatekeeper-svc.yaml\` — ClusterIP :8000 in front of it
- \`templates/gatekeeper-sa.yaml\` — dedicated SA, no auto-token
- \`templates/keycloakclient.yaml\` — persistent \`linstor-gui-client\`
+ \`linstor-gui-auth-config\` Secrets, \`KeycloakClient\` CRD that
auto-provisions the OIDC client with \`redirectUris:
[/oauth2/callback/*]\`
- \`templates/ingress.yaml\` — Ingress to the gatekeeper Service with
cert-manager ClusterIssuer, gated on expose-services + oidc-enabled
- \`tests/ingress_auth_test.yaml\` — unit tests for each conditional
branch, KeycloakClient rendering, oauth2-proxy args
README updated with Option 1 (Keycloak-protected Ingress) and Option 2
(port-forward).
### Release note
\`\`\`release-note
Added an opt-in Keycloak-protected Ingress for the linstor-gui package.
Add \`linstor-gui\` to \`publishing.exposedServices\` to publish
\`https://linstor-gui.<root-host>\` behind the cluster OIDC realm.
\`\`\`
## Test plan
- [ ] Deploy to dev10 with \`linstor-gui\` added to
\`publishing.exposedServices\`
- [ ] Verify cert-manager issues the TLS certificate
- [ ] Open \`https://linstor-gui.<root-host>\` in a browser, confirm
Keycloak login challenge
- [ ] After login, confirm LINSTOR node list loads via the nginx → mTLS
proxy
- [ ] Verify helm-unittest cases pass in CI
- [ ] Sanity-check that when \`oidc-enabled=false\` or the service is
not in \`expose-services\`, the Ingress + gatekeeper resources are
skipped
## What this PR does
Adds a new Cozystack system package `linstor-gui` that ships [LINBIT's
LINSTOR web UI](https://github.com/LINBIT/linstor-gui) (GPL-3.0) so
operators can manage nodes, resources, volumes and snapshots from a
browser instead of the `linstor` CLI.
Changes:
- `packages/system/linstor-gui/` — umbrella chart with:
- `images/linstor-gui/Dockerfile` builds the image from the upstream
`pkg.linbit.com` tarball (v2.3.0) on top of
`nginxinc/nginx-unprivileged:1.29-alpine`, mirroring the build pattern
already used for `piraeus-server` and `linstor-csi`.
- `templates/configmap-nginx.yaml` — chart-supplied `nginx.conf` that
proxies `/v1` and `/metrics` to
`linstor-controller.cozy-linstor.svc:3371` over **mTLS**, using the
existing `linstor-client-tls` secret created by the `linstor` package.
- `templates/deployment.yaml` — read-only rootfs, non-root (UID 101),
`runAsNonRoot`, `RuntimeDefault` seccomp, no SA token mounted,
`reloader.stakater.com/auto` on the Deployment.
- `templates/service.yaml` — **ClusterIP only**. No Ingress is shipped
because the LINSTOR controller API is a privileged cluster-wide storage
surface and auth depends on the deployment's OIDC setup; operators wire
up ingress + auth explicitly.
- `tests/deployment_test.yaml` — `helm-unittest` covering Service shape,
TLS secret mount, securityContext, and nginx proxy+mTLS config.
- `packages/core/platform/sources/linstor-gui.yaml` — new
`PackageSource` depending on `cozystack.linstor`.
- `packages/core/platform/templates/bundles/system.yaml` — registers
`linstor-gui` as an **optional** system package (controlled via
`bundles.enabledPackages`), alongside `velero`, `telepresence`, etc.
### Why standalone (not a sidecar in the controller pod)
LINBIT's upstream Dockerfile is designed for this standalone
reverse-proxy topology, and keeping the UI as its own Deployment means
it scales, upgrades, and restarts independently of `linstor-controller`.
The price is the small amount of mTLS config in `nginx.conf`, all of
which lives in the chart's ConfigMap.
### Release note
```release-note
[linstor-gui] Add an opt-in system package that deploys LINBIT's linstor-gui web UI alongside the LINSTOR controller. Enable via `bundles.enabledPackages: [cozystack.linstor-gui]`, then `kubectl -n cozy-linstor port-forward svc/linstor-gui 3373:80` to access.
```
## Test plan
- [ ] `make unit-tests` (requires `helm-unittest` plugin) passes on the
new `tests/deployment_test.yaml`
- [ ] `make image` in `packages/system/linstor-gui/` builds and pushes
the image
- [ ] Install the package on a dev cluster with LINSTOR already
deployed; confirm the pod comes up
- [ ] `kubectl -n cozy-linstor port-forward svc/linstor-gui 3373:80` →
browser loads the UI at `localhost:3373`
- [ ] UI lists LINSTOR nodes, storage pools and resources correctly
(proves mTLS proxy to `/v1` works)
- [ ] Disable via `bundles.enabledPackages` removal; verify the
`Package` resource is cleaned up
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* LINSTOR GUI added as an optional system package deployable to the
cozy-linstor namespace.
* Web UI serves on port 3373, proxies LINSTOR controller over mTLS, and
exposes a /healthz endpoint.
* Image build and release targets added to produce multi-arch container
images.
* **Documentation**
* New README with deployment guidance, connection examples, and
configurable options (endpoint, client secret, image, replicas).
* **Tests**
* Helm chart tests validating service, deployment, config and mTLS
behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
* Add changelog for v1.2.2 (`docs/changelogs/v1.2.2.md`)
* Add changelog for v1.1.6 (`docs/changelogs/v1.1.6.md`)
Once merged, the `update-releasenotes.yaml` workflow will sync these to
the corresponding GitHub releases.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed Docker image tagging during builds; pinned system PostgreSQL
images.
* Corrected Cilium AppArmor handling and BPF load‑balancing exclusions
for external VM LoadBalancer services.
* Restored monitoring dashboard rendering for default platform variant.
* **New Features**
* Upgraded LINSTOR (piraeus-server) to v1.33.2 with backported
reliability patches.
* **Documentation**
* Expanded website docs: controller naming, Talos/version pairing,
troubleshooting, bundle naming, new --take-ownership flag, networking.*
fields, OpenAPI refresh, badges, and release notes.
* **Chores**
* Switched CI/CD from long‑lived PATs to short‑lived GitHub App tokens
and updated bot commit identity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The package is wired into the iaas bundle where LINSTOR replicated storage is
available. Set storageClass to "replicated" by default and add a note that the
full image set needs ~320Gi (16 images × 20Gi).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Running make generate at repo root regenerates the zz_generated.deepcopy.go
files. Key fix: Source.DeepCopyInto in the vmdisk package now handles
the new Disk *SourceDisk pointer field, which was previously missing and
would cause pointer aliasing on deep-copy.
Other generated changes are incidental updates to vminstance (Networks
field added, Subnet renamed to Network) and backups CRD manifests picked
up by controller-gen from the current HEAD.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Accessing imgName["properties"].(map[string]any) and
diskName["properties"].(map[string]any) without an ok-check would panic
if the schema does not contain a "properties" key. Wrap both assertions
in ok-guarded form consistent with the outer defensive style.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The x86_64 architecture token was placed inside the filename segment
(CentOS-Stream-GenericCloud-x86_64-N-latest.x86_64.qcow2) which does
not match the actual CentOS mirror layout. Correct both CentOS Stream 9
and 10 URLs to the canonical form without the extra architecture infix.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The \$dv variable assigned by the lookup call on the image PVC was never
read. Remove the dead line to keep the template tidy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The PackageSource for vm-default-images existed but was never included
in any bundle, so it would never be installed. Add it to iaas.yaml
immediately after kubevirt-cdi (which it depends on) so it is deployed
together with the rest of the KubeVirt IaaS stack.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The vm-disk package previously referenced golden-image DataVolumes with
the prefix "vm-image-<name>" in the cozy-public namespace. The new
vm-default-images package creates them as "vm-default-images-<name>",
so any existing vm-disk that sources an image by name would fail to
find its PVC after upgrade.
Migration 38 renames every DataVolume in cozy-public whose name starts
with "vm-image-" to "vm-default-images-" (dropping the old object and
re-creating with the new name) so live vm-disk resources keep resolving
their source PVC transparently.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The previous commit blocked /v1/security/* on the LINSTOR controller,
but that turned out to be wishful thinking: that path doesn't exist in
the LINSTOR REST API at all (the controller responds 404 for everything
under /v1/security), and the linstor-gui SPA never calls it.
The actual storage for the GUI's authentication panel is two LINSTOR
KeyValueStore instances managed by the SPA itself:
- __gui__settings — holds `authenticationEnabled` (the on/off toggle)
- __gui__users — holds the encrypted admin credential
The SPA reads them on every page load via
`GET /v1/key-value-store/__gui__settings`, so we cannot blanket-block
the path or the UI fails to render. Instead, restrict to read-only:
allow GET/HEAD (so the SPA can confirm auth=off and skip its login
screen), reject every mutating method (PUT/POST/PATCH/DELETE) with the
same explanatory 403 JSON. Other GUI key-value entries (e.g.
__gui__mode) are unaffected.
The dev10 cluster had this footgun tripped already
(authenticationEnabled=true, an admin entry in __gui__users); cleared
manually with `linstor key-value-store modify __gui__settings
authenticationEnabled false` so the existing release stops gating the
UI behind a login the user no longer has the password for.
Verified end-to-end on dev10 after upgrade:
GET /v1/key-value-store/__gui__settings -> 200
PUT /v1/key-value-store/__gui__settings -> 403 (with JSON body)
POST /v1/key-value-store/__gui__users -> 403
PUT /v1/key-value-store/__gui__mode -> 200 (other KV writes ok)
GET /v1/nodes -> 200
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The upstream linstor-gui SPA exposes a "Users" / sign-in panel that
POSTs to /v1/security/* on the LINSTOR controller. If a cozystack user
turns on HTTP auth via that panel, every subsequent REST call —
including from this very GUI, which talks to the controller via mTLS,
not bearer tokens — starts returning 401, locking the user out of
LINSTOR with no in-product recovery path.
Now that authentication for the cozystack-shipped GUI is enforced one
layer up at the Ingress (oauth2-proxy + Keycloak), the in-app auth is
both redundant and a footgun. Short-circuit /v1/security/* in the
gateway nginx with a 403 + explanatory JSON body so the setting cannot
be enabled regardless of what the SPA renders. Other LINSTOR REST paths
(/v1/nodes, /v1/resource-definitions, /metrics, …) continue to proxy
through unchanged.
Verified on dev10:
POST /v1/security/sign-in -> 403 (with explanation body)
GET /v1/security/users -> 403
GET /v1/nodes -> 200
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Keycloak access+refresh+id tokens push the oauth2-proxy session cookie
to ~8-10 KB (split across multiple Set-Cookie headers), which overflows
ingress-nginx's default proxy_buffer_size and produces "upstream sent too
big header" -> HTTP 502 on /oauth2/callback right after a successful
Keycloak login.
Mirror the dashboard ingress: proxy-buffer-size: 100m and
proxy-buffers-number: "4". Verified on dev10 — unauthenticated requests
now return 302 to Keycloak instead of 502.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Narrowing to (instance=linstor-gui, name=gatekeeper) also blocked the
transient cm-acme-http-solver pods that cert-manager spins up in the
same namespace during http-01 challenges — the solver pods carry
different labels, so the Certificate never became Ready and TLS
handshakes failed.
Broaden the rule to the whole cozy-linstor namespace to mirror
allow-to-dashboard / allow-to-keycloak. The ingress controller only
initiates upstream HTTP calls driven by Ingress resources, so allowing
namespace-wide egress is not a practical escalation.
Verified on dev10 after patching the live CNP: Certificate
linstor-gui-ingress-tls becomes Ready (Let's Encrypt prod), and the
Ingress returns HTTP 302 to the Keycloak authorize endpoint with the
correct client_id and redirect_uri.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The tenant chart declares per-namespace egress CNPs so the tenant-root
ingress-nginx can reach auth-fronted UIs in the system namespaces
(dashboard, keycloak, cdi-upload-proxy). linstor-gui's gatekeeper in
cozy-linstor needs the same allowance — without it, requests to
https://linstor-gui.<root-host> time out at the Ingress with 504
because tenant-root → cozy-linstor egress is denied by default-deny.
Scoped to the gatekeeper pods (instance=linstor-gui, name=gatekeeper)
rather than the whole cozy-linstor namespace — the ingress controller
does not need to reach the LINSTOR controller, satellites, or CSI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Two issues surfaced while deploying to dev10:
1. The shared linstor-gui.labels / linstor-gui.selectorLabels helpers
set app.kubernetes.io/name=linstor-gui, which overrode the
app.kubernetes.io/name=gatekeeper that the gatekeeper templates set
first. The resulting Deployment had selector.matchLabels=gatekeeper
but template.metadata.labels=linstor-gui, failing admission with
"selector does not match template labels". Drop the helper include
from the three gatekeeper templates and inline the remaining common
labels (managed-by, part-of) so the gatekeeper keeps its own
name label.
2. Copying client.session.idle.timeout=86400 and
client.session.max.lifespan=604800 from the dashboard KeycloakClient
caused the keycloak-operator to reject the client ("Client session
idle timeout cannot exceed realm SSO session idle timeout"). Our
dev10 realm has smaller defaults, and dashboard's own client is
also stuck in this failed state. Drop the overrides and let the
realm defaults apply — the UI has no reason to demand longer
sessions than the realm is willing to grant.
Verified on dev10:
* gatekeeper Deployment rolls out cleanly
* KeycloakClient reconciles to status=OK
* https://linstor-gui.<root-host>/ returns HTTP 302 to
keycloak.<root-host>/realms/cozy/protocol/openid-connect/auth
with client_id=linstor-gui and the correct callback URI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Follow-up to the initial linstor-gui package (#2382). Publishes the UI on
https://linstor-gui.<root-host> behind the cluster Keycloak realm using
the same oauth2-proxy gatekeeper pattern that dashboard uses, so the
LINSTOR REST API is never exposed unauthenticated through Ingress.
Rendered only when both gates are satisfied:
* _cluster.oidc-enabled == "true"
* "linstor-gui" is listed in _cluster.expose-services
(i.e. in publishing.exposedServices in the core cozystack values).
If OIDC is disabled cluster-wide we intentionally do not ship an
unauthenticated token-proxy fallback — unlike dashboard, there is no
reason to front a raw storage-management API with k8s bearer tokens.
Operators can still reach the UI via `kubectl port-forward` to the
ClusterIP service.
New resources:
* templates/gatekeeper.yaml — oauth2-proxy Deployment (OIDC mode)
* templates/gatekeeper-svc.yaml — ClusterIP :8000 in front of it
* templates/gatekeeper-sa.yaml — dedicated SA, no autotoken
* templates/keycloakclient.yaml — persistent client + cookie
Secrets, KeycloakClient CRD with
redirectUris for /oauth2/callback
* templates/ingress.yaml — Ingress to the gatekeeper Service,
cert-manager ClusterIssuer, gated
on expose-services + oidc-enabled
* tests/ingress_auth_test.yaml — unit tests for each conditional
branch and the client CRD
README.md updated with both Option 1 (Keycloak-protected Ingress) and
the existing Option 2 (port-forward).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The matchRegex assertion used 5 literal spaces but the template
renders 9 (aligned with proxy_ssl_trusted_certificate). Use \s+
so the test is resilient to alignment changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback:
- Sync scope lists between PR template and contributing guide
- Add 'maintenance' to Other scopes in contributing guide
- Add scope to the docs example for format consistency
- Simplify rebase push example to drop unnecessary refspec mapping
- Add 'text' language tag to trailer code fence (MD040)
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Refocus docs/agents/contributing.md on project-side conventions only
and drop personal agent-behavior guidance.
Switch commit format from [component] prefix to Conventional Commits
(type(scope): description) in contributing.md, overview.md, AGENTS.md,
.gemini/styleguide.md and the PR template.
Reference .github/PULL_REQUEST_TEMPLATE.md instead of duplicating the
PR body template in contributing.md.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Fix /healthz Content-Type header: nginx silently drops `add_header`
after `return` because the response is already finalized. Switch to
`default_type text/plain;` placed BEFORE the `return 200` so the
probe response is actually labeled as text/plain.
- Drop the redundant `checksum/nginx-config` pod-template annotation:
the deployment already carries `reloader.stakater.com/auto: "true"`,
so Stakater Reloader handles ConfigMap rollouts. Matches the sister
`linstor` package convention.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Adds configuration for the two AI reviewers active on this repository —
Gemini Code Assist and CodeRabbit.
Previously neither had a repo-level config, so both ran with defaults:
Gemini had no knowledge of the project's vendoring model and would flag
issues in vendored upstream charts and generated code; CodeRabbit only
ran on initial PR open, with no review on subsequent pushes.
Changes:
- `.gemini/config.yaml` — ignore patterns for vendored charts,
`vendor/`, generated code, build artifacts, dashboard JSONs, and
upstream patch files; severity threshold set to `LOW` for strict review
of hand-written code; comment cap to prevent floods on chart-update PRs.
- `.gemini/styleguide.md` — natural-language guide covering project
architecture, vendoring rules (where to send bug fixes instead of
editing `charts/`), commit and PR format requirements, sensitive
components, Go code standards, and explicit anti-patterns the reviewer
should not flag.
- `.coderabbit.yaml` — enables incremental reviews so CodeRabbit
re-reviews each push, and skips drafts.
### Release note
```release-note
[ci] Add Gemini Code Assist and CodeRabbit configuration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added code review automation configuration settings
* Added repository style guide and development conventions documentation
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Makes `cozystack-scheduler` LINSTOR storage-aware by configuring it to
call the existing `linstor-scheduler-extender` as a scheduler extender.
This fixes#2328 (phase 1): pods with both a `SchedulingClass` and
LINSTOR PVCs now get storage-locality-aware placement instead of
bypassing LINSTOR's filter/prioritize logic.
Changes:
- **linstor-scheduler package**: expose the extender sidecar (port 8099)
via a new ClusterIP Service; patch the vendored deployment to add a
`app.kubernetes.io/component: scheduler` label for clean Service
targeting
- **cozystack-scheduler package**: bump to v0.3.0 which adds
configurable `extenders` support; set the LINSTOR extender URL in
wrapper values
- **tests**: helm-unittest for extender rendering in the ConfigMap
### Release note
```release-note
[scheduler] cozystack-scheduler now calls the LINSTOR scheduler extender for storage-aware pod placement. Pods assigned to a SchedulingClass that also use LINSTOR-backed PVCs will prefer nodes with local volume replicas.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added scheduler extender support: exposes an extender Service and adds
pod metadata to enable extender routing.
* **Chores**
* Bumped scheduler chart and image versions to 0.3.0.
* Added default extender configuration and a sample remote extender
entry.
* Updated build scripts to apply an additional patch and added a test
target.
* **Tests**
* Added Helm tests validating rendered ConfigMap with and without
extender configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Replace `3297617+cozystack-ci[bot]@users.noreply.github.com` with
`cozystack@users.noreply.github.com` across all CI workflows
- DCO probot rejects the `[bot]` brackets in email as invalid, causing
release PRs to fail DCO checks
## Test plan
- [ ] Verify DCO passes on release PRs after backport to release
branches
The cozystack-ci[bot] noreply email with brackets is rejected by DCO
probot as an invalid email address, causing release PRs to fail DCO
checks.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
The aggregated API previously accepted tenant names containing hyphens
(e.g. `foo-bar`) because `ValidateApplicationName` delegated entirely to
`IsDNS1035Label`, which permits them. The tenant Helm chart's
`tenant.name` helper then rejected the release at template time — so
users saw a successful `kubectl apply` followed by a confusing
Flux/HelmRelease reconciliation error.
This PR extends `ValidateApplicationName` with a `kindName` parameter
and enforces an alphanumeric-only rule (`^[a-z][a-z0-9]*$`) for the
Tenant kind. A `TenantKind` constant centralizes the kind string, and
`REST.validateNameFormat` wraps the check symmetrically with the
existing `validateNameLength`. The tenant chart README is updated to
match the already-correct website documentation.
Test coverage includes:
- Unit tests for the validation package (format + error message
contract)
- REST wrapper and Update→Create fall-through path tests with a fake
client
- E2E BATS regression test with explicit exit-code checks and cleanup
Fixes#2375
### Release note
```release-note
[api] Tenant names containing dashes are now rejected at API level during creation, with a specific error message, instead of being silently accepted and failing later during Helm reconciliation.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Tenant names now require lowercase letters and digits only, must start
with a letter; dashes are rejected with a tenant-specific error.
* **Documentation**
* Tenant naming guidelines updated to reflect the stricter
alphanumeric-only requirement.
* **Tests**
* New unit and end-to-end tests added to verify tenant naming
enforcement and tenant-specific error messaging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Three corrections from PR review:
- Package group list was incomplete. Added library/ (reusable helper
charts) and tests/ (tests for library charts, which are not directly
testable otherwise).
- CRD generation was described incorrectly. Static CRDs are generated
by hack/update-codegen.sh from types under api/v1alpha1/, api/backups/,
and api/dashboard/. Types under api/apps/v1alpha1/ are not static
CRDs — they are registered at runtime from ApplicationDefinition.
- Removed the tgz anti-pattern. No *.tgz files exist outside _out/,
which is gitignored. The guidance was fabricated.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Expose the existing linstor-scheduler-extender sidecar as a ClusterIP
Service and configure cozystack-scheduler to call it during the
scheduling cycle. Pods with both a SchedulingClass and LINSTOR PVCs
now get storage-locality-aware placement.
- Add extender Service (port 8099) for linstor-scheduler
- Patch vendored deployment to label pods for Service selector
- Bump cozystack-scheduler to v0.3.0 (configurable extenders)
- Add "linstor" PackageSource variant with extender values
- Default variant ships without extender for non-LINSTOR clusters
- Select linstor variant in system bundle
- Add helm-unittest tests for both packages
Ref: #2328
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## Summary
- Add `--match 'v*'` to all `git describe` calls in
`hack/common-envs.mk`
- The `api/apps/v1alpha1/*` subtags share the same commit as release
tags, causing `git describe --exact-match` to pick
`api/apps/v1alpha1/vX.Y.Z` instead of `vX.Y.Z`, producing invalid Docker
image tags like `ghcr.io/.../image:v1.35-api/apps/v1alpha1/v1.1.6`
## Test plan
- [ ] Rerun Versioned Tag pipelines after merge and backport to release
branches
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Refined version and tag derivation in the build system to selectively
recognize Git tags, improving consistency in how release versions are
identified.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The api/apps/v1alpha1/* subtags share the same commit as the release
tags. git describe --exact-match picks the first match alphabetically,
returning api/apps/v1alpha1/vX.Y.Z instead of vX.Y.Z, which produces
invalid Docker image tags.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add a minimal .coderabbit.yaml that ensures CodeRabbit re-reviews each
push to a PR (incremental review on new commits) and skips drafts.
Without this file, the org-level configuration left incremental reviews
disabled, so only the initial PR open triggered a review.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Fix two issues caught in PR review:
- Use quadruple-backtick fence around the release-note example so the
inner triple-backtick block renders correctly and satisfies markdownlint.
- Correct the platform migration path. The previous text pointed at
scripts/migrations/, which does not exist. The actual migration flow
lives in packages/core/platform/templates/migration-hook.yaml and
packages/core/platform/images/migrations/.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## Summary
- Replace `GH_PAT` (cozystack-bot PAT) with `cozystack-ci` GitHub App
token in `pull-requests.yaml`
- This was missed in #2351 and broke the `resolve_assets` / `e2e` jobs
for release PRs (draft releases not visible with invalid token)
## Test plan
- [ ] Re-run the release PR pipeline for `release-1.1.6` and verify
`resolve_assets` job passes
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated continuous integration authentication mechanism to use GitHub
App tokens instead of personal access tokens.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Follow-up to 4b76a93d: the assertion still expected defaultMode 0400
and would fail in CI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The linstor-client-tls secret was mounted with defaultMode 0400 and
owned by root, so the nginx process (UID 101) got EACCES on tls.crt
and crash-looped with:
[emerg] cannot load certificate "/etc/linstor/client/tls.crt":
BIO_new_file() failed ... Permission denied
Set defaultMode to 0444. The secret volume is pod-local and
readOnlyRootFilesystem is on, so making it world-readable inside the
pod is not a broader disclosure.
Verified on dev10: pod Ready, /healthz 200, /v1/nodes proxied through
mTLS to linstor-controller:3371 returns real node data.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The GH_PAT secret tied to cozystack-bot is no longer valid after
migrating release workflows to the cozystack-ci GitHub App in #2351.
This broke the resolve_assets and e2e jobs for release PRs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
- Dockerfile: chown html root to 101:101 (match runAsGroup, not 101:0)
- Makefile: declare .PHONY targets (image, image-linstor-gui, test)
- deployment: drop priorityClassName system-cluster-critical (GUI is
not control-plane critical; chart does not need to reserve the slot)
- configmap-nginx:
- fail the render if linstor.endpoint is not https:// so a misconfig
cannot silently downgrade the mTLS-protected controller path
- hoist proxy_set_header and proxy_ssl_* directives to the server
block instead of duplicating across /v1 and /metrics locations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Ships LINBIT's LINSTOR web UI (GPL-3.0) as an opt-in system package under
packages/system/linstor-gui so operators can manage LINSTOR nodes, resources,
and volumes from a browser instead of the linstor CLI.
The UI is built from the upstream pkg.linbit.com tarball onto an
nginx-unprivileged base image. A chart-supplied nginx.conf proxies /v1 and
/metrics to the LINSTOR controller REST API over mTLS using the existing
linstor-client-tls secret created by the linstor package.
Only a ClusterIP Service is created; no Ingress is shipped, because LINSTOR's
controller API is a privileged cluster-wide storage surface and auth depends
on the deployment's OIDC setup. Operators opt in via bundles.enabledPackages
and wire up ingress + auth themselves (port-forward works out of the box).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Configure the Gemini Code Assist GitHub reviewer with project-specific
ignore patterns and a natural-language style guide.
The config silences noise from vendored upstream Helm charts, generated
Go code, build artifacts, and Grafana dashboard JSONs, while keeping the
severity threshold at LOW for strict review of hand-written code.
The style guide teaches the reviewer about the vendoring model, commit
format requirements, sensitive components like packages/core/platform/,
and lists anti-patterns not to flag (patch files, go.sum, upstream forks).
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace bare "Tenant" string literals with the validation.TenantKind constant in all test table entries, struct fields, and TypeMeta. Prevents silent drift if the canonical kind string is ever renamed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The convertHelmReleaseToApplication fall-through that populates Status.Namespace for tenant resources still compared r.kindName against the bare string "Tenant". The validation package already exposes TenantKind for exactly this reason: if the kind string ever diverges from the ApplicationDefinition source of truth, the constant and the call site drift together rather than silently desynchronizing.
Add TestConvertHelmReleaseToApplication_TenantNamespaceKindGate to pin both sides of the gate: tenant kind populates Status.Namespace, non-tenant kind leaves it empty. Also rewrite the TestUpdate comment to describe the upsert invariant directly instead of referencing an issue number.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Four follow-ups from review round 4:
1. BLOCKER: the Update(forceAllowCreate=true) path delegates to Create() when the object does not yet exist (rest.go:452) — the typical kubectl apply upsert flow. Add TestUpdate_ForceAllowCreate_RejectsTenantDashName using a fake client so a future refactor of that delegation cannot silently bypass the tenant name check that r.validateNameFormat alone cannot catch.
2. BLOCKER: the e2e BATS test used || true inside the command substitution, which swallowed the kubectl exit code. Rework the test to capture exit code and stdout+stderr explicitly, then assert the exit code is non-zero before asserting on the error message. This distinguishes validation-success (kubectl exit 0 — regression) from environmental failures (exit non-zero but wrong message) from the happy path.
3. Extract TenantKind = "Tenant" as a named constant in the validation package with a comment pointing at the upstream ApplicationDefinition source of truth, and switch the kindName check to use it.
4. Add a clarifying comment on TestValidateApplicationName_TenantLengthFallthrough that it pins an architectural layering decision and is not a user-facing requirement, so a future promotion of tenant length into tenant-specific wording is a legitimate change rather than a test regression.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Three follow-ups from review:
1. Add defensive cleanup of the foo-bar tenant before and after the e2e regression test. If a prior run left the object in the cluster (e.g. a transient validation regression), the test no longer inherits that state; and if the test itself trips a regression in the future, the cluster is left clean for subsequent tests.
2. Switch kubectl --validate=false to --validate=ignore. The bool form was deprecated in kubectl 1.25 and only kept as an alias; --validate=ignore is the modern, stable spelling.
3. Drop the brittle Contains("63") assertion in TestValidateApplicationName_TenantLengthFallthrough. It pinned the test to the exact DNS-1035 error text from k8s.io/apimachinery, which could break on unrelated upstream wording changes. The surviving assertions still cover the intent: an error exists and it is not the tenant-specific message.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Follow-up from review round 2:
The BATS regression test now pins a namespace precondition, disables kubectl client-side validation with --validate=false so the request is guaranteed to reach the server-side name check, and documents the intent of each assertion. Previously a client-side schema rejection could have produced a false negative on the tenant-specific error grep, and a missing tenant-root namespace could have produced an unrelated failure that was hard to diagnose.
Also pin the edge case where a tenant name consists of valid characters but exceeds the DNS-1035 63-char label limit. The resulting error is intentionally the generic DNS-1035 message because length is not a tenant-specific constraint — the package-level function is not responsible for the stricter Helm-release-prefix length budget that REST.validateNameLength enforces.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two follow-ups from review:
1. The original tenantNameRegex ^[a-z0-9]+$ let leading-digit names like 123foo slip past the tenant check and receive a generic DNS-1035 error, which defeated the goal of surfacing a tenant-specific message. Require a leading lowercase letter so every tenant-invalid name returns the tenant-contract error.
2. Add a BATS regression test in hack/e2e-install-cozystack.bats that exercises the aggregated API end-to-end. Unit tests construct REST{kindName: "Tenant"} by hand, so a future change to ApplicationDefinition kind registration could break real behavior without breaking the unit tests.
Also pin the error-message contract with a new TestValidateApplicationName_TenantErrorMessage that asserts every tenant-invalid input returns a message containing 'tenant names must' (not the generic DNS-1035 text).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The README stated that dashes are 'allowed but discouraged' in tenant names, but the tenant.name Helm helper explicitly fails on release names that contain more than one dash, and the platform guide on the website already says tenant names must be alphanumeric. Bring the chart README in line with both the website documentation and the enforcement now present on the aggregated API.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
ValidateApplicationName previously delegated to IsDNS1035Label, which permits hyphens. The tenant Helm chart's tenant.name helper rejects them at template time, so users saw a successful kubectl apply followed by a Flux reconciliation error.
Extend ValidateApplicationName with a kindName parameter and enforce alphanumeric-only names for the Tenant kind, matching the documented naming rules. Wire the check into REST.Create via a small validateNameFormat wrapper that mirrors validateNameLength.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds a host preflight diagnostic script (`hack/check-host-runtime.sh`)
that warns operators when a standalone `containerd.service` or
`docker.service` is running on the host alongside the embedded k3s
runtime used by the cozystack `generic` variant (k3s / kubeadm on
Ubuntu).
**Why it matters.** K3s ships its own containerd at
`/run/k3s/containerd/containerd.sock` and
`/var/lib/rancher/k3s/agent/containerd`, while a system-package
containerd or docker uses `/run/containerd/containerd.sock` and
`/var/lib/containerd`. The two runtimes do not fight over sockets, so
both keep running silently. Over time the standalone one accumulates
unpruned images and build cache in `/var/lib/containerd` — enough to
fill the root disk, trigger `DiskPressure`, and put `cozystack-api` into
an eviction loop. This is silent on day zero and surfaces as a
mysterious production incident weeks later. The script exists to warn
the next operator before the failure mode surfaces.
The script is warning-only — it always exits 0 and never blocks the
install. It detects:
- `containerd.service` or `docker.service` active via `systemctl
is-active`
- Standalone runtime sockets at well-known paths, with a fallback that
works on hosts without systemd
- Standalone data directory sizes via `du -sh`
When a warning fires, the HINT names only the detected services and
instructs the operator to disable them with `sudo systemctl disable
--now <service>`. Reclaiming the data directory is called out separately
with an explicit note not to delete it blindly — the data may still be
in use.
**Entry points.**
- `make preflight` runs the script directly (for operators preparing a
generic-variant host)
- `make unit-tests` now runs `bats-unit-tests` alongside
`helm-unit-tests`, auto-discovering every `hack/*.bats` file that is not
an e2e test
**Test coverage.** `hack/check-host-runtime.bats` (11 cases, run via
`hack/cozytest.sh`) covers clean hosts with and without systemd,
single-service detection, both services simultaneously, socket-only
fallback for both runtimes, glob-expansion regression guard, explicit
exit-code-0 assertion, `sudo` prefix assertion, `du` failure robustness,
and the "service + socket = exactly one warning" de-duplication
invariant. Every test is self-contained with `trap 'rm -rf $STUB_DIR'
EXIT` for clean recovery on assertion failure, and has no runtime
dependencies beyond bash and core utilities — no python3, no real
systemd.
Irrelevant on Talos where the container runtime lifecycle is fully
managed by the distribution.
### Release note
```release-note
[hack] Add `check-host-runtime.sh` and `make preflight` target that warn when a standalone containerd or docker runtime is running alongside the embedded k3s runtime on the cozystack generic variant.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a non-blocking preflight check that detects standalone container
runtimes, reports disk-usage estimates, and displays an actionable hint
to disable detected services.
* **Tests**
* Added BATS-based unit tests with comprehensive coverage for the
preflight validations and various host scenarios.
* **Chores**
* Build updated to run the BATS unit tests alongside the existing test
suite.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The aggregated `cozystack-api` validates each individual Tenant name
length against the Helm release limit (46 chars for the `tenant-`
prefix), but it never validates the **computed workload namespace** that
is built by dash-joining the parent namespace with the tenant name. A
user can therefore `kubectl apply` a nested `Tenant` whose own name
passes the per-name check, only to have Kubernetes later reject the
generated `Namespace` for exceeding the 63-character DNS-1123 label
limit. The failure surfaces as an opaque HelmRelease reconcile error,
and the Tenant CR is left stranded in the cluster with a
`status.namespace` that can never exist.
This PR closes that gap at the aggregation layer by rejecting such
requests synchronously in `Create()` with a clear `field.Invalid` error
that names the computed namespace and its length, so the failure is
reported at `kubectl apply` time before any HelmRelease is created.
The change is scoped to the `Tenant` kind only. `Update()` is not
touched because Kubernetes names and namespaces are immutable and the
existing code already intentionally skips name validation there. No Helm
chart, CRD, or codegen changes are required.
The PR is split into two commits that document the bug and its fix via
TDD:
1. **Pin the gap** — adds a test that demonstrates the pre-fix behavior:
the pre-fix Create() path has no function that rejects a realistic
nested-tenant combination whose computed namespace exceeds 63 chars.
Reviewers can check out this commit and observe the gap.
2. **The fix** — adds `validateTenantNamespaceLength`, the `kindName ==
"Tenant"` gate in `Create()`, and `TestValidateTenantNamespaceLength`
covering boundary cases (pass at 63, fail at 64, fail at 68). The
pinning test from commit 1 is removed in the same commit now that the
gap is closed.
### Release note
```release-note
[platform] cozystack-api now rejects Tenant creation at admission time when the ancestor-chain namespace would exceed the 63-character Kubernetes namespace limit, instead of allowing the resource through and failing later at HelmRelease reconcile.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Tenant application creation now validates computed tenant namespace
length against the Kubernetes DNS-1123 63-character limit; creations
that would produce too-long namespace names now fail with a clear
validation error.
* **Tests**
* Added tests covering tenant namespace length validation, including
multiple parent/tenant-name scenarios, boundary conditions, and
verification that error messages include the offending namespace and its
length.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! -->
## What this PR does
Cilium 1.19 init containers `mount-cgroup`, `apply-sysctl-overwrites`
and `clean-cilium-state` call `nsenter` to join the host cgroup/mount
namespace. On any distribution that loads the
`cri-containerd.apparmor.d` AppArmor profile by default for containerd
workloads (Ubuntu 22.04+, Debian with AppArmor), the kernel denies the
operation and the audit log reports it as a blocked `ptrace` operation:
```text
apparmor="DENIED" operation="ptrace" class="ptrace"
profile="cri-containerd.apparmor.d" comm="nsenter"
requested_mask="read" denied_mask="read" peer="unconfined"
```
Symptom on non-Talos clusters: after a cilium rollout the new pod stays
in `Init:CrashLoopBackOff`, and the rest of the platform cascades into
`dependency not ready` until cilium is healthy on every node.
This PR opts the affected containers out of the
`cri-containerd.apparmor.d` profile via deprecated per-container
`container.apparmor.security.beta.kubernetes.io/<name>: unconfined`
annotations. Four containers are annotated: `cilium-agent`,
`clean-cilium-state`, `mount-cgroup`, `apply-sysctl-overwrites`.
`mount-bpf-fs` is intentionally excluded because it renders with its own
`securityContext.privileged: true`, and the kernel does not apply
AppArmor profiles to privileged containers.
### Why pod annotations instead of `podSecurityContext.appArmorProfile`
The upstream Cilium chart already sets
`podSecurityContext.appArmorProfile.type: Unconfined` (the modern k8s >=
1.30 API), but containerd's CRI implementation silently ignores this
field today and keeps applying `cri-containerd.apparmor.d` — see
kubernetes/kubernetes#125069 (still open). Only the deprecated
per-container annotations take effect end-to-end on current k3s /
kubeadm + containerd setups.
### Why a dedicated values file limited to non-Talos variants
On Talos (`isp-full` bundle) `values-talos.yaml` sets
`cgroup.autoMount.enabled: false`, so the upstream chart does not render
the `mount-cgroup` init container. The kube-apiserver rejects a
DaemonSet that carries a per-container AppArmor annotation for a
container that is not in the pod spec:
```
DaemonSet.apps "cilium" is invalid: spec.template.annotations[
container.apparmor.security.beta.kubernetes.io/mount-cgroup]:
Invalid value: "mount-cgroup": container not found
```
To keep the Talos path untouched and only add the annotations where they
are both safe and useful, this PR puts the `cilium.podAnnotations` block
in a new `packages/system/cilium/values-apparmor.yaml` and wires it into
only the non-Talos PackageSource variants (`cilium-generic`,
`kubeovn-cilium-generic`). Those variants always run with
`cgroup.autoMount.enabled: true` (explicit bundle override in
`packages/core/platform/templates/bundles/system.yaml:83-86`), so every
annotated container really exists in the rendered DaemonSet.
On Talos the annotations are omitted entirely — that is safe because the
Talos kernel does not load the AppArmor LSM anyway.
### Vendored chart patch to avoid duplicate keys on unsupported k8s <
1.30
On k8s < 1.30 the upstream cilium chart template emits the same
annotation keys from its own `semverCompare "<1.30.0"` branch in
`cilium-agent/daemonset.yaml`. Letting both sources write to the
annotations mapping would produce a YAML map with duplicate keys.
Cozystack's supported k8s matrix starts at 1.30
(`packages/apps/kubernetes/files/versions.yaml`), so the duplicate
rendering is never observed in practice, but to keep the rendered
manifest clean on any k8s version this PR also adds a `perl -i -0pe`
multi-line delete to `packages/system/cilium/Makefile` that strips the
entire upstream `{{- if not .Values.securityContext.privileged }} / {{-
if semverCompare "<1.30.0" }}` block from the vendored template. A
fail-fast `grep` guard after the `perl` invocation turns a
silently-failed patch (e.g. if upstream reformats the block) into a loud
error. The same patch is applied to the currently vendored copy so the
repository state matches what `make update` produces. This follows the
existing sed-patch pattern already used for `values.yaml` (`Used in
iptables`, `SYS_MODULE`).
### Scope
- `packages/system/cilium/values-apparmor.yaml` — new file containing
the four `cilium.podAnnotations`
- `packages/system/cilium/values.yaml` — unchanged apart from reverting
the previously-added annotations block (the annotations now live in
`values-apparmor.yaml`)
- `packages/system/cilium/Makefile` — extends the `update:` target with
a `perl` multi-line delete (plus a `grep` fail-fast guard) that drops
the entire upstream AppArmor annotation block on every vendoring refresh
-
`packages/system/cilium/charts/cilium/templates/cilium-agent/daemonset.yaml`
— the vendored file with that block already removed to match the
Makefile patch
- `packages/core/platform/sources/networking.yaml` — adds
`values-apparmor.yaml` to the `cilium-generic` and
`kubeovn-cilium-generic` variants only
### Compatibility across supported platforms
- **Talos Linux** (`isp-full` bundle, `kubeovn-cilium` variant) —
AppArmor LSM is not loaded, `cgroup.autoMount.enabled: false`,
`values-apparmor.yaml` is not included. Zero annotations in the rendered
DaemonSet. No behavioural change. Verified with `helm template` +
`kubectl apply --dry-run=server` against a live k3s v1.35.2 cluster: no
errors, no AppArmor lines.
- **Ubuntu 22.04 / 24.04, Debian with AppArmor** (`isp-full-generic`
bundle, `kubeovn-cilium-generic` / `cilium-generic` variants) — the four
annotations are present in the rendered DaemonSet, kubelet starts the
listed containers with the unconfined profile, `nsenter` succeeds.
Verified with `helm template` + `kubectl apply --dry-run=server` against
a live k3s v1.35.2 cluster: no errors, exactly four
`container.apparmor.security.beta.kubernetes.io/*: unconfined` lines.
Verified on a fresh k3s + Ubuntu 24.04 deployment end-to-end.
- **RHEL / Rocky / AlmaLinux / Fedora** — use SELinux, not AppArmor.
When deployed via `*-generic` variants these platforms pick up the
annotations; kubelet silently ignores AppArmor annotations when the LSM
is not loaded, SELinux policy is unaffected.
- **k8s < 1.30 and k8s >= 1.30** — both now render the same four
annotations (on non-Talos), sourced exclusively from
`cilium.podAnnotations`. Verified via `helm template --kube-version
1.29.0` and `--kube-version 1.35.0` → exactly 4 annotations with no
duplicates.
### Release note
```release-note
[cilium] Opt four cilium-agent containers (cilium-agent, clean-cilium-state, mount-cgroup, apply-sysctl-overwrites) out of the cri-containerd.apparmor.d AppArmor profile via per-container annotations, applied only on non-Talos cilium variants (cilium-generic, kubeovn-cilium-generic). This unblocks cilium rollouts on Ubuntu 22.04+, Debian and other distributions that load this profile for containerd by default, where the kernel previously denied nsenter's namespace entry (reported as a blocked ptrace operation) and the cilium agent landed in Init:CrashLoopBackOff. Talos variants are untouched.
```
## Summary
Replaces the `cozystack-bot` user account + personal access token
(`GH_PAT`) with the `cozystack-ci` GitHub App across all CI/CD release
workflows.
### Why
- **Security**: GitHub App tokens are short-lived (1h) and scoped, vs. a
long-lived PAT tied to a user account
- **Auditability**: App actions are clearly attributed to
`cozystack-ci[bot]`
- **No 2FA issue**: The bot account lacked 2FA; GitHub Apps don't
require it
- **Best practice**: GitHub recommends Apps over PATs for automation
### Changes
| Workflow | What changed |
|---|---|
| `tags.yaml` | App token in all 3 jobs: prepare-release,
generate-changelog, update-website-docs |
| `auto-release.yaml` | App token for daily auto-patch tagging |
| `pull-requests-release.yaml` | App token for release PR finalization |
All `cozystack-bot` git identity replaced with `cozystack-ci[bot]`.
### New secrets (already configured at org level)
- `COZYSTACK_CI_APP_ID` — GitHub App ID
- `COZYSTACK_CI_PRIVATE_KEY` — GitHub App private key
### After merge
- [ ] Delete `GH_PAT` repo-level secret
- [ ] Remove `cozystack-bot` from the cozystack org
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated CI/CD automation authentication mechanisms across release and
tagging workflows for improved security practices.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Reject Tenant creation when the computed workload namespace (parent
namespace + "-" + tenant name) would exceed the 63-character DNS-1123
label limit. Previously only the per-name length was checked against
the Helm release limit, so a deeply-nested tenant could pass Create()
and then fail later as an opaque HelmRelease reconcile error.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The disk_usage helper shells out to 'du -sh $path' for reporting in
the warning message. On the exact directories this script is meant
to warn about (a /var/lib/containerd accumulating millions of files
from months of unpruned builds), traversing the tree with du can
take minutes and stall the preflight indefinitely.
Wrap the call in 'timeout 5s' so the helper returns quickly even on
a pathologically slow filesystem. If the timeout binary is absent
(e.g. minimal busybox userland), the pipeline still exits 0 via the
existing '|| true' guard and usage stays empty — the warning still
prints, just without the size detail — so the change is backward
compatible.
Addresses an inline review comment from gemini-code-assist on the
open PR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
E2E revealed that the kube-apiserver rejects a DaemonSet carrying a
per-container AppArmor annotation for a container that is not present
in the pod spec:
DaemonSet.apps 'cilium' is invalid:
spec.template.annotations[container.apparmor.security.beta.kubernetes.io/mount-cgroup]:
Invalid value: 'mount-cgroup': container not found
On Talos (isp-full bundle) values-talos.yaml sets
cgroup.autoMount.enabled=false, so the upstream chart does not render
the mount-cgroup init container, and our unconditional podAnnotations
entry for it becomes an invalid reference.
Move the podAnnotations block from the shared values.yaml into a new
values-apparmor.yaml and include it only in the cilium-generic and
kubeovn-cilium-generic PackageSource variants, where
cgroup.autoMount.enabled is explicitly true. Talos variants
(cilium, cilium-kilo, kubeovn-cilium) get no AppArmor annotations,
which is safe because the Talos kernel does not load the AppArmor
LSM anyway.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Broaden the grep guard in the Makefile update: target to match any
container.apparmor.security.beta.kubernetes.io key, not only the
quoted cilium-agent line. The previous pattern would silently pass
if upstream switched to an unquoted annotation value, masking a
failed perl patch; grepping for the prefix catches any residual
hardcoded AppArmor annotation regardless of format.
- Rewrite the values.yaml comment to distinguish the two Talos-safe
reasons: mount-cgroup is simply not rendered when
cgroup.autoMount.enabled is false (Talos default), while the other
annotations are harmless because kubelet ignores AppArmor metadata
on nodes without the LSM loaded.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace the misleading 'nsenter --ptrace' shorthand with an accurate
description: nsenter joins the host cgroup/mount namespace, and the
cri-containerd.apparmor.d profile reports the denial as a blocked
'ptrace' operation (operation name, not an nsenter CLI flag).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The perl multi-line delete in the update: target silently becomes a
no-op if the upstream template is ever reformatted. Verify the patch
applied by grepping for one of the stripped annotation keys after the
perl runs; fail the update: target loudly if it still exists, so a
future upstream reformat is caught instead of silently reintroducing
the duplicate-key rendering on k8s < 1.30.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The previous sed patch removed only the four annotation lines and
left an empty nested '{{- if ... }}{{- end }}{{- end }}{{- end }}'
skeleton behind. That skeleton was dead code, but it also meant that
the next make update would re-fetch the upstream chart with the
annotations back inside the same block, and the sed would strip them
again — leaving the skeleton behind forever.
Replace the per-line sed patch with a single perl -i -0pe multi-line
delete that removes the entire <1.30 AppArmor block in one step, and
apply the same patch to the currently vendored copy for consistency.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The upstream chart emits per-container AppArmor unconfined annotations
inside a k8s<1.30 branch of cilium-agent/daemonset.yaml. Cozystack now
owns these annotations via cilium.podAnnotations in values.yaml so they
also take effect on k8s>=1.30 (where upstream drops them in favour of
the containerd-broken podSecurityContext.appArmorProfile). Having both
sources emit the same keys produced a duplicate mapping on k8s<1.30.
Add a new SED_INPLACE invocation to the package Makefile that removes
the four hardcoded annotation lines from the vendored template, and
apply the patch to the currently vendored copy so make update will
reproduce the same state on future refreshes.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
$(wildcard ...) returns a space-separated list, so a bats file with
a literal space in its name would silently split into multiple
tokens in the subsequent 'for' loop. All current bats files use
hyphen-separated names, so this is not an immediate risk, but the
caveat is worth calling out so a future contributor who adopts a
different naming convention rewrites the recipe instead of tripping
over it.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address three review findings on the bats suite:
- The 'standalone containerd service active' and 'standalone docker
service active' tests previously asserted 'systemctl disable --now
<service>' but did not require the 'sudo ' prefix, so the two-
service test was the only guard against a silent drop of the
prefix. Extend both single-service tests to require the prefix
exactly as the 'both services active' test does.
- The 'both services active' test now captures the exit code
explicitly with 'bash ... || status=$?' and asserts '$status -eq
0'. The script contract ('always exits 0') was previously enforced
implicitly via 'set -e', which produces a generic test failure on
a regression instead of a contract-specific error. The explicit
check locks in the contract and makes regressions readable.
- New test: 'docker socket paths with glob chars do not expand' —
sets COZYSTACK_DOCKER_SOCKET_PATHS to a literal glob against real
directories and asserts no docker warning fires. Locks in the
array-based parsing of the path list.
- Extend the file header with a 'title syntax constraints' section
documenting how cozytest.sh's awk parser treats double quotes and
non-alphanumeric characters in @test titles, so future
contributors do not stumble on the parser's quirks.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
'set -euo pipefail' does not include 'set -f', so the previous
'for sock in $DOCKER_SOCKET_PATHS' loop relied on both word splitting
AND unintended glob expansion. If the variable ever contained a
literal '*' or '?' — for example a test setting
COZYSTACK_DOCKER_SOCKET_PATHS='/run/docker-*.sock' against a tree
that happens to have real files matching the pattern — the loop
would iterate over directory entries instead of the intended socket
paths, producing false-positive docker warnings.
Replace the implicit split+glob loop with 'read -ra' into a bash
array and iterate 'for sock in "${_socks[@]}"'. This keeps the
space-separated input format, disables glob expansion, and removes
the fragile word splitting that shellcheck otherwise has to be
talked out of.
Covered by a new regression test ('docker socket paths with glob
chars do not expand') that sets the env var to a literal '*' pattern
pointing at real directories; without this fix the test fails with
an unexpected docker.service warning.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
DOCKER_SOCKET_PATHS is a space separated list of socket paths (its
default is "/run/docker.sock /var/run/docker.sock") that the loop
iterates by word splitting. Socket paths never contain whitespace on
Linux hosts, so this is the documented and correct idiom — not a
shellcheck oversight. Add an inline comment so a future maintainer (or
a linter that has not seen the comment) does not break the loop by
adding quotes around the expansion.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Without a guard, running make bats-unit-tests in a tree where the
hack/*.bats wildcard matches nothing (for example a rename, a shallow
clone, or a future reorganization that moves the tests elsewhere)
would silently succeed with zero tests run. Add an explicit guard that
fails the target with a clear error message when BATS_UNIT_FILES
expands to an empty list, so the developer cannot accidentally believe
tests passed when no tests executed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two review findings:
1. The socket fallback tests previously branched on 'command -v
python3' and fell through to 'return 0' when it was missing.
cozytest.sh has no SKIP concept — 'return 0' is indistinguishable
from a real pass, so on a runner without python3 the socket
fallback paths were silently unverified. Since the script uses
'[ -e "$sock" ]' rather than '[ -S ... ]', a regular file is
sufficient to exercise the detection path. Replace the python3
unix-socket creation with 'touch "$SOCK"' so the tests run
unconditionally on every runner.
2. Extend the 'both services active' HINT assertion to require the
'sudo ' prefix on the 'systemctl disable --now' line. Without sudo
the operator would be instructed to run a privileged command as a
non-root user and hit a silent failure; the prefix is as important
as the verb itself and must be locked in by a test.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two small Makefile tweaks:
1. bats-unit-tests now iterates over every hack/*.bats file that is
not an e2e-*.bats file (wildcard + filter-out), so dropping a new
unit bats file into hack/ is enough to get it picked up by CI on
the next run. The previous hard coded single file path would have
required a Makefile edit for every new unit suite.
2. Add a new 'preflight' target that runs hack/check-host-runtime.sh.
This gives the script a discoverable entry point — an operator
preparing a generic (k3s/kubeadm) host for cozystack can now run
'make preflight' from the repository root to get the same warning
that is described in the script header, without having to know
the exact path under hack/.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Expand the bats suite to address a second review pass:
- Assert that a containerd only warning produces a HINT that names
containerd.service and NOT docker.service, and mirror the check in
the docker only test. When both fire, assert the HINT lists both
services in a single systemctl disable invocation.
- New test: 'clean host without systemctl exits silently' — exercises
the COZYSTACK_PREFLIGHT_FORCE_NO_SYSTEMCTL=1 path when no standalone
sockets exist. Previously the tests only covered the systemd enabled
clean host, leaving the non systemd clean path unverified.
- New test: 'docker service plus socket still emits exactly one
warning' — mirrors the existing containerd service+socket test and
locks in the gated check in check_docker.
- Replace silent 'return 0' on missing python3 with a visible
'# SKIP: python3 unavailable' message on stderr so CI logs make it
obvious which tests were skipped on a given runner.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When only containerd.service was active, the HINT block still advised
the operator to disable both containerd.service and docker.service,
which is misleading and potentially dangerous on hosts where docker is
legitimately in use. Track per service warnings (CONTAINERD_WARN and
DOCKER_WARN) and build the HINT systemctl disable argument from the
services that actually fired.
Also guard the ANSI color escapes behind an 'is stderr a TTY' check so
log files (CI and reviewer captures) do not accumulate raw escape
sequences.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback on the preflight test suite:
- Add a trap 'rm -rf $STUB_DIR' EXIT in every test immediately after
mktemp -d, so temp dirs are cleaned up even when an assertion fails
and terminates the test early under set -e.
- Assert the HINT block and 'systemctl disable --now' line in the
'both services active' test so a future silent removal or typo in
the HINT output is caught by CI.
- New test: 'docker socket fallback fires when systemctl is
unavailable' — mirrors the existing containerd socket fallback test
and exercises the same code path in check_docker.
- New test: 'containerd service plus socket still emits exactly one
warning' — documents and locks in the intent of the symmetric gated
check in check_containerd. Uses grep -c to assert the warning is not
double printed when both signals fire.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two small issues caught in review:
1. check_containerd probed the socket unconditionally even when
service_active had already set found=1, while check_docker short
circuited the same path behind 'if \[ $found -eq 0 \]'. The
asymmetry was not user visible but violated least surprise for
future maintainers. check_containerd now uses the same gated pattern.
2. The HINT block recommended 'sudo rm -rf /var/lib/docker
/var/lib/containerd' as a casual follow up, which could destroy data
the operator still needs. Replace that line with a warning telling
the operator to inspect and reclaim standalone runtime storage
manually rather than deleting it blindly.
Also drop a no op 'printf' from disk_usage that served no purpose.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add a bats-unit-tests Make target that runs hack/cozytest.sh against
hack/check-host-runtime.bats, and make unit-tests depend on it so the
existing CI step (make unit-tests in .github/workflows/pull-requests.yaml)
exercises the preflight script on every PR. Without this the bats file
exists in the tree but is never executed, leaving future regressions
undetected.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Ubuntu hosts running the cozystack "generic" variant (k3s or kubeadm)
sometimes end up with a standalone containerd.service or docker.service
running alongside the embedded k3s runtime. The two runtimes do not fight
over sockets — k3s uses /run/k3s/containerd/containerd.sock while the
standalone package uses /run/containerd/containerd.sock — so both keep
running silently, and the standalone one accumulates unpruned images and
build cache in /var/lib/containerd. Over time this fills the root disk,
triggers DiskPressure, and sends cozystack-api into an eviction loop.
hack/check-host-runtime.sh warns an operator about this before install
without blocking it. The script probes systemctl and well-known socket
paths, reports disk usage of the standalone data directory when present,
prints a hint on how to disable the shadow runtime, and always exits 0.
Covered by hack/check-host-runtime.bats (six test cases).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add hack/check-host-runtime.bats with six self-contained test cases that
exercise the check-host-runtime.sh preflight script: clean host exits
silently, standalone containerd.service warns, standalone docker.service
warns, both services warn, du failures do not suppress warnings, and the
socket-only fallback fires when systemctl is unavailable.
Tests inject a stub systemctl and du binary via PATH and redirect the
script's probe paths through COZYSTACK_PREFLIGHT_* environment variables,
so they run without root and on any host (including non-systemd macOS).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
On Kubernetes versions below 1.30 the upstream cilium chart still emits
the deprecated per-container AppArmor annotations from its own
semverCompare '<1.30.0' branch. Our podAnnotations override then emits
the same keys again, yielding a YAML mapping with duplicate entries
(identical values, last-wins by kubernetes API server JSON semantics).
Cozystack's supported Kubernetes matrix starts at 1.30
(packages/apps/kubernetes/files/versions.yaml), so this rendering
quirk never affects any shipped version. Document the behaviour in a
comment so future readers are not surprised.
No functional change.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
mount-bpf-fs runs as privileged in the vendored chart, and the Linux
kernel does not apply AppArmor profiles to privileged containers. The
per-container unconfined annotation for this container is a no-op and
only widens the pod's admission-policy surface for no reason.
Keep the four meaningful annotations (cilium-agent, clean-cilium-state,
mount-cgroup, apply-sysctl-overwrites) which target unprivileged agent
and init containers actually affected by the cri-containerd.apparmor.d
profile.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Cilium 1.19 init containers mount-cgroup, apply-sysctl-overwrites,
mount-bpf-fs and clean-cilium-state call nsenter --ptrace, which is
denied by the cri-containerd.apparmor.d profile on Ubuntu 22.04+ and
any other distro loading this AppArmor profile. As a result cilium
agent pods land in Init:CrashLoopBackOff and downstream HelmReleases
cascade into "dependency not ready".
The modern k8s >= 1.30 pod-level appArmorProfile: Unconfined field is
accepted into the DaemonSet spec but not honoured by containerd CRI
at runtime (kubernetes/kubernetes#125069), so the deprecated
per-container AppArmor annotations remain the only reliable opt-out.
Add the annotations via podAnnotations in the wrapper values.yaml.
The change is harmless on systems without AppArmor (Talos,
RHEL/Rocky/AlmaLinux with SELinux) because kubelet silently ignores
unknown AppArmor annotations when the LSM is not loaded.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Fix system postgresql images to 17.7-standard-trixie
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Database Helm charts now preserve and reuse existing PostgreSQL image
settings, normalizing tags to the standard-trixie variant when
appropriate.
* Migration configuration bumped to target version 38.
* **Bug Fixes**
* Migration logic updated to correctly locate and adjust CNPG Cluster
resources and to log per-cluster patch/skip decisions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
- Adds `cozy-monitoring` namespace to the infra dashboards rendering
condition in `dashboards.yaml`
- The default variant deploys the monitoring chart to `cozy-monitoring`,
but the condition from #2197 only checked for `tenant-root`, causing all
infrastructure dashboards to be missing
- This is consistent with the existing pattern in `vmagent.yaml` which
already handles both namespaces
## Release note
```release-note
[monitoring] Fix infrastructure dashboards not rendering in default variant where monitoring is deployed to cozy-monitoring namespace
```
Fixes#2349
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Infrastructure & Monitoring**
* Expanded Grafana dashboard availability across additional deployment
environments for infrastructure monitoring.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- update `packages/system/linstor` to `linstor-server` `v1.33.1`
- keep only the selected `piraeus-server` backports needed for this
update
- document the upstream/source commits for each retained patch
- drop the older local patch set that is no longer carried in this
package revision
## Selected backports
- PR #475: backport maintainer implementation from
`3d97f71c95a493588d3d521c63eac4d846935fb3`
- PR #476: backport the TCP port preservation fix from
`79d6375c55d6181b35a7b7f0fe8dbdfb86e126cd` and
`bcc89902f4f61ac1589dd07ebb7f5aae1935370d`
- PR #472: backport the LUKS header sizing fix from
`ccc85fbd2c65f0b97c52403fa80f1efdb886ec4e` plus the required
`optimal_io_size` dependency from
`71b601554f41bcb50cd5bd06989c5b0d3a814acd`
## Verification
- applied all retained patches on top of upstream `v1.33.1` with `git
apply --check`
- verified the combined backport builds with `./gradlew --no-daemon
compileJava`
- ran `make image` in `packages/system/linstor` successfully
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded LINSTOR server to v1.33.2 and refreshed related package
metadata.
* **Bug Fixes / Reliability**
* Safer toggle-disk flows with retry/abort handling and preservation of
DRBD TCP ports.
* Automatic retry/cleanup for stale bitmap attach errors to improve
recovery.
* **Storage / Compatibility**
* Improved LUKS2 header sizing, optimal I/O size detection, and related
patches for more reliable disk formatting and alignment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The default variant deploys monitoring to the cozy-monitoring namespace,
but the infra dashboards condition only checked for tenant-root. This
adds cozy-monitoring to the condition, consistent with the existing
pattern in vmagent.yaml.
Fixescozystack/cozystack#2349
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
add ability for restoring VMI to copy in another namespace
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* RestoreJob CR supports a free-form spec.options payload; restores now
support in-place preserves (IP/MAC, PVC handling), cross-namespace copy
restores, and safe rename semantics.
* **Documentation**
* Added admin and tenant walkthroughs for Velero-backed
VMInstance/VMDisk backup & restore.
* **Examples**
* New stepwise demo scripts, helpers, orchestration (run-all) and
cleanup scripts.
* **Tests**
* Extensive unit tests covering restore option parsing, target
resolution, and restore side effects.
* **Chores**
* RBAC updated to allow HelmRelease create/delete and namespace listing
for restore flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
All existing patches (allow-toggle-disk-retry, fix-duplicate-tcp-ports,
fix-luks-header-size, retry-adjust-after-stale-bitmap) apply cleanly on
the new version without modifications.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Adds the `service.kubernetes.io/service-proxy-name: "cozy-proxy"` label
to VM LoadBalancer services when `external: true`. This tells Cilium to
completely ignore the service — no BPF entries, no DNAT, no wildcard
drops.
Routing continues to work through kube-ovn
(`networking.cozystack.io/wholeIP` annotation) and MetalLB L2
advertisement is unaffected.
**This fixes two issues:**
1. **Inter-tenant connectivity via public LB IPs**: With
`kube-proxy-replacement: true`, Cilium DNATs traffic destined to LB IPs
(on ports declared in the Service) to the backend pod IP *before*
evaluating network policies. The `CiliumClusterwideNetworkPolicy` then
sees a cross-tenant pod-to-pod flow and blocks it. Ports not in the
Service pass as "world" traffic and work fine — so does ICMP. With this
label, Cilium doesn't DNAT anything, all traffic is classified as
"world", and the existing `allow-external-communication` policy allows
it. Pod IP isolation between tenants is preserved.
2. **WholeIP broken on Cilium 1.19+** (#2327): Cilium 1.19 introduced
wildcard service drop entries (`cilium/cilium#40684`) that drop all
traffic to a LB IP on ports not declared in the Service, before it
reaches netfilter/cozy-proxy. With this label, no BPF entries are
created at all, so there's nothing to drop.
**Tested on our lab cluster:**
- Cilium 1.18.6: inter-tenant via LB IP ✅, pod IP blocked ✅, external
SSH ✅, WholeIP all ports ✅
- Cilium 1.19.1: same results ✅, no wildcard drops ✅
- MetalLB L2 advertisement: unaffected ✅
- Other services without the label: unaffected ✅
**Note:** The `externalPorts` / PortList filtering is currently not
enforced at the network level (the `wholeIP` annotation causes kube-ovn
to route all ports regardless). This was the case before this change and
is a separate concern — per-VM CiliumNetworkPolicies could be
re-introduced for that (ref: `4cbf562a`, #1601).
```release-note
[virtual-machine] Fix inter-tenant connectivity via public LB IPs and WholeIP on Cilium 1.19+
```
Increase Kafka CR readiness timeout from 60s to 300s to account for
slow Strimzi startup on QEMU-based CI sandbox (4 JVM pods).
Add wait-for-delete before re-applying to prevent race condition where
kubectl apply hits a still-deleting resource on retry attempts.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add service.kubernetes.io/service-proxy-name label to LoadBalancer
services when external: true. This prevents Cilium from adding the
service to its BPF service map, fixing two issues:
1. Inter-tenant connectivity via public LB IPs: Cilium's kube-proxy
replacement DNATs traffic to LB IPs before policy evaluation,
causing the CiliumClusterwideNetworkPolicy to block legitimate
cross-tenant traffic through public IPs.
2. WholeIP broken on Cilium 1.19+ (#2327): wildcard service drop
entries block all ports not declared in the Service spec before
traffic reaches cozy-proxy's nftables rules.
With this label, Cilium completely ignores the Service. Routing is
handled by kube-ovn (via the wholeIP annotation) and MetalLB
continues to advertise the IP via L2 unaffected.
Tested on Cilium 1.18.6 and 1.19.1: inter-tenant via LB IP works,
pod IP isolation preserved, external access unaffected.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Replace the cozystack-bot personal access token (GH_PAT) with a
GitHub App (cozystack-ci) installation token across all CI/CD
workflows. This improves security by using short-lived, scoped
tokens instead of a long-lived PAT tied to a user account.
Changes:
- tags.yaml: use app token in all 3 jobs (prepare-release,
generate-changelog, update-website-docs)
- auto-release.yaml: use app token for daily patch releases
- pull-requests-release.yaml: use app token for release finalization
The cozystack-ci GitHub App (ID: 3297617) is installed org-wide
with contents:write and pull-requests:write permissions. Secrets
COZYSTACK_CI_APP_ID and COZYSTACK_CI_PRIVATE_KEY are set at org
level.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## What this PR does
This PR adds configurable values for
`packages/system/keycloak-configure`:
- `login.userRegistration` to allow user self-registration
- `login.verifyEmail` to enforce email verification
- `smtp` which defines the SMTP configuration which will be used by
Keycloak
Fixes: aenix-org/cozyportal#365
### Release note
```release-note
["cchore(keycloak-configure): verify email and smtp config for Keycloak"]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added configurable Keycloak login settings with user registration and
email verification controls disabled by default.
* Introduced SMTP configuration templates supporting server connection
details, TLS options, and secure credential management for email
delivery.
* Enabled conditional rendering of login and SMTP configuration sections
based on provided environment values.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
CNPG operator in v1.2.0 updates default PostgreSQL image to v18
and cannot migrate from the previous major version automatically,
causing PostgreSQL clusters to fail after upgrade.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Add OpenSSF Best Practices passing badge to README alongside existing
project badges
- Badge links to https://www.bestpractices.dev/projects/10177
## Test plan
- [ ] Verify badge renders correctly on GitHub README
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added OpenSSF Best Practices badge to the project status section in
the README.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds `SECURITY.md` to the repository.
The document defines:
- supported release lines
- how to report vulnerabilities without posting details publicly
- disclosure expectations
- how security fixes are communicated
It is written to match the current state of the project and avoids
claiming private channels or security tooling that are not clearly
published yet.
### Release note
```release-note
[docs] add SECURITY.md with vulnerability reporting and disclosure guidance
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added comprehensive security policy documentation outlining
vulnerability reporting procedures, responsible disclosure practices,
and response timelines for security concerns.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- refactor backups status underlying resources structure
```
now it looks like:
```
underlyingResources:
kind: VMInstance
apiVersion: apps.cozystack.io/v1alpha1
dataVolumes:
- dataVolumeName: vm-disk-ubuntu-source
applicationName: ubuntu-source
ip: "10.244.2.98"
mac: "9e:92:c9:0c:10:9b"
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Backup status now stores underlying resources as a generic, opaque
JSON payload instead of a fixed typed structure.
* CRD and controller now preserve unknown fields in that payload,
allowing arbitrary application-specific metadata to be retained.
* Restore and controller flows were updated to consistently read and
forward this opaque payload through backup/restore operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Introduce more accurate in-place backup/restores for VMDisk and VMInstance:
- keep IP/MAC of VM
- keep HR of VMDisk related to VMI volumes
- avoid Velero restore of DVs because it conflicts with DV from HR of VMDisk
- propagate to BackupJob/RestoreJob failure messages from Velero to be more clear for namespace users when some job fails
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Capture and persist underlying resource metadata (data volumes, OVN
IP/MAC) in Backup status and Velero annotations; use it to better scope
backups/restores.
* New Backup controller to ensure Velero cleanup for deleted backups.
* Pre-restore preparation: suspend releases, halt VMs, rename PVCs,
delete DataVolumes, and apply resource-modifier rules for claim adoption
and network annotations.
* **Bug Fixes**
* Improved Velero failure messages (includes hooks, async item ops, and
DataUpload failures).
* **Chores**
* Expanded RBAC and added RoleBinding to support Velero/restore
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.2.1`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.2.1.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes - v1.2.1
* **Bug Fixes**
* Fixed LINSTOR DRBD TCP port preservation during toggle operations
* Resolved Multus CNI initialization race conditions
* Improved Multus DEL operation reliability
* **New Features**
* Added custom Keycloak theme injection support
* **Documentation**
* Included guide for using generated Go types
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Always force-create and force-push the API submodule tag so it stays
in sync with the main version tag, even when re-tagging.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
- Remove broken `git push origin HEAD` in detached HEAD state (commit
artifacts are already pushed via the "Create release branch" step)
- Make subtag push idempotent: use explicit refs/tags/ prefix and
tolerate already-existing remote tags after verifying they point to
the correct commit
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Set explicit `domain.resources` with ephemeral-storage on
VirtualMachine spec in KubevirtMachineTemplate
- Limits and requests are calculated via `cozy-lib.resources.sanitize`
using the configured `ephemeralStorage` value and allocation ratio
## Problem
Without explicit ephemeral-storage resources on the VirtualMachine spec,
virt-launcher pods inherit default limits from the namespace LimitRange.
These defaults can be significantly lower than the actual emptyDisk
capacity (e.g. 2Gi limit vs 20Gi disk). When the VM's containerd unpacks
container images, it writes to emptyDir volumes on the host, quickly
exceeding the pod's total ephemeral-storage limit. This causes kubelet
to evict the pod, crashing the VM and creating a restart loop.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved virtual machine resource configuration with explicit
ephemeral storage management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Without explicit ephemeral-storage resources on the VirtualMachine spec,
virt-launcher pods inherit default limits from the namespace LimitRange,
which can be much lower than the emptyDisk capacity. This causes kubelet
to evict the pod when the VM's containerd unpacks images exceeding the
limit.
Set domain.resources with ephemeral-storage calculated via cozy-lib
allocation ratio based on the configured ephemeralStorage value.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Pin multusMasterCNI to `05-cilium.conflist` to prevent race condition
at boot
- Fixes issue where multus auto-detects wrong conflist, bypassing Cilium
CNI chain
## Problem
During node boot (e.g. Talos upgrade), multus auto-detects the master
CNI conflist by scanning
the CNI config directory. If kube-ovn writes `10-kube-ovn.conflist`
before Cilium writes
`05-cilium.conflist`, multus selects the wrong file. Pods on the
affected node then bypass
the Cilium chain entirely, have no Cilium endpoint, and their traffic
gets blocked by
cluster-wide network policies.
## Fix
Set `multusMasterCNI: "05-cilium.conflist"` in the multus daemon config
to explicitly
pin the master CNI file, eliminating the race condition.
## Upstream
- Issue: https://github.com/k8snetworkplumbingwg/multus-cni/issues/1499
- Tracking: https://github.com/cozystack/cozystack/issues/2310
## Test plan
- [x] Verified `multusMasterCNI` JSON key matches upstream struct tag
- [ ] Deploy and verify `00-multus.conf` points to `05-cilium.conflist`
after node reboot
Set multusMasterCNI to explicitly use 05-cilium.conflist instead of
auto-detecting the first available conflist at boot. This prevents a
race condition where multus picks 10-kube-ovn.conflist if it appears
before 05-cilium.conflist during node startup, bypassing the Cilium
CNI chain entirely.
Upstream issue: https://github.com/k8snetworkplumbingwg/multus-cni/issues/1499
Tracking issue: https://github.com/cozystack/cozystack/issues/2310
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Build custom multus-cni image with a patch that fixes sandbox cleanup
deadlock
- When CNI ADD never completes, DEL now returns success instead of
failing
- Prevents stale sandbox name reservations from permanently blocking pod
creation
## Problem
After a node disruption (e.g. DRBD/kube-ovn issues during upgrade),
containerd accumulates
stale sandbox name reservations. Cleanup fails because multus calls
delegate plugins for DEL
without cached state, and they reject the incomplete config. This blocks
all new pods on the
affected node indefinitely.
## Changes
- Add `images/multus-cni/` with Dockerfile and patch for custom image
build
- Add `image` target to multus Makefile
- Add multus to root Makefile build targets
- Update DaemonSet template to use the custom image
## Upstream
- PR: https://github.com/k8snetworkplumbingwg/multus-cni/pull/1498
- Tracking: https://github.com/cozystack/cozystack/issues/2310
## Test plan
- [x] Built and pushed image to ghcr.io/cozystack/cozystack/multus-cni
- [x] Deployed to cluster, verified fix in multus logs ("no cache
found... skip DEL")
- [x] Force-deleted stuck pods, all 24 pods started successfully within
70s
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Multus CNI container image build process to the system.
* **Bug Fixes**
* Fixed delete operation behavior when cache is unavailable, improving
reliability of network cleanup operations.
* **Chores**
* Updated Multus CNI container image reference in deployment
configuration to latest version with build improvements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When CNI ADD never completes, multus DEL fails because delegate plugins
require runtime state that only exists after a successful ADD. This
creates a deadlock where containerd cannot release sandbox name
reservations, permanently blocking pod creation on the affected node.
Build a custom multus-cni image with a patch that returns success on
DEL when no cache file exists (ADD was never completed).
Upstream PR: https://github.com/k8snetworkplumbingwg/multus-cni/pull/1498
Tracking issue: https://github.com/cozystack/cozystack/issues/2310
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Set `DrbdOptions/Net/verify-alg` to `crc32c` at the controller level
to prevent DRBD connection failures on kernels where `crct10dif` is
unavailable
- Fixes an issue where all DRBD resources become Diskless after
upgrading to Talos v1.12.6 (kernel 6.18.18)
## Why
LINSTOR's auto-verify algorithm selection picks `crct10dif` by default,
but this kernel crypto module is no longer available in newer kernels.
This causes DRBD peer connections to fail with `VERIFYAlgNotAvail:
failed to allocate crct10dif for verify`, resulting in lost quorum on
all nodes.
## Test plan
- [ ] Deploy to a cluster running Talos v1.12.6 (kernel 6.18.18)
- [ ] Verify DRBD resources connect and replicate normally
- [ ] Confirm `drbdsetup` uses `crc32c` as verify-alg
Closes#2302
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Configuration**
* Added network verification algorithm configuration option for DRBD
cluster setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.1.5`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.1.5.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Platform packages are now preserved when moved to disabled
configuration or removed from enabled packages.
* DRBD toggle-disk operations now maintain existing TCP ports across
updates, preventing connection failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
This PR ensures PostgreSQL version consistency across the platform by:
- Adding migration 37 to set version v17 for existing PostgreSQL
resources
- Hardcoding PostgreSQL 17.7 image for monitoring databases (Grafana and
Alerta)
## Motivation
CloudNativePG operator defaults to PostgreSQL 18.3 when no explicit
image is specified. However, monitoring queries are configured for
PostgreSQL 17 features (pg_stat_checkpointer, updated pg_stat_bgwriter).
This mismatch could cause issues with existing deployments.
## Changes
- **Migration 37**: Backfills spec.version="v17" for all
postgreses.apps.cozystack.io resources without a version set
- **Monitoring databases**: Explicitly set imageName:
ghcr.io/cloudnative-pg/postgresql:17.7 for:
- Grafana database
- Alerta database
## Testing
- [ ] Migration script tested on existing PostgreSQL resources
- [ ] Verified PostgreSQL 17.7 image is available
- [ ] Confirmed monitoring databases deploy with correct version
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added an automated migration to backfill PostgreSQL version fields
across existing resources.
* Explicitly set PostgreSQL image to 17.7 for monitoring and system
components: Alerta, Grafana, Harbor, Keycloak, and SeaweedFS.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add explicit PostgreSQL 17.7 image to Harbor, SeaweedFS, and Keycloak
databases to ensure consistent version across all system components.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add migration 37 to backfill spec.version=v17 for existing PostgreSQL
resources without a version set.
Hardcode PostgreSQL 17.7 image in monitoring databases (Grafana and Alerta)
to ensure compatibility with monitoring queries that expect PostgreSQL 17
features (pg_stat_checkpointer, updated pg_stat_bgwriter).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The crct10dif kernel crypto module is no longer available in Talos
v1.12.6 (kernel 6.18.18). DRBD auto-verify selects crct10dif by
default, causing peer connections to fail with VERIFYAlgNotAvail.
Setting verify-alg explicitly to crc32c at the controller level
ensures all resources use an available algorithm.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This patch lets Cozystack admins specify initContainers that will run
`cp -r /themes/ /opt/keycloak/themes/` on startup, effectively providing
an interface for operators to inject custom themes into the keycloak
deployment to customize the UI.
### Release note
```release-note
[keycloak] Enable injection of user-provided themes for Keycloak via
initContainers.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for custom Keycloak themes through configuration,
allowing users to customize the appearance of the authentication
interface.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Adds script that tracks reconciliation of platform by checking readiness
of Packages, ArtifactGenerators, ExternalArtifacts, and HelmReleases
resources. It may be ran as singleshot or continuous check while running
upgrades or platform changes. Intended for cozystack devs and platform
admins.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Added check-readiness.sh script
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added new readiness check tool for Kubernetes custom resource types
including packages, artifact generators, external artifacts, and Helm
releases. Enables watch mode (`-w` flag) for continuous monitoring with
configurable refresh intervals, one-shot verification checks, colored
status indicators for clear resource health status, and support for
environment-specific configurations via KUBECONFIG.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes a regression introduced in the bundle restructure (2d022e38) where
`cpuAllocationRatio`, `memoryAllocationRatio`, and
`ephemeralStorageAllocationRatio` from `platform/values.yaml` became
no-ops — never propagated to child packages.
The old monolithic bundles read these values from the cozystack
ConfigMap via `lookup` and passed them explicitly. After migration to
the Package CRD model, the bridge was lost: cozy-lib expects the ratios
in `_cluster` config (via `cozystack-values` Secret), but platform never
wrote them there, so the hardcoded defaults (10, 1, 40) were always used
regardless of what the user configured.
This commit restores the propagation by:
- Adding `cpu-allocation-ratio`, `memory-allocation-ratio`, and
`ephemeral-storage-allocation-ratio` to the `_cluster` section in the
`cozystack-values` Secret, so cozy-lib in all managed applications picks
them up
- Passing `cpuAllocationRatio` to the kubevirt Package component values,
so the KubeVirt CR gets the configured value
### Release note
```release-note
[platform] Fix propagation of resource allocation ratios (cpu, memory, ephemeral-storage) to managed applications and KubeVirt, which were silently ignored since the bundle restructure.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added new resource allocation configuration fields to platform
templates for flexible resource management.
* Enhanced conditional handling of resource settings within container
infrastructure components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Renames the misleading `subnets` field to `networks` in VMInstance. The
old field
name implied the VM was creating subnets, when in reality it attaches
the VM to
existing VPC network attachments (NetworkAttachmentDefinitions). This
change:
- Adds a new `networks` field as the primary way to attach VMs to VPC
networks
- Keeps `subnets` as deprecated with backward-compatible fallback via
Helm `default`
- Fails with a clear error if both `networks` and `subnets` are set
simultaneously
- Adds an API-backed dropdown selector for `networks[].name` listing
available
NetworkAttachmentDefinitions in the namespace
- Hides the deprecated `subnets` field from the dashboard UI
### Release note
```release-note
[vm-instance] Rename `subnets` field to `networks` in VMInstance for clarity.
The old `subnets` field is deprecated but still supported for backward compatibility.
A dropdown selector for available networks has been added to the dashboard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added native networks field for VM attachments and a UI dropdown to
select network attachments; UI hides deprecated subnet fields for VM
instances.
* **Deprecations**
* subnets is deprecated; use networks moving forward (subnets retained
for backward compatibility).
* **Chores / Migration**
* Added a migration to copy existing subnets into networks and bumped
platform migration target version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Enables replication unconditionally in the MariaDB CR, regardless of
replica count.
Previously, single-replica MariaDB instances created a general service
(`mariadb-<name>`) without `-primary`/`-secondary` suffixes. This
caused:
- Dashboard not displaying the service (both dashboard-resourcemap RBAC
and the ApplicationDefinition expect `-primary`/`-secondary`)
- Backup CronJob referencing non-existent `<name>-secondary` service
Also removes a duplicate `template` key in the backup-cronjob YAML that
was silently ignored by the parser.
**BREAKING CHANGE:** Single-replica MariaDB instances will now expose
services as `<name>-primary` and `<name>-secondary` instead of the bare
`<name>`. Applications connecting via the old service DNS name must
update their connection strings.
### Release note
```release-note
[mariadb] BREAKING: MariaDB now always enables replication, creating -primary/-secondary services even for single-replica instances. This fixes dashboard visibility and backup functionality for single-replica setups. Existing single-replica users must update connection strings from `<name>` to `<name>-primary`.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Removed a redundant backup job restart policy that could cause
conflicting behavior.
* Ensured replication is consistently enabled across all deployment
sizes.
* Made external service LoadBalancer settings apply more predictably for
external deployments.
* Adjusted database host selection so single-replica deployments target
the primary instance, improving connection correctness.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Updates the `fix-duplicate-tcp-ports` patch to preserve existing TCP
ports when DrbdRscData is recreated during toggle-disk operations.
Without this fix, `removeLayerData()` frees TCP ports from the number
pool, and `ensureStackDataExists()` may allocate different ports. If the
satellite misses the update (e.g. due to controller restart), it keeps
the old ports while peers receive the new ones, causing DRBD connections
to fail with StandAlone state.
The fix adds `copyDrbdTcpPortsIfExists()` which saves existing TCP ports
into the `LayerPayload` before `removeLayerData()` deletes them.
Also adds `dh_strip_nondeterminism` override in Dockerfile to fix build
failures on some JAR files.
Upstream:
https://github.com/LINBIT/linstor-server/pull/476#issuecomment-4147527442
### Release note
\`\`\`release-note
[linstor] Fix TCP port mismatches after toggle-disk operations that
could cause DRBD resources to enter StandAlone state
\`\`\`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed an issue where DRBD TCP ports were not correctly preserved
during disk toggle operations, which could result in TCP port mismatches
between the controller and satellite nodes.
* Improved robustness of the build and packaging process by addressing
non-determinism handling for Java library dependencies.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Adds annotation `helm.sh/resource-policy: keep` to all packages to
prevent its automatic deletion when:
1, Package added to `disabledPackages` parameter in
`cozystack.cozystack-platform` package
2. Package removed form `enabledPackages` parameter in
`cozystack.cozystack-platform` package
This change bring back [docs explained
behavior](https://cozystack.io/docs/v1/operations/configuration/components/)
- platform administrator should manually delete package if needed.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Prevent installed packages deletion
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Helm templates to configure resource-policy annotations on
package resources, ensuring proper resource lifecycle management and
retention policies during cluster deployments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add securityContext to theme init containers matching the main
container security posture. Add input validation for theme entries:
required fields, DNS-1123 name sanitization, duplicate detection,
and container name length limit. Add imagePullSecrets support for
private registries and sizeLimit on the emptyDir volume.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Update kamaji from edge-26.2.4 to 26.3.5-edge and remove two patches
that have been accepted upstream:
- `increase-startup-probe-threshold.diff` → upstream #1086 added
configurable startupProbeFailureThreshold
- `disable-datastore-check.diff` → upstream #1087 refactored
DataStore initialization
The remaining `fix-kubelet-config-compat.diff` (PR #1084) is still
needed — the maintainer has not accepted the upstream fix for kubelet
config compatibility with K8s < 1.35.
Also updates Makefile to match the new upstream tag format
(`26.x.x-edge` instead of `edge-26.x.x`) introduced in upstream
PR #1094.
Note: `make image` needs to be run to rebuild the container image
and update `values.yaml` with the new tag/digest.
Changes:
- Makefile: update tag grep pattern for new format
- Dockerfile: bump VERSION to 26.3.5-edge
- Remove 2 upstreamed patches
- Update vendored charts from 26.3.5-edge
### Release note
```release-note
[kamaji] Update to 26.3.5-edge, drop 2 upstreamed patches
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Configurable probe tuning for Control Plane components (API Server,
Controller Manager, Scheduler)
* DataStore readiness visible in kubectl and status conditions tracking
* **Improvements**
* Stronger networkProfile validation (CIDR checks, DNS/IP consistency)
* DataStore driver is immutable after creation
* **Chores**
* Removed two validating webhooks
* Updated default packaged manager version used for builds
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.0.7`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.7.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Bug Fixes**
* Resolved RBAC permission issues enabling app creation
* Fixed Service details page access errors
* Corrected Pod serving table rendering
* Repaired navigation links in backup sidebar
* Improved alert accuracy and reduced false positives
* **Documentation**
* Added Backup and Recovery guide for virtualization
* Updated developer documentation with architecture details
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
After migration 36 copies subnets to networks, both fields are populated.
The fail guard would break Helm reconciliation. Instead, networks simply
takes priority via the existing default fallback.
Also handle missing VMInstance CRD gracefully in migration 36.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Existing VMInstance resources store network attachments in the deprecated
spec.subnets field. This migration copies subnets to the new spec.networks
field so the dashboard UI correctly displays attached networks. The old
subnets field is kept intact because migrations run before the new CRD is
applied.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Add fail validation when both 'networks' and 'subnets' are set to
prevent silent data loss during migration. Compute $networks variable
once at template top level to eliminate DRY violation. Regenerate CRD
via make generate instead of manual editing.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Rename the misleading 'subnets' field to 'networks' in VMInstance to
better reflect that it attaches the VM to VPC network attachments, not
creates subnets. The old 'subnets' field is kept as deprecated with
fallback logic for backward compatibility.
Add an API-backed dropdown (listInput) for networks[].name that lists
available NetworkAttachmentDefinitions in the namespace, and hide the
deprecated subnets field from the dashboard UI.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Since the bundle restructure in 2d022e38, the cpuAllocationRatio,
memoryAllocationRatio and ephemeralStorageAllocationRatio values from
platform values.yaml were no longer propagated to child packages.
The old monolithic bundles read these values from the cozystack
ConfigMap via lookup and passed them explicitly. After migration to
the Package CRD model, the bridge was lost: cozy-lib expects the
ratios in _cluster config (via cozystack-values Secret), but platform
never wrote them there, so the hardcoded defaults were always used.
This commit restores the propagation by:
- Adding cpu/memory/ephemeral-storage allocation ratios to the
_cluster section in the cozystack-values Secret, so cozy-lib in
all managed applications picks them up.
- Passing cpuAllocationRatio to the kubevirt Package component, so
the KubeVirt CR gets the configured value.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Update fix-duplicate-tcp-ports patch to preserve existing TCP ports when
DrbdRscData is recreated during toggle-disk operations. Without this,
removeLayerData() frees ports and ensureStackDataExists() may allocate
different ones, causing port mismatches between controller and satellites
if the satellite misses the update.
Also add dh_strip_nondeterminism override in Dockerfile to fix build
failures on some JAR files.
Upstream: https://github.com/LINBIT/linstor-server/pull/476#issuecomment-4147527442
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
This PR adds the changelog for release `v1.2.0`.
✅ Changelog has been created in `docs/changelogs/v1.2.0.md` based on the
complete commit history between v1.1.0 and v1.2.0, including all bug
fixes merged up to the final v1.2.0 tag.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added v1.2.0 release notes documenting user-facing platform updates: a
fully managed OpenSearch PaaS with secure multi-role topology and
optional dashboards; tenant-level scheduling controls; VPC peering for
multi-tenant networking; clustered logging for improved durability and
scaling; storage improvements for automatic replica relocation after
clones/restores; monitoring and Grafana enhancements; dashboard/RBAC
fixes and various platform bug fixes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR prepares the release `v1.2.0`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Chores
- Updated container images across core infrastructure components
(operator, controllers, API services) with new versions and digests
- Migrated Ubuntu container disk image references from temporary to
permanent registry
- Updated image digests for cluster-autoscaler, kubevirt CSI driver, and
monitoring components
- Refreshed dashboard and system service container image versions
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes swapped `job` labels in the `cozy-linstor` VictoriaMetrics
`VMPodScrape` templates.
Before this change:
- `linstor-satellite` metrics were relabeled as `job=linstor-controller`
- `linstor-controller` metrics were relabeled as `job=linstor-satellite`
This caused `linstorControllerOffline` alerts to fire for satellite
endpoints such as `:9942` while reporting that the controller was
unreachable.
After this change:
- `linstor-satellite` metrics keep `job=linstor-satellite`
- `linstor-controller` metrics keep `job=linstor-controller`
This restores consistent alerting and dashboard semantics for LINSTOR
monitoring.
### Release note
```release-note
[linstor] Fix swapped VictoriaMetrics `VMPodScrape` job labels in `cozy-linstor` so `linstorControllerOffline` alerts target the controller instead of satellite endpoints.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected Prometheus relabeling so Linstor controller and satellite
scrapes now receive the correct job labels.
* Ensures metrics are attributed to the proper components, improving
accuracy of monitoring and alerting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
This fixes two issues in the LINSTOR alerts shipped by
`cozy-piraeus-operator`:
- `linstorSatelliteErrorRate` uses a non-existent `name` label in
annotations, which results in `Satellite ""` in notifications.
- `linstorSatelliteErrorRate` can produce false positives when the
`linstor-controller` scrape flaps and historical
`linstor_error_reports_count` counters reappear inside the alert window.
## Test plan
- Verified the rendered rule updates `{{ $labels.name }}` to `{{
$labels.hostname }}` for LINSTOR satellite alerts.
- Verified the `linstorSatelliteErrorRate` expression now requires
stable `up{job="linstor-controller"}` for the full 15 minute window.
- Applied the same logic in a live cluster and confirmed the
false-positive `linstorSatelliteErrorRate` alerts stopped firing once
controller scrape recovered.
## Release note
```release-note
Fix LINSTOR satellite alert annotations and reduce false positives when controller scraping flaps.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Controller alert now evaluates deployment availability and waits 3
minutes before firing to reduce transient noise.
* Satellite error alerts include an additional check requiring the
controller to have been continuously up recently to avoid spurious
triggers.
* Notifications and dashboards now include hostnames for clearer
identification.
* **New Features**
* Added a dedicated warning for controller metrics scrape failures with
a 10-minute delay.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Previously, removing a database from `values.databases` had no effect —
the database and its associated roles (`<db>_admin`, `<db>_readonly`)
persisted in PostgreSQL indefinitely. This made it impossible to
declaratively manage database lifecycle via Helm values.
Added two cleanup stages to the init script, mirroring the existing user
deletion logic:
- **Delete databases** that have the `database managed by helm` comment
but are no longer listed in `values.databases` — active connections are
terminated before dropping
- **Delete orphaned roles** (`<db>_admin`, `<db>_readonly`) with proper
membership revocation (`REVOKE ... FROM`) before `REASSIGN OWNED` /
`DROP OWNED` / `DROP ROLE`
This also fixes the reported issue where creating a database, removing
it, and creating it again would silently retain stale data from the
first instance.
### Release note
```release-note
[postgres] Databases removed from `values.databases` are now properly dropped along with their associated roles. Previously removed databases and roles would persist indefinitely.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced database initialization to automatically remove databases and
roles that were removed from your Helm configuration, avoiding orphaned
resources.
* Now force-terminates active sessions, reassigns owned objects, and
cleans up role memberships to ensure reliable removals.
* Improves consistency of database state during upgrades and
configuration changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes a crash on the Tenant details page when `resourceQuotas` are
configured. The dashboard renders a ResourceQuota table with a "Used"
column that references `$.status.used[_flatMapData_Key]`. When the
flatMap placeholder is not resolved during initial render, the raw
placeholder is passed to the JSONPath parser, causing:
```text
Parse error on line 1: $.status.used[_flatMapData_Key]
Expecting 'STAR', 'SCRIPT_EXPRESSION', 'INTEGER', ... got 'IDENTIFIER'
```
The original `flatmap-dynamic-key.diff` patch that handled this was
removed during the 1.4.0 dashboard upgrade, assuming upstream included
the fix. Upstream adopted the code reordering (flatMap expansion before
customFields resolution) but not the fallback protection for unresolved
placeholders.
Adds a patch to `openapi-k8s-toolkit` that checks for unresolved
`_flatMap*_Key` placeholders after regex substitution and returns `null`
instead of calling `jp.query()` with an invalid expression.
### Release note
```release-note
[dashboard] Fix crash on Tenant details page when resourceQuotas are configured
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Tables now gracefully handle unresolved placeholder fields by treating
them as empty/null instead of attempting lookup, preventing display
errors and preserving previously correct rendering of resolved fields.
* Reduces unexpected array-wrapping in single-value cells so values
display consistently.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds `discovery.k8s.io/endpointslices` read permissions (get, list,
watch) to `cozy:tenant:base` and `cozy:tenant:view:base` ClusterRoles.
Dashboard requests EndpointSlices to display the "Pod serving" section
on the Service details page. Without this permission, tenant users see a
403 error.
### Release note
```release-note
[system] Fix 403 error on Service details page by granting tenants read access to EndpointSlices
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced system permission configurations to improve platform
reliability and ensure proper access controls for core services.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes the "Pod serving" (EndpointSlice) table on the service details
page which displayed "Raw:" prefixes and "Invalid Date" values. The root
cause was a missing `CustomColumnsOverride` for the
`factory-kube-service-details-endpointslice` customization ID referenced
by the `EnrichedTable` component.
Adds column definitions: Pod (`.targetRef.name`), Addresses
(`.addresses`), Ready (`.conditions.ready`), Node (`.nodeName`).
<img width="1113" height="260" alt="Pod serving table"
src="https://github.com/user-attachments/assets/5bf92080-feee-44b3-942f-a019ba388427"
/>
### Release note
```release-note
[dashboard] Fix EndpointSlice rendering in service details — Pod serving table now correctly displays pod name, addresses, readiness, and node.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added new columns to the service endpoints view in the dashboard,
displaying Pod name, IP addresses, Ready status, and assigned Node
information for better visibility into pod serving details.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When replicas=1, the secondary service has no endpoints because the
only pod is the primary. Route backups to the primary service in this
case to ensure they work correctly.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Enable replication unconditionally regardless of replica count.
Previously, single-replica instances created a general service
without -primary/-secondary suffixes, causing dashboard and
backup-cronjob to reference non-existent services.
Also fix duplicate YAML key in backup-cronjob template.
BREAKING CHANGE: Single-replica MariaDB instances will now have
service names with -primary/-secondary suffixes instead of the
bare instance name. Applications connecting via the old service
DNS name need to be updated.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Adds `get`/`list`/`watch` permissions for
`storage.k8s.io/storageclasses` to the `cozystack-dashboard-readonly`
ClusterRole.
The dashboard UI fetches StorageClasses to populate dropdowns (e.g. in
the Postgres form), but authenticated users lacked permissions on this
resource, causing "Error" to be displayed instead of the StorageClass
name.
### Release note
```release-note
[dashboard] Fix StorageClass dropdown showing "Error" by adding storageclasses read access to the dashboard readonly ClusterRole
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced dashboard read-only permissions to include access to storage
classes, improving visibility into cluster storage resources and
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Switch kilo from the cozystack/kilo fork to the upstream squat/kilo
repository (v0.7.0).
All functional patches from the fork have been upstreamed:
- `--internal-cidr` flag for IP auto-detection filtering (#403)
- Allowed-location-ips overlap with node IPs fix (#404)
- Auto-detect WireGuard MTU from underlay interface (#406)
- Preferred source for WireGuard routes (#408)
- Cilium IPIP overlay support (#409)
The fork only retained rebranding and CI-specific commits beyond
upstream, which are no longer needed.
Changes:
- Makefile: point to `squat/kilo` repo and `ghcr.io/squat/kilo` image
- values.yaml: updated via `make update` to `0.7.0` with digest pin
### Release note
```release-note
[kilo] Switch from cozystack/kilo fork to upstream squat/kilo v0.7.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Updates**
* Updated the kilo component to use a different source repository and
new version configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Reduces flakiness in E2E tests by addressing the most common failure
modes.
### Kubernetes tenant tests (`run-kubernetes.sh`)
- Increase node Ready timeout from 2m to 5m — CI runners are shared and
resource-constrained
- Fail fast when nodes are not Ready — saves ~7 minutes per failed
attempt by not running LB/NFS tests that will also fail
- Delete stale Kubernetes resources at test start — retries provision
from scratch instead of patching stuck state
- Wait for port-forward to be ready before using it (fixes race
condition)
- Reduce version check polling interval from 5s to 1s
### All app tests (postgres, redis, kafka, clickhouse, mariadb, mongodb,
qdrant, foundationdb, openbao, vminstance, bucket)
- Add pre-cleanup of stale resources from previous failed retries so
each attempt starts clean
### Test runner (`cozytest.sh`)
- Clean up temp directory on test failure (was only cleaned on success,
leaking `/tmp` dirs)
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Improved e2e robustness by deleting stale resources before creation
(ignore-if-missing, bounded time) and killing leftover port-forwards.
* Added an exit cleanup to consistently remove temporary artifacts and
teardown port-forwards on any exit.
* Added readiness polling for forwarded endpoints (curl with timeout);
made API/version retries more responsive.
* Extended node readiness wait (2m → 5m), dump debug info and fail fast
if unready.
* Made load‑balancer reachability checking explicit and more reliable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds 7 missing application resources to the `cozy:tenant:admin:base`
ClusterRole for `apps.cozystack.io`:
foundationdbs, harbors, mongodbs, openbaos, opensearches, qdrants, vpns.
Without these permissions, tenant admins could not create these
applications — the "Add" button was inactive in the dashboard.
### Release note
```release-note
[platform] Fix tenant admins unable to create FoundationDB, Harbor, MongoDB, OpenBAO, OpenSearch, Qdrant, and VPN applications by adding missing RBAC permissions
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Expanded tenant administrator permissions to manage additional
platform resources, including databases, search services, and network
infrastructure components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds missing `baseFactoriesMapping` entries for backup resources (plans,
backupjobs, backups) to the static Navigation resource.
Backup resources are not ApplicationDefinitions, so `ensureNavigation()`
never creates their navigation mappings. Without these mappings, the
OpenUI frontend cannot resolve the `{cluster}` context for backup pages,
producing broken sidebar links with an empty cluster segment (e.g.
`/openapi-ui//tenant-root/api-table/backups.cozystack.io/v1alpha1/plans`
instead of `/openapi-ui/default/tenant-root/...`).
### Release note
```release-note
[dashboard] Fix broken backup menu links by adding missing baseFactoriesMapping entries for backup resources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Backup resources are now integrated into dashboard navigation,
providing quick access to backup plans, backup job monitoring, and
detailed backup information for streamlined operations management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cozy:tenant:admin:base ClusterRole was missing several application
resources from apps.cozystack.io, preventing tenant admins from creating
them (the "Add" button was inactive in the dashboard).
Add the following resources: foundationdbs, harbors, mongodbs, openbaos,
opensearches, qdrants, vpns.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The dashboard UI fetches StorageClasses via the Kubernetes API to populate
dropdowns (e.g. in the Postgres form), but the cozystack-dashboard-readonly
ClusterRole did not include storage.k8s.io/storageclasses. This caused
authenticated users to see "Error" instead of the StorageClass name.
Add get/list/watch permissions for storageclasses to the dashboard readonly
role, consistent with the existing backupclasses entry.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Update COSI controller/sidecar from commit `c2f6e65` to upstream
release `v0.2.2` and remove two patches that have been merged:
- `89-reconciliation.diff` — fix reconciliation logic for existing
and deleted objects (upstream PR #89)
- `90-bucket-name.diff` — prefix bucket names with `bucket-` using
UID only (upstream PR #90)
Both were contributed by kvaps and included in the v0.2.2 release
(2025-12-08).
Also removes `git` from Dockerfile dependencies as it was only
needed for applying patches.
Note: `make image` needs to be run to rebuild the container image
and update `values.yaml` with the new tag/digest.
### Release note
```release-note
[objectstorage-controller] Update to v0.2.2, drop upstreamed patches
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Updated object storage controller build configuration to streamline
upstream component retrieval and image build process.
* Modified resource deletion handling so deletion flows are centralized
and consistently enforced across create/update events.
* Changed bucket naming scheme—buckets will no longer include the bucket
class name in their identifier and follow the simplified naming pattern.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Extends the `monitoring` system chart with additional optional
configuration for the Alerta Slack plugin, a VMAgent environment label,
and adds a Grafana plugin to the custom image build.
### Alerta — Slack plugin (`alerta.alerts.slack`)
| Value | Env var | Description |
|---|---|---|
| `dashboardUrl` | `DASHBOARD_URL` | Links Slack alert messages back to
the Alerta console |
| `summaryFmt` | `SLACK_SUMMARY_FMT` | Jinja2 template string for
customizing Slack message format |
All new fields are optional and default to empty, preserving existing
behavior when unset.
### VMAgent — external labels (`vmagent.externalLabels`)
- Added optional `environment` label that gets attached to all scraped
metrics (e.g. `Staging`, `Production`)
- When left empty, no label is added and Alerta falls back to its
built-in default of `"Production"`
### Grafana — custom image
- Added
[`marcusolsson-dynamictext-panel`](https://grafana.com/grafana/plugins/marcusolsson-dynamictext-panel/)
plugin to the Grafana image build
Added marcusolsson-dynamictext-panel plugin to the Grafana image build
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[monitoring] Add optional Slack DASHBOARD_URL and SLACK_SUMMARY_FMT configuration for Alerta, optional vmagent environment external label, and marcusolsson-dynamictext-panel Grafana plugin
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an additional Grafana visualization panel to expand dashboard
options.
* Extended Slack alerting with optional dashboard URL and customizable
summary format.
* Added an optional environment label for external metric ingestion to
improve context and filtering.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The hunk header claimed 8/13 lines but only 6/11 were present,
causing "corrupt patch at line 18" during git apply in the
openapi-ui image build.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The service details page showed "Raw:" and "Invalid Date" in the Pod
serving table because the EnrichedTable referenced customizationId
"factory-kube-service-details-endpointslice" which had no corresponding
CustomColumnsOverride. Add column definitions for Pod, Addresses, Ready,
and Node fields.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
## Developer Experience
### Go types published on pkg.go.dev
Per-app configuration types (Postgres, Kubernetes, Tenant, Redis, etc.)
are now published as a standalone Go module at
`github.com/cozystack/cozystack/api/apps/v1alpha1`. Each Cozystack
application has its own package with strongly-typed structs generated
from `values.yaml`. The module is tagged automatically on every release.
Import example:
```go
import "github.com/cozystack/cozystack/api/apps/v1alpha1/postgresql"
```
### OpenAPI schema released as a static artifact
The full OpenAPI 3.0 schema for the `apps.cozystack.io` API is now
generated at release time and attached to each GitHub release as
`openapi.json`. Previously this schema was only available from a live
running cluster. The schema covers all managed application kinds and
their configuration fields.
### Automated website docs updates on release
When a new Cozystack version is released, a pull request is
automatically opened in the
[cozystack/website](https://github.com/cozystack/website) repository
with updated application reference documentation and the new
`openapi.json`. This replaces the previous daily cron-based sync that
ran in the website repo.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[docs] Added per-app Go types as a standalone module at `github.com/cozystack/cozystack/api/apps/v1alpha1`
[docs] Added `openapi.json` artifact generation on every release
[docs] Automatic PRs on website repo on every release to keep docs up to date
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for many new apps and VM/network CRD schemas
(ClickHouse, FoundationDB, Harbor, Kafka, MariaDB, MongoDB, PostgreSQL,
Redis, RabbitMQ, NATS, Qdrant, and more).
* Produce a versioned OpenAPI v3 spec (openapi.json) as a build
artifact.
* **Documentation**
* Release workflow now updates website docs from each release and
creates/updates PRs for docs changes.
* **Chores**
* Improved release tagging, asset publishing, generation workflows and
Makefile generation integration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace separate pg_terminate_backend + DROP DATABASE calls with a
single DROP DATABASE ... WITH (FORCE) statement. This eliminates the
race window where new sessions could reconnect between termination
and drop. Available since PostgreSQL 13.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Root cause: EXIT trap called _k8s_test_cleanup after subshell exit,
but variables ${port} and ${test_name} were not available in that
context. set -u caught the unset variable and exited with error,
marking a successful test as failed. This caused every kubernetes
test to "fail" on attempt 1, then retry 2 more times with real
failures (NFS PVC timeout because the cluster was being deleted).
Fix: remove trap and cleanup function entirely. Pre-cleanup at
test start handles stale resources. Inline cleanup at test end
handles normal path. On failure, resources are cleaned up by
pre-cleanup on the next retry.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Root cause of NFS test failures: port-forward had `timeout 500s`
(8.3 minutes), but tests after it take up to 26 minutes (node
join 8m + node Ready 5m + LB test 2.5m + NFS 10m). Port-forward
was guaranteed to die mid-test.
Remove the timeout entirely. Cleanup is already handled by:
- EXIT trap (_k8s_test_cleanup does pkill port-forward)
- Explicit cleanup at every exit point
- Job-level timeout-minutes: 120 as backstop
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Increase NFS PVC Bound timeout from 2m to 5m — RWX via kubevirt
CSI provisions an NFS server pod which takes time
- Add pod describe and events dump when NFS test pod fails to
complete, making the root cause visible in CI logs
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Without explicit timeouts, stuck jobs use the GitHub Actions
default of 360 minutes (6 hours). This blocks the concurrency
group and prevents new runs from starting.
Set reasonable limits:
- Build: 30 minutes (normally ~5 min)
- E2E: 120 minutes (normally ~60 min)
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
On resource-constrained CI runners, HelmReleases may take close
to 15 minutes to reconcile, leaving very little time for CAPI
deployments to start and become available. Increase timeouts:
- CAPI deployment discovery: 60s -> 180s
- CAPI deployment availability: 1m -> 5m
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The explicit _k8s_test_cleanup calls cover known error paths, but
set -e can terminate the script at any kubectl/wait command. Add
an EXIT trap to catch these unexpected exits. The function runs
in a cozytest.sh subshell, so the trap is scoped and does not
affect parent traps. Removed 2>/dev/null from kubectl delete in
cleanup to preserve useful error output.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Use a _k8s_test_cleanup helper called at every exit point
instead of a process-scoped EXIT trap. This avoids any
potential trap scope issues and makes cleanup behavior
explicit and visible at each error path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Expand the EXIT trap in run_kubernetes_test to also delete the
Kubernetes tenant resource. Previously, early exit paths (exit 1
on node readiness failure, LB failure, etc.) would skip the final
kubectl delete, leaving the resource behind.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Add pre-cleanup for VMInstance in second vminstance.bats test
- Fix indentation of LB_ADDR block in run-kubernetes.sh
- Add --ignore-not-found to final Kubernetes resource cleanup
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Add EXIT trap in run-kubernetes.sh to clean up port-forward
process and tenantkubeconfig file on any exit path
- Fix LoadBalancer retry loop: success on attempt 20 was falsely
reported as failure due to $i -eq 20 check after break
- Kill stale port-forward in bucket.bats before retry
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Add pre-cleanup of stale resources to all app E2E tests so
retries start fresh instead of patching stuck state
- Wait for port-forward to be ready before using it in
kubernetes tests (fixes race condition)
- Reduce version check sleep from 5s to 1s for faster retries
- Clean up temp directory on test failure in cozytest.sh
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Three changes to reduce flakiness in kubernetes E2E tests:
- Increase node Ready timeout from 2m to 5m to accommodate
resource-constrained CI runners
- Fail fast when nodes are not Ready instead of continuing to
LB/NFS tests that will also fail, saving ~7 minutes per attempt
- Delete stale Kubernetes resources at test start to ensure
retries provision from scratch instead of patching stuck state
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace commit hash with v0.2.2 tag for better readability.
GitHub archive URLs work with tags identically.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Use -F/ with $NF (last field) instead of -F'[/^]' with $3.
The ^{} separator is unnecessary since the grep pattern already
filters out annotated tag dereferences.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update COSI controller/sidecar from commit c2f6e65 to v0.2.2
(f75d475) and remove two patches that have been merged upstream:
- 89-reconciliation.diff: fix reconciliation logic for existing
and deleted objects (PR #89)
- 90-bucket-name.diff: prefix bucket names with bucket- using
UID only (PR #90)
Both patches were contributed by kvaps and included in the
upstream v0.2.2 release (2025-12-08).
Also remove git from Dockerfile dependencies as it was only
needed for applying patches.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update kamaji from edge-26.2.4 to 26.3.5-edge and remove patches
that have been accepted upstream:
- increase-startup-probe-threshold.diff: upstream PR #1086 added
configurable startupProbeFailureThreshold field
- disable-datastore-check.diff: upstream PR #1087 refactored
DataStore initialization, removing the blocking startup check
The remaining fix-kubelet-config-compat.diff patch (PR #1084) is
still needed as the maintainer has not accepted the upstream fix
for kubelet config compatibility with K8s < 1.35.
Also update Makefile to match the new upstream tag format
(26.x.x-edge instead of edge-26.x.x) introduced in PR #1094.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
All functional patches from the cozystack/kilo fork have been
upstreamed and included in squat/kilo v0.7.0:
- --internal-cidr flag (PR #403)
- allowed-location-ips overlap fix (PR #404)
- Auto-detect WireGuard MTU (PR #406)
- Preferred source for WireGuard routes (PR #408)
- Cilium IPIP overlay support (PR #409)
The fork only contained rebranding and CI-specific commits beyond
upstream, which are no longer needed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Dashboard requests EndpointSlices from discovery.k8s.io API group
to display "Pod serving" section on Service details page. Without
this permission, tenant users see a 403 error.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Install the cozystack-scheduler package and the SchedulingClass CRD by
default.
### Release-note
```release-note
[cozystack-scheduler] Enable the cozystack-scheduler and SchedulingClass
CRD by default.
```
## What this PR does
Install the cozystack-scheduler package and the SchedulingClass CRD by
default.
### Release-note
```release-note
[cozystack-scheduler] Enable the cozystack-scheduler and SchedulingClass
CRD by default.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Removes the `apiServerEndpoint` fallback for `MASTER_NODES` in the
`isp-full-generic` variant.
When `MASTER_NODES` is not explicitly set (the default), the template
used to parse a single IP from `apiServerEndpoint` and pass it to
kube-ovn. This prevented kube-ovn from discovering all control-plane
nodes, breaking OVN RAFT consensus on multi-master clusters
(`ovn-central` crashed with `host ip X not in env NODE_IPS`).
Now kube-ovn uses its built-in Helm `lookup` to find all control-plane
nodes by the `MASTER_NODES_LABEL`
(`node-role.kubernetes.io/control-plane=true`), correctly discovering
all masters.
Explicit `MASTER_NODES` override via platform values is preserved.
The `isp-full` (Talos) variant is not affected — it uses a separate code
path.
Fixes#2242
### Release note
```release-note
[kube-ovn] Fix multi-master support for generic Kubernetes clusters by letting kube-ovn auto-discover control-plane nodes by label instead of falling back to a single API server IP
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced support for automatic control plane node discovery using
Kubernetes node labels, providing greater flexibility and scalability
options for cluster configurations.
* **Bug Fixes**
* Removed unreliable automatic node configuration fallback logic to
improve overall deployment consistency and predictability. Master nodes
now require explicit configuration or automatic discovery through
Kubernetes node labels.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add bilateral VPC peering using Kube-OVN's native `vpcPeerings`
mechanism
- Auto-allocate link-local peering IPs (`169.254.0.0/16`)
deterministically from sorted VPC pair hash
- Add static routes support (`staticRoutes`) for fine-grained inter-VPC
routing
- Enrich ConfigMap with peer discovery info
- Add JSON Schema validation with `^tenant-` namespace pattern
enforcement
Closes#2151
## How it works
Each tenant independently declares peers in their VPC spec. The template
computes the remote VPC's internal ID using the same deterministic hash
formula, and derives complementary peering IPs from a sorted pair key.
Kube-OVN only activates the peering tunnel when **both sides** declare
each other — no custom controller needed.
Example (Tenant A):
```yaml
spec:
subnets:
- name: web
cidr: "172.16.0.0/24"
peers:
- vpcName: "backend-vpc"
tenantNamespace: "tenant-beta"
```
## Test plan
- [ ] `helm template` without peers → no `vpcPeerings` in output
(backward compatible)
- [ ] `helm template` with peers → correct `vpcPeerings` with
auto-allocated IPs
- [ ] Reciprocal peering: both sides reference each other with
complementary IPs in same /30
- [ ] Schema rejects `tenantNamespace` not matching `^tenant-`
- [ ] Deploy two VPCs with reciprocal peering on a live cluster →
Kube-OVN activates tunnel
- [ ] Remove one side → peering deactivated
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* VPC Peering: declare peer VPCs (tenant + name) to enable conditional
peering and automatic per-peer connection entries.
* Per-peer metadata: subnet/config outputs include peer-specific
connection details (remote VPC ID and local connect IP).
* Static Routes: optionally add custom CIDR routes with next-hop IPs for
a VPC.
* Values/schema: configuration and schema now accept peers and routes
arrays.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.0.6`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.6.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes v1.0.6
* **Bug Fixes**
* Resolved multi-node storage volume publication issues
* Fixed dashboard authentication to properly sync credentials
* Enhanced etcd stability during cluster upgrades to prevent service
interruptions
* **Documentation**
* Added new guides for authentication configuration
* Published v1.0 release announcement blog post
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.1.3`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.1.3.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved Kubernetes CiliumNetworkPolicy handling for multi-node RWX
scenarios
* Enhanced Keycloak client secret reconciliation for the dashboard
* Optimized etcd defrag CronJob behavior during system upgrades
* **Documentation**
* Updated authentication configuration documentation
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Use [^\]]+ (same as existing code in utils.ts) instead of [^[\]]+ for
consistency.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
When viewing Tenant details with resourceQuotas, the "Used" column
references $.status.used[_flatMapData_Key]. If the flatMap placeholder
is not resolved (key missing from expanded row), the raw placeholder
is passed to the JSONPath parser which crashes with a parse error.
The original flatmap-dynamic-key.diff patch that handled this was
removed during the 1.4.0 upgrade, assuming upstream included the fix.
Upstream adopted the reordering (flatMap before customFields) but not
the fallback protection.
Add a patch that checks for unresolved _flatMap*_Key placeholders
after regex substitution and returns null instead of calling jp.query()
with an invalid expression.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Update the Cozystack scheduler to its latest version, supporting
automatic placement of label selectors for affinity and topology spread
constraints.
### Release note
```release-note
[cozystack-scheduler] Update Cozystack scheduler to v0.2.0 with support
for reusable affinity terms.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for configurable default label selector keys to
customize scheduler behavior.
* **Chores**
* Updated scheduler component to version 0.2.0 with refreshed container
image.
* Enhanced Kubernetes permissions for scheduling class access.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch lets Cozystack admins specify initContainers that will run
`cp -r /themes/ /opt/keycloak/themes/` on startup, effectively providing
an interface for operators to inject custom themes into the keycloak
deployment to customize the UI.
```release-note
[keycloak] Enable injection of user-provided themes for Keycloak via
initContainers.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
When only leaf packages change (accounting, dashboard, etc.) but the
platform chart
itself remains unchanged, the Flux ArtifactGenerator produces an
ExternalArtifact with
the same content digest. This prevents the platform HelmRelease from
reconciling, so
`lookup()` in `repository.yaml` never runs and `cozystack-packages`
OCIRepository does
not pick up the new OCI digest.
This PR writes a content-based hash of the working tree state to
`core/platform/.build-revision` before pushing the OCI artifact,
ensuring the platform
chart content always changes on every build.
### Release note
```release-note
[platform] Fixed cozystack-packages OCIRepository not updating when only leaf packages change
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved build system infrastructure with enhanced revision tracking
and artifact management configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Previously, removing a database from values.databases had no effect —
the database and its associated roles persisted in PostgreSQL. This
made it impossible to cleanly delete databases via Helm.
Add two cleanup stages to the init script:
- Delete databases that have the 'database managed by helm' comment
but are no longer listed in values.databases
- Delete orphaned roles (db_admin, db_readonly) with proper membership
revocation before dropping
This mirrors the existing user deletion logic and completes the
declarative lifecycle for databases.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## Changes
Mount the `/data` subdirectory of the NFS export instead of the root
filesystem in `NodePublishVolume`, so tenant pods no longer see internal
LINSTOR/CDI artifacts (`disk.img`, `lost+found`) in their RWX PVC mount
points.
### Backward compatibility
On first mount after upgrade, any existing user files at the NFS root
are automatically migrated into `/data`. Internal artifacts (`disk.img`,
`lost+found`, `data`) are skipped. This is transparent to the user.
### How it works
1. Temp-mount the NFS root to a scratch directory
2. Create `/data` subdirectory if it doesn't exist
3. Move any user files from root into `/data` (migration for pre-fix
volumes)
4. Unmount the temp mount
5. Mount `<nfs-export>/data` at the pod's target path
## Test plan
- [ ] Create a new RWX PVC on a tenant cluster → `ls` shows no
`disk.img` or `lost+found`
- [ ] Write a file from one pod, read it from another pod on a different
node
- [ ] Mount a pre-existing RWX PVC (created before fix) → user data is
migrated into `/data` automatically
- [ ] `go build ./...` passes in
`packages/apps/kubernetes/images/kubevirt-csi-driver/`
- [ ] `make generate` passes at repo root
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* NFS mounts now expose only the data subdirectory, hiding internal
artifacts and avoiding clutter at mount targets.
* Existing read-only and already-mounted behaviors preserved.
* User files on existing mounts are migrated into the data subdirectory
during transition to avoid data loss; migration failures now cause the
publish to fail to protect data.
* Improved error handling, cleanup, and detailed logging during mount,
migration, and unmount steps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
upgrades CNPG operator to v1.27.3
Used chart version 0.27.1. Image in default values set to 1.27.3. The
default App version for this chart is 1.28.1.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[postgres-operator] Update to v1.27.3
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Deployment update strategy and topologySpreadConstraints for operator
pods
* Webhook startupProbe support and new database mutation/validation
webhooks
* Monitoring: added pg_extensions query and target_databases
* Expanded RBAC to include failoverquorums and image/catalog resources
* Namespace templating standardized across manifests
* **Bug Fixes**
* Corrected documentation typo
* **Chores**
* Helm chart bumped to v0.26.1 / appVersion 1.27.1; operator image tag
set to 1.27.3
* Added SPDX license headers and updated README badges and docs
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When only leaf packages change but the platform chart remains unchanged,
the Flux ArtifactGenerator produces an ExternalArtifact with the same
content digest. This prevents the platform HelmRelease from reconciling,
so lookup() in repository.yaml never runs and cozystack-packages
OCIRepository does not pick up the new OCI digest.
Write a content-based hash of the working tree state to
core/platform/.build-revision before pushing the OCI artifact, ensuring
the platform chart content always changes on every build.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
ReadDir and Rename errors during migration are now hard failures
instead of warnings, preventing user data from becoming invisible
behind the /data subpath mount. Tolerate os.IsNotExist on Rename
for concurrent publish calls that already moved the file.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
## What this PR does
Update the Cozystack scheduler to its latest version, supporting
automatic placement of label selectors for affinity and topology spread
constraints.
### Release note
```release-note
[cozystack-scheduler] Update Cozystack scheduler to v0.2.0 with support
for reusable affinity terms.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
- Use os.MkdirTemp instead of predictable path (fixes race condition/TOCTOU)
- Use defer for cleanup (unmount + remove temp dir)
- Handle os.ReadDir and os.Rename errors with klog warnings
- Log unmount failures instead of ignoring them
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Remove the apiServerEndpoint fallback for MASTER_NODES in the
isp-full-generic variant. When MASTER_NODES is not explicitly set,
kube-ovn now uses its built-in Helm lookup to find all control-plane
nodes by the MASTER_NODES_LABEL, correctly discovering all masters
instead of just the single API server IP.
The previous fallback always produced a single IP (the API endpoint),
which broke OVN RAFT consensus on multi-master clusters because
ovn-central was configured with only one node in NODE_IPS.
Explicit MASTER_NODES override is preserved for users who need it.
Fixes: #2242
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Mount the /data subdirectory of the NFS export instead of the root,
so pods no longer see internal LINSTOR/CDI artifacts (disk.img,
lost+found). On first mount after upgrade, any existing user files
at the NFS root are automatically migrated into /data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
## What this PR does
Adds infrastructure support for restricting tenant workloads to specific
data centers (or node groups) via SchedulingClass CRs.
- Adds `schedulingClass` parameter to the Tenant model, inherited by
child
tenants (children cannot override parent's scheduling constraints)
- Adds `scheduling.cozystack.io/class` label to tenant namespaces
- Adds SchedulingClass dropdown to the Tenant creation form in the
dashboard
- Extends lineage-controller-webhook to inject `schedulerName:
cozystack-scheduler`
and `scheduler.cozystack.io/scheduling-class` annotation into all Pods
in
namespaces with the scheduling label
- Webhook verifies the referenced SchedulingClass CR exists before
injection;
if missing (scheduler not installed), pods are left untouched
### Operators without native schedulerName support
The following operators do not support `schedulerName` in their CRDs,
but are still covered by the webhook-based approach since it mutates
pods directly:
- etcd-operator
- redis-operator (spotahome)
- mariadb-operator
- clickhouse-operator (altinity)
### Release note
```release-note
[scheduler] Add SchedulingClass support for tenant workloads.
Administrators can assign a SchedulingClass to a tenant, restricting
where its workloads are scheduled. The lineage-controller-webhook
injects the custom scheduler into all pods in the tenant namespace.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a schedulingClass parameter for tenant configuration; it can be
set in values, appears in schema/docs, inherits to namespaces, and is
included in exported tenant manifests.
* When present, the schedulingClass is propagated to namespace metadata
and applied to workloads so scheduler selection and scheduling
annotations reflect the setting.
* **Bug Fixes**
* Improved owner-resolution and labeling so resources and scheduling are
associated more accurately.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Import SchedulingClassAnnotation, SchedulingClassLabel, SchedulerName,
and GVR components from the cozystack-scheduler/pkg/apis sub-module
instead of defining them locally. This ensures the webhook and scheduler
stay in sync on label/annotation keys.
Also standardize the namespace label from scheduling.cozystack.io/class
to scheduler.cozystack.io/scheduling-class for consistency with the
scheduler, and resolve scheduling class from the owner Application CR
(via a new SchedulingClass() stub method) before falling back to the
namespace label.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Adds a `secret-hash` annotation to the dashboard `KeycloakClient` CRD
resource containing the SHA256 hash of the client secret value.
Without this annotation, if the `dashboard-client` Kubernetes Secret is
recreated with a new value (e.g. after upgrade or reinstall), the
`KeycloakClient` spec remains unchanged, the EDP Keycloak operator skips
reconciliation, and Keycloak retains the stale secret — causing
authentication failures for the dashboard.
With the annotation, any change to the secret value updates the hash in
metadata, triggering operator reconciliation and syncing the new secret
to Keycloak.
This aligns the implementation with cozyportal, where the same pattern
is already used.
### Release note
```release-note
[dashboard] Fix dashboard-client secret desynchronization with Keycloak after upgrades by adding secret-hash annotation to KeycloakClient resource
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced authentication service configuration metadata tracking to
improve system management and operational reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds missing protective limits to the etcd defrag CronJob to prevent job
accumulation during cluster upgrades.
After upgrading CozyStack to v1.1.2, defrag CronJobs accumulated
hundreds of running/failed pods across tenants because etcd was
temporarily unavailable during the upgrade. Without `concurrencyPolicy:
Forbid`, new jobs kept being created hourly while previous ones were
still failing.
Changes:
- `concurrencyPolicy: Forbid` — prevent parallel defrag jobs
- `startingDeadlineSeconds: 300` — don't catch up on missed schedules
older than 5 minutes
- `failedJobsHistoryLimit: 1` — limit failed job retention
- `activeDeadlineSeconds: 1800` — 30 minute timeout per job
- `backoffLimit: 2` — limit retries to 2 instead of default 6
### Release note
```release-note
[etcd] Fix defrag CronJob accumulating hundreds of pods during cluster upgrades by adding concurrencyPolicy, job timeout, and retry limits
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated etcd defragmentation job configuration to enhance reliability
and operational stability in production environments. New controls
prevent concurrent job executions to avoid resource contention, enforce
startup and execution deadlines, limit retry attempts for failed
operations, and maintain failure history for improved monitoring and
troubleshooting. These improvements ensure more predictable and stable
defragmentation operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The required fields in schema are auto-generated by cozyvalues-gen
and must be kept in sync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthieu <matthieu@hidora.com>
Remove sysctl DaemonSet in favor of operator's built-in setVMMaxMapCount
init container, fix schema conflicts with required+empty default, fix
update-versions.sh YAML output format, and use $(MAKE) in Makefile.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthieu <matthieu@hidora.com>
Without concurrencyPolicy and job limits, the defrag CronJob can
accumulate hundreds of running/failed pods during cluster upgrades
when etcd is temporarily unavailable. This was observed after
upgrading to v1.1.2 where defrag jobs piled up across tenants.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Backup resources (plans, backupjobs, backups) are not
ApplicationDefinitions, so ensureNavigation() never adds their
baseFactoriesMapping entries. Without these mappings the OpenUI
frontend cannot resolve the {cluster} context for backup pages,
producing broken sidebar links with an empty cluster segment
(e.g. /openapi-ui//tenant-root/...).
Add the three missing entries to the static Navigation resource.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Add secret-hash annotation to the dashboard KeycloakClient CRD resource
so that when the Kubernetes Secret value changes, the operator detects
the CRD update and reconciles the client secret in Keycloak.
Without this annotation, if the dashboard-client Secret is recreated
with a new value (e.g. after upgrade), the KeycloakClient spec remains
unchanged, the operator skips reconciliation, and Keycloak retains the
stale secret — causing authentication failures for the dashboard.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
When an NFS-backed RWX volume is published to multiple VMs, the
`CiliumNetworkPolicy` `endpointSelector.matchLabels` only included the
first VM. Subsequent `ControllerPublishVolume` calls added
`ownerReferences` but never broadened the selector, causing Cilium to
block NFS egress — mounts hang on all nodes except the first.
This PR switches from `matchLabels` to `matchExpressions` (`operator:
In`) so the selector can list multiple VM names, and rebuilds it
whenever ownerReferences are added or removed.
### Release note
```release-note
[kubernetes] Fixed CiliumNetworkPolicy endpointSelector not being updated when NFS-backed RWX volumes are published to multiple VMs, which caused NFS mounts to hang on all nodes except the first.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* KubeVirt CSI driver now supports selecting and targeting multiple
virtual machines for volume publishing.
* **Improvements**
* Network policy targets are rebuilt automatically when VM ownership
references change, improving correctness and lifecycle handling in
multi-VM scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Adds `inlineScrapeConfig` support to the tenant `VMAgent` resource in
`packages/system/monitoring/templates/vm/vmagent.yaml`
- Adds commented usage example in both
`packages/system/monitoring/values.yaml` and
`packages/extra/monitoring/values.yaml`
## Problem
The tenant VMAgent resource does not support custom scrape
configurations. Using `additionalScrapeConfigs` (which references a
Kubernetes Secret) is not viable because tenant users have no access to
create or read Secrets — they can only manage resources under
`apps.cozystack.io`. Instead, `inlineScrapeConfig` accepts inline YAML
directly through Helm values, which is consistent with tenant
permissions.
Relates to #2194
## Usage
```yaml
vmagent:
inlineScrapeConfig: |
- job_name: "custom"
static_configs:
- targets: ["my-service:9090"]
```
## Test plan
- [ ] `helm template` monitoring with `inlineScrapeConfig` set — block
rendered
- [ ] `helm template` monitoring without it — no `inlineScrapeConfig` in
output
- [ ] Deploy and verify custom scrape targets are picked up by vmagent
```release-note
Add inlineScrapeConfig support to tenant vmagent
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Support for including an inline scrape configuration into VMAgent
monitoring when provided.
* **Documentation**
* Added a commented example showing how to supply an inline scrape job
and targets.
* Reordered remote-write URL entries in monitoring configuration for
clearer ordering.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds `authentication.oidc.keycloakInternalUrl` platform value that
allows oauth2-proxy
in the dashboard to route backend calls (token exchange, JWKS, userinfo,
logout) through
an internal cluster URL while keeping browser redirects on the external
Keycloak URL.
When set, oauth2-proxy uses `--skip-oidc-discovery` and explicit
endpoint URLs pointing
to the internal Keycloak service. This avoids external DNS lookups and
TLS overhead for
pod-to-pod communication.
Fully backward-compatible: when the value is empty (default), behavior
is unchanged.
### Release note
```release-note
[dashboard] Added `authentication.oidc.keycloakInternalUrl` platform value to route oauth2-proxy backend requests through internal Keycloak service URL, bypassing external DNS and TLS.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configuration support for an internal KeyCloak URL, enabling
backend authentication requests to be routed through an alternative
endpoint while maintaining existing external URLs for browser
interactions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When an NFS-backed RWX volume is published to multiple VMs, the
CiliumNetworkPolicy egress rule only allowed traffic from the first VM.
The endpointSelector.matchLabels was set once on creation and never
broadened, causing NFS mounts to hang on all nodes except the first.
Switch from matchLabels to matchExpressions (operator: In) so the
selector can list multiple VM names. Rebuild the selector whenever
ownerReferences are added or removed.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[backups] Added fix to roles and changed backupstrategy-controller location
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated backup controller namespace configuration.
* Enhanced backup controller permissions for leader election and event
recording capabilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Remove merged RWX validation patch (`001-rwx-validation.diff`)
- Add new patch (`001-relocate-after-clone-restore.diff`) that includes:
- Randomized node selection for snapshot restore
- `linstor.csi.linbit.com/relocateAfterClone` StorageClass parameter to
relocate replicas to optimal nodes after clone
- `snap.linstor.csi.linbit.com/relocate-after-restore`
VolumeSnapshotClass parameter to relocate replicas to optimal nodes
after snapshot restore
- Both parameters are **disabled by default**
- Placing the snapshot restore parameter in VolumeSnapshotClass prevents
unwanted relocation when Velero creates temporary PVCs during data mover
backup
Upstream PRs:
- https://github.com/piraeusdatastore/linstor-csi/pull/418
- https://github.com/piraeusdatastore/linstor-csi/pull/419
## Test plan
- [x] Clone a PVC and verify relocation logic executes
- [x] Restore a PVC from snapshot and verify replicas get migrated to
optimal nodes
- [x] Verified on dev5 cluster (3-node) — snapshot restore triggered
actual migration (node0 → node2)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Automatic resource relocation after clone and restore operations
optimizes storage placement and load balancing across nodes.
* RWX block attachment validation with optional disable flag ensures
proper multi-pod access control.
* **Chores**
* Updated CDI clone strategy to use CSI-clone for improved efficiency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.1.2`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.1.2.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed S3 Manager endpoint alignment with BucketInfo secrets
* Resolved spurious OpenAPI post-processing errors on startup
* **Documentation**
* Added troubleshooting guidance for DependenciesNotReady
* Enhanced installation documentation with Ansible guide
* Added CA rotation operations documentation
* Improved backup and recovery guidance
* Expanded metrics and architecture references
* Reorganized operator-first installation guidance
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fix versions.yaml to use unquoted keys for valid YAML mapping, and
use release-based naming for sysctl DaemonSet to avoid conflicts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthieu <matthieu@hidora.com>
When set, oauth2-proxy skips OIDC discovery and routes all backend calls
(token exchange, JWKS, userinfo, logout) through the internal cluster URL
while keeping browser redirects on the external URL. This avoids external
DNS lookups and TLS overhead for pod-to-pod communication with Keycloak.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Before injecting schedulerName and annotation, the webhook now
checks that the referenced SchedulingClass CR actually exists.
If the CRD is not installed or the CR is missing, injection is
skipped so that pods are not stuck Pending on a non-existent
scheduler.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
When a namespace carries the scheduling.cozystack.io/class label,
the webhook injects schedulerName=cozystack-scheduler and the
scheduler.cozystack.io/scheduling-class annotation into every Pod.
This covers all workloads in the tenant namespace regardless of
whether the operator CRD supports schedulerName natively.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The label is read by the lineage-controller-webhook to inject
schedulerName and scheduling-class annotation into all pods
in the tenant namespace.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Add an API-backed listInput dropdown for the schedulingClass field
in the Tenant creation form. The dropdown lists available
SchedulingClass CRs from cozystack.io/v1alpha1.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Allow administrators to assign a SchedulingClass CR to a tenant.
The schedulingClass is inherited by child tenants and cannot be
overridden once set by a parent.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
This PR adds the changelog for release `v1.0.5`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.5.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved spurious error messages in OpenAPI post-processing for
certain configurations.
* **Documentation**
* Enhanced troubleshooting guides and installation instructions.
* Expanded operational procedures for backups and CA rotation.
* Added custom metrics collection guidance and architecture
documentation.
* Completed comprehensive v1 documentation refresh.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The OpenAPI `PostProcessSpec` callback is invoked for every registered
group-version (apps, core, version, etc.), but the Application schema
cloning logic only applies to `apps.cozystack.io`. When called for other
GVs the base Application schemas are absent, producing a spurious error
on every API server start:
```
ERROR klog Failed to build OpenAPI v3 for group version, "base Application* schemas not found"
```
This PR changes the post-processor (both v2 and v3) to return early
when the base schemas are not found, instead of returning an error.
### Release note
```release-note
[platform] Fix spurious "base Application* schemas not found" error logged on cozystack-api startup
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved error handling for missing OpenAPI schema components. The
system now gracefully continues processing instead of halting when
certain base schemas are unavailable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes s3manager UI deployment to use the actual S3 endpoint from
BucketInfo (COSI) instead of constructing it from the tenant namespace
host.
The deployment was using `s3.<tenant>.<cluster-domain>` while
credentials issued by COSI point to the root-level S3 endpoint. This
mismatch caused "invalid credentials" errors on login even with correct
credentials from the bucket secret.
Falls back to the constructed namespace host on first deploy before
BucketAccess secrets exist.
### Release note
```release-note
[bucket] Fix s3manager endpoint mismatch causing "invalid credentials" errors in login mode
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Deployment configuration now supports per-user endpoint customization.
Endpoints are dynamically retrieved from account-specific settings,
enabling flexible configurations while maintaining backward
compatibility for standard deployments without custom settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR adds version specific ubuntu base images to fix errors when base
image has new deb packages of kubeadm and kubelet installed, but at
runtime it was downgraded by replacing just binaries. Now update by
replacing binaries works as intended - latest patch version of minor
version used.
Core issue was in kubeadm<1.32 expecting conntrack binary in its
preflight checks but it was not found. It happened because kubelet deb
package dropped conntrack dependency since 1.32 (actually it absent in
1.31.14 too).
So now status of supported tenant k8s versions is:
- 1.30 - works because kubelet package provided conntrack, also
conntrack preflight check ignored (see 1.31).
- 1.31 - works because conntrack preflight check ignored (for some
reason kubelet 1.31.14 did't provide conntrack dependency, unlike
1.31.13 did).
- \>=1.32 - works because conntrack preflight check removed from
`kubeadm init` entirely.
Conntrack preflight check ignoring is legit for tenant kubernetes
clusters because until 1.32 it was used in kube-proxy but cozystack k8s
approach doesn't use kube-proxy (replaced with cilium).
Issue with conntrack may be mitigated with only `ignorePreflightErrors`,
but I think proper base image build will help to avoid similar bugs in
future.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Fixed tenant k8s older than 1.32 creation by adding version specific ubuntu base images
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added multi-version Kubernetes support with version-specific container
images.
* Enhanced compatibility with newer Kubernetes releases, including
version 1.31.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When dependencies are not ready the reconciler returned without
requeueing, relying solely on watch events to re-trigger. If a watch
event was missed (controller restart, race condition, dependency already
ready before watch setup), the package would stay stuck in
DependenciesNotReady forever.
Add RequeueAfter: 30s so dependencies are periodically rechecked.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The OpenAPI PostProcessSpec callback is invoked for every group-version
(apps, core, version, etc.), but the Application schema cloning logic
only applies to apps.cozystack.io. When called for other GVs the base
Application schemas are absent, causing a spurious error log on every
API server start.
Return early instead of erroring when the base schemas are not found.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The deployment template was constructing the S3 endpoint from the tenant's
namespace host (e.g. s3.freedom.infra.example.com), while COSI credentials
are issued for the actual SeaweedFS endpoint (e.g. s3.infra.example.com).
This mismatch caused 'invalid credentials' errors when users tried to log
in with valid credentials from the bucket secret.
Now the endpoint is resolved from BucketInfo (same source as credentials),
with a fallback to the constructed namespace host for first-time deploys
before BucketAccess secrets are created.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Hardcode relocateAfterClone=true and relocateAfterRestore=false as
defaults in the CSI driver patch instead of exposing them via
StorageClass/VolumeSnapshotClass parameters.
Remove the extra linstor-snapshots-ephemeral VolumeSnapshotClass (Velero)
and the relocateAfterRestore parameter from the default VolumeSnapshotClass.
This is a temporary measure while upstream linstor-server is deciding on
the interface for the rebalance feature (see LINBIT/linstor-server#487).
Once upstream provides native rebalance support, these hardcoded defaults
will be replaced by the proper upstream mechanism.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Adds the cozystack-scheduler as an optional system package, vendored
from https://github.com/cozystack/cozystack-scheduler. The scheduler
extends the default kube-scheduler with SchedulingClass-aware affinity
plugins, allowing platform operators to define cluster-wide scheduling
constraints via a SchedulingClass CRD. Pods opt in via the
`scheduler.cozystack.io/scheduling-class` annotation.
The package includes:
- Helm chart with RBAC, ConfigMap, Deployment, and CRD
- PackageSource definition for the cozystack package system
- Optional inclusion in the platform system bundle
### Release note
```release-note
[cozystack-scheduler] Add cozystack-scheduler as an optional system
package. The custom scheduler supports SchedulingClass CRDs for
cluster-wide node affinity, pod affinity, and topology spread constraints.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added cozystack-scheduler component as an optional system package.
* Introduced SchedulingClass custom resource for advanced scheduling
configurations.
* Scheduler supports node affinity, pod affinity, pod anti-affinity, and
topology spread constraints.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Updates the Keycloak Operator Helm chart to v1.32.0 and builds a custom
operator image from upstream master
(facbc36)
with the group rename detection patch from PR
https://github.com/epam/edp-keycloak-operator/pull/309 applied on top.
Bumps Keycloak server from 26.0.4 to 26.5.2, which is required by the
new operator client (sends `description` field rejected by older
versions).
Adds SSO session settings (idleTimeout: 86400, maxLifespan: 604800) to
the ClusterKeycloakRealm to match the dashboard client's session
attributes,
as Keycloak 26 enforces realm-level session limits strictly.
Removes `authorizationServicesEnabled` from the dashboard
KeycloakClient,
which is incompatible with Keycloak 26's stricter validation.
### Release note
```release-note
[keycloak-operator] Update the operator to v1.32.0 with group rename fix
(https://github.com/epam/edp-keycloak-operator/pull/309). Bump Keycloak to 26.5.2.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Optional webhook support with cert-manager integration and webhook
service; feature toggles for webhooks and owner refs
* New realm login and session settings, organizations resource/CRD, and
expanded client/user schemas (including ClientRolesV2)
* **Documentation**
* Chart bumped to 1.32.0; README documents new values (clusterDomain,
podLabels, image.registry, securityContext, containerSecurityContext)
* **Security / RBAC**
* RBAC updated to cover organization resources and webhook bindings;
consolidated operator permissions
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Add external-dns as a standalone self-managed application in
`packages/extra/external-dns/`, allowing tenants to deploy and configure
their own DNS management directly from the dashboard.
## Motivation
Tenants need the ability to manage their own DNS domains with their own
provider. Following the [developers
guide](https://github.com/cozystack/website/pull/413), this is
implemented as an extra package (like `ingress` and `seaweedfs`) using
the HelmRelease-based pattern, rather than embedding it in the tenant
chart.
This enables multi-tenant scenarios where:
- Tenant A uses Cloudflare for `domain-a.com`
- Tenant B uses AWS Route53 for `domain-b.com`
- Each tenant deploys and manages external-dns independently from the
dashboard
## Changes
- **New package**: `packages/extra/external-dns/` — standalone
HelmRelease-based application
- **New PackageSource**:
`packages/core/platform/sources/external-dns-application.yaml` —
references `system/external-dns` and `extra/external-dns` components
- **Cleaned tenant chart**: removed the previously embedded
`externalDns` block from `packages/apps/tenant/`
## Features
- Support for 9 DNS providers: cloudflare, aws, azure, google,
digitalocean, linode, ovh, exoscale, godaddy
- Per-provider credential configuration with full JSON schema validation
- Domain filtering via `domainFilters`
- Configurable sync policy (`sync` or `upsert-only`)
- Namespaced operation (`namespaced: true`) for tenant isolation
- Unique `txtOwnerId` per namespace to prevent DNS record conflicts
- Resource sizing via presets or explicit CPU/memory
## Usage Example
Deploy from the dashboard, or via values:
```yaml
# Cloudflare
provider: cloudflare
domainFilters:
- example.com
cloudflare:
apiToken: "your-cloudflare-api-token"
```
```yaml
# AWS Route53
provider: aws
domainFilters:
- example.org
aws:
accessKeyId: "AKIAXXXXXXXX"
secretAccessKey: "your-secret-key"
region: "us-east-1"
```
## Test plan
- [ ] `helm template external-dns packages/extra/external-dns/ --set
provider=cloudflare --set cloudflare.apiToken=test` renders correctly
- [ ] `helm template external-dns packages/extra/external-dns/` fails
(provider required)
- [ ] `helm template wrong-name packages/extra/external-dns/ --set
provider=cloudflare` fails (release name check)
- [ ] Deploy external-dns from tenant dashboard
- [ ] Verify HelmRelease is created in tenant namespace with namespaced
RBAC
- [ ] Create an Ingress and verify DNS record is created
- [ ] Verify no conflict with global external-dns instance
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an External DNS package for automatic DNS record management.
* Support for 9 DNS providers: Cloudflare, AWS, Azure, Google,
DigitalOcean, Linode, OVH, Exoscale, GoDaddy.
* Helm-based deployment with namespaced/system variants and release
configuration options.
* Configurable synchronization policies, domain filtering, provider
credentials, extra args, and resource presets.
* **Documentation**
* New README and schema-driven values documentation for installation and
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Updates the Keycloak Operator Helm chart to v1.32.0 and builds a custom
operator image from upstream master (epam/edp-keycloak-operator@facbc36)
with the group rename detection patch from PR
epam/edp-keycloak-operator#309 applied on top.
Bumps Keycloak server from 26.0.4 to 26.5.2, which is required by the
new operator client (sends `description` field rejected by older versions).
Adds SSO session settings (idleTimeout: 86400, maxLifespan: 604800) to
the ClusterKeycloakRealm to match the dashboard client's session attributes,
as Keycloak 26 enforces realm-level session limits strictly.
Removes `authorizationServicesEnabled` from the dashboard KeycloakClient,
which is incompatible with Keycloak 26's stricter validation.
### Release note
```release-note
[keycloak-operator] Update the operator to v1.32.0 with group rename fix
(epam/edp-keycloak-operator#309). Bump Keycloak to 26.5.2.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Migrates VictoriaLogs from the deprecated single-node `VLogs` CR to
`VLCluster` (cluster mode) with vlinsert/vlselect/vlstorage components
for reliability and horizontal scalability.
**Operator upgrade:**
- Upgrades victoria-metrics-operator from v0.55.0 to v0.68.1 to add
VLCluster CRD support
**VLCluster deployment:**
- Replaces `VLogs` (v1beta1) with `VLCluster` (v1) — 2 replicas per
component, consistent with VMCluster
- Adds VPA for all VLCluster components (vlinsert, vlselect, vlstorage)
- Updates WorkloadMonitors for the three-component architecture
**Endpoint updates:**
- Fluent-bit outputs: `vlogs-generic:9428` → `vlinsert-generic:9481`
- Grafana datasource: `vlogs-{name}:9428` → `vlselect-{name}:9471`
- ExternalName service: `vlogs-generic` → `vlinsert-generic`
**Migration (35 → 36):**
- Adds `helm.sh/resource-policy: keep` annotation to existing VLogs
resources so they are preserved during upgrade
- Users need to verify the new VLCluster is working, then optionally
migrate historical data and manually delete old VLogs resources
### Release note
```release-note
[monitoring] Migrate VictoriaLogs from single-node VLogs to VLCluster (cluster mode). Old VLogs resources are preserved with `helm.sh/resource-policy: keep` annotation. After upgrade, verify the new cluster is working, then optionally migrate historical data and delete old VLogs resources manually.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* CRD upgrade workflow with configurable hooks, Job, and ServiceAccount
support
* Monitoring storage split into vlinsert / vlselect / vlstorage with
corresponding VPAs
* Service traffic distribution option and optional shareProcessNamespace
toggle
* **Updates**
* VictoriaMetrics Operator bumped to v0.68.1
* Fluent Bit and Grafana endpoints/ports updated to new monitoring
targets
* Global extra-labels support for resources
* Migration target advanced to version 36
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The victoria-metrics-operator v0.68.1 renamed VMCluster status field
from .status.clusterStatus to .status.updateStatus.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
This PR adds the changelog for release `v1.1.1`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.1.1.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed MarketplacePanel visibility and disabled state management in
dashboard sidebar
* Fixed External IP address display in dashboard resource tables
* Fixed MAC address preservation during virtual machine migrations
* Resolved deprecated component image issue
* Improved migration handling for missing component dependencies
* Fixed Keycloak health monitoring and application stability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.0.4`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.4.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated v1.0.4 changelog with system, platform, and dashboard
improvements
* Added OIDC self-signed certificates configuration guide
* Documented fixes for health probes, virtual machine migration, and
dashboard features
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds the cozystack-scheduler as an optional system package, vendored from
https://github.com/cozystack/cozystack-scheduler. The scheduler extends
the default kube-scheduler with SchedulingClass-aware affinity plugins,
allowing platform operators to define cluster-wide scheduling constraints
via a SchedulingClass CRD. Pods opt in via the
`scheduler.cozystack.io/scheduling-class` annotation.
The package includes:
- Helm chart with RBAC, ConfigMap, Deployment, and CRD
- PackageSource definition for the cozystack package system
- Optional inclusion in the platform system bundle
### Release note
```release-note
[cozystack-scheduler] Add cozystack-scheduler as an optional system
package. The custom scheduler supports SchedulingClass CRDs for
cluster-wide node affinity, pod affinity, and topology spread constraints.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## Summary
- Splits `dashboards.list` into tenant-facing dashboards and
infrastructure dashboards (`dashboards-infra.list`)
- Infrastructure dashboards (VictoriaMetrics, Flux, Hubble, LINSTOR,
control-plane, etc.) are only rendered for `tenant-root`
- Tenant-facing dashboards (ingress, db, kafka, nats, clickhouse, vm)
remain available to all tenants
## Problem
All tenants currently receive infrastructure dashboards
(VictoriaMetrics, Hubble, LINSTOR, Flux, control-plane, etc.) that are
only relevant to platform operators.
Relates to #2194
## Test plan
- [ ] `helm template` monitoring in `tenant-root` namespace — both lists
rendered
- [ ] `helm template` monitoring in a child tenant namespace — only
`dashboards.list` rendered
- [ ] Verify no dashboard names collide between the two lists
```release-note
Scope infrastructure dashboards to tenant-root only
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Infrastructure dashboards exposed via the monitoring system for
tenant-root deployments.
* Added new ingress dashboards including vhosts and vhost-detail views.
* **Chores**
* Removed a large set of legacy dashboards to streamline the monitoring
surface.
* Reorganized dashboard generation to separate infra-specific dashboards
from standard sets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Adds a `CiliumClusterwideNetworkPolicy` allowing egress from tenant
pods to `virt-handler` in `cozy-kubevirt` namespace on port 8443/TCP
- Conditional on `.Values.monitoring` being enabled
## Problem
Tenant vmagent cannot scrape KubeVirt VM metrics from `virt-handler`
because no network policy allows the traffic.
Relates to #2194
## Test plan
- [ ] `helm template` tenant with `monitoring: true` — virt-handler
policy present
- [ ] `helm template` tenant with `monitoring: false` — virt-handler
policy absent
- [ ] Deploy and verify vmagent can scrape kubevirt_vmi_* metrics
```release-note
Allow tenant egress to virt-handler for VM metrics scraping
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a network egress policy that, when monitoring is enabled, allows
tenant namespaces to reach the virt-handler service on TCP port 8443,
improving connectivity for monitoring-related traffic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
After migrating VictoriaLogs from VLogs to VLCluster, the e2e test
still waited for the old vlogs/generic resource which no longer exists.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## Summary
- Sidebar menu was showing all resources regardless of their
MarketplacePanel `hidden` state
- Fetch MarketplacePanels during sidebar reconciliation and skip
resources where `hidden=true`
- Hiding a resource from the marketplace now also removes it from the
sidebar navigation
## Test plan
- [ ] Set `hidden: true` on a MarketplacePanel (e.g. qdrant)
- [ ] Trigger controller reconciliation
- [ ] Verify the resource is removed from the sidebar menu
- [ ] Set `hidden: false` and verify the resource reappears in the
sidebar
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Sidebar can now hide resources based on MarketplacePanel configuration
parsed from panel definitions.
* Hidden resources are filtered early when assembling sidebar
categories, preventing them from contributing to menu items.
* Listing failures are non-fatal: if configuration fetch fails, no
hiding is applied and the dashboard remains functional.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The `pull-requests.yaml` workflow used `paths-ignore` at the trigger
level to skip runs for docs-only changes. This prevented the entire
workflow from triggering, so the required "E2E Tests" check was never
created — blocking merge for non-admin users.
This PR replaces trigger-level `paths-ignore` with a `detect-changes`
job using `dorny/paths-filter@v3`. The workflow now always triggers (so
all checks are reported to GitHub), but `build` and downstream jobs are
skipped when only `docs/` files change.
| PR type | build | resolve_assets | e2e |
| --- | --- | --- | --- |
| Code PR | runs | skipped | runs |
| Release PR | skipped (label) | runs | runs |
| Docs-only PR | skipped | skipped | skipped |
### Release note
```release-note
[ci] Fix required E2E check blocking merge of docs-only pull requests
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* CI pipeline optimized to skip builds when only documentation changes
occur.
* Added a checks step that detects whether code changed and gates the
build accordingly.
* Build now runs only if code changes are present and the PR is not
marked as a release, reducing unnecessary build runs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR updates the cilium system package to the version 1.19.1
### Release note
```release-note
[cilium] Update cilium system package to the version 1.19.1
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Upgraded to version 1.19.1 with enhanced security and observability
capabilities
* Added standalone DNS proxy support for improved DNS handling
* Enhanced multi-cluster service mesh support with automatic CoreDNS
configuration
* Expanded cloud provider integrations with improved node resource
management
* Added ztunnel encryption support
* **Improvements**
* Enhanced TLS certificate management and auto-generation
* Extended observability and profiling options
* Improved endpoint and service handling with updated resource
management
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Fix "Disabling features from menu and marketplace is not working" by
preserving user-set `disabled` and `hidden` values during controller
reconciliation
- The controller was hardcoding `disabled=false` and `hidden=false` on
every reconcile loop, overwriting any changes made through the dashboard
UI
## Test plan
- [ ] Disable a service from the dashboard marketplace panel
- [ ] Verify the service stays disabled after controller reconciliation
- [ ] Hide a service from the dashboard menu
- [ ] Verify the service stays hidden after controller reconciliation
- [ ] Create a new ApplicationDefinition and verify its MarketplacePanel
defaults to disabled=false, hidden=false
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed an issue where user-configured "disabled" and "hidden" settings
in the marketplace panel could be reset during updates. These
preferences are now preserved when the panel is created or updated, and
the system avoids applying unnecessary configuration changes when values
haven't actually changed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Add CiliumClusterwideNetworkPolicy for vmagent egress to virt-handler
- Restrict endpointSelector to vmagent pods only via app.kubernetes.io/name label
Signed-off-by: Mattia Eleuteri <mattia.eleuteri@hidora.io>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
## Summary
- Fix External IPs page showing empty rows in the dashboard by
correcting EnrichedTable properties in the `external-ips` factory
- Replace `clusterNamePartOfUrl` with `cluster` and change `pathToItems`
from array format to dot-path string to match convention used by all
other EnrichedTable instances
## Test plan
- [ ] Open Administration → External IPs in dashboard for a tenant with
LoadBalancer services
- [ ] Verify table columns (Name, ClusterIP, LoadbalancerIP, Created)
are rendered
- [ ] Verify service data is displayed correctly in the rows
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed configuration handling for the external-ips dashboard tab to
ensure cluster names display correctly and service items are
consistently listed, improving stability and data presentation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
During the virtual-machine → vm-instance migration (script 29), VM MAC
addresses
are not preserved. Kube-OVN reads MAC exclusively from the pod
annotation
`ovn.kubernetes.io/mac_address`, not from the IP resource
`spec.macAddress`.
Without the annotation, migrated VMs get a new random MAC, breaking
OS-level
network config that matches by MAC (e.g. netplan).
This adds a Helm `lookup` for the Kube-OVN IP resource in the
vm-instance chart
template. When the resource exists, its `macAddress` and `ipAddress` are
automatically injected as pod annotations. This approach is reliable
across
HelmRelease reconciliations — unlike postRenderers, the annotations
cannot be
accidentally lost.
Fixes#2166
### Release note
```release-note
[platform] Fix VM MAC address not preserved during virtual-machine → vm-instance migration, causing network loss on VMs with MAC-based netplan config
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* VM templates now automatically populate network annotations (MAC and
IP) from kubeovn IP records when available. This streamlines VM network
setup on deployment, reduces manual annotation steps, and lowers risk of
misconfiguration by ensuring VMs receive the correct address and MAC
information from associated network records.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Implement bilateral VPC peering using Kube-OVN's native vpcPeerings
mechanism. Each VPC can declare peers by specifying the remote VPC name
and tenant namespace. Peering is only activated by Kube-OVN when both
sides declare each other, ensuring mutual consent.
Key features:
- Deterministic remote VPC ID resolution via sha256 hash
- Auto-allocated link-local peering IPs (169.254.0.0/16) derived from
sorted pair hash, eliminating manual IP coordination
- Static routes support for fine-grained inter-VPC routing
- ConfigMap enrichment with peer discovery info
- Schema validation enforcing tenant- namespace prefix pattern
Signed-off-by: Mattia Eleuteri <mattia.eleuteri@hidora.io>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Enable relocate-after-restore on the default linstor-snapshots class
so that PVCs restored from snapshots get replicas relocated to optimal
nodes.
Add a separate linstor-snapshots-velero class with the Velero
annotation and without relocation, so Velero's temporary data mover
PVCs are not relocated unnecessarily.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Use CSI clone instead of host-assisted copy for CDI DataVolume
cloning. This leverages LINSTOR's native rd-clone mechanism which
is significantly faster than pod-based data copying, and works
together with the new relocateAfterClone parameter.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Move snapshot restore relocation parameter from StorageClass to
VolumeSnapshotClass to avoid unwanted relocation during Velero
data mover backup. Change relocateAfterClone default to false.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Remove merged RWX validation patch and add new patch that includes:
- Randomized node selection for snapshot restore
- Relocate replicas to optimal nodes after clone and snapshot restore
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Kube-OVN reads MAC address exclusively from the pod annotation
ovn.kubernetes.io/mac_address, not from the IP resource spec.macAddress.
Without pod-level annotations, migrated VMs receive a new random MAC,
breaking OS-level network config that matches by MAC (e.g. netplan).
Add a Helm lookup for the Kube-OVN IP resource in the vm-instance chart
template. When the IP resource exists, its macAddress and ipAddress are
automatically injected as pod annotations. This removes the need for
fragile Flux postRenderers on the HelmRelease — the chart itself handles
MAC/IP preservation based on actual cluster state.
Remove the postRenderers approach from migration 29 since the chart now
handles this natively.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The pull-requests workflow used paths-ignore at the trigger level, which
prevented the entire workflow from running on docs-only PRs. This meant
the required "E2E Tests" check was never created, blocking merge for
non-admin users.
Replace trigger-level paths-ignore with a detect-changes job using
dorny/paths-filter. The workflow now always triggers (so checks are
always reported), but build and downstream jobs are skipped when only
docs files change.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The external-ips factory used incorrect EnrichedTable properties causing
empty rows in the dashboard. Replace `clusterNamePartOfUrl` with
`cluster` and change `pathToItems` from array to dot-path string format
to match the convention used by all other working EnrichedTable instances.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The controller was hardcoding disabled=false and hidden=false on every
reconciliation, overwriting any user changes made through the dashboard
UI. Move spec building inside the CreateOrUpdate mutate function to read
and preserve current disabled/hidden values from the existing resource.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The sidebar was generated independently from MarketplacePanels, always
showing all resources regardless of their hidden state. Fetch
MarketplacePanels during sidebar reconciliation and skip resources
where hidden=true, so hiding a resource from the marketplace also
removes it from the sidebar navigation.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## Summary
- Replace deprecated `gcr.io/kubebuilder/kube-rbac-proxy:v0.16.0` with
`quay.io/brancz/kube-rbac-proxy:v0.18.1` in the vendored etcd-operator
chart
- The GCR-hosted image became unavailable after March 18, 2025
Fixes#2172#488
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated proxy component to v0.18.1 with configuration changes for
improved stability and compatibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The gcr.io/kubebuilder/kube-rbac-proxy image is no longer available
since GCR was deprecated. Replace it with quay.io/brancz/kube-rbac-proxy
from the original upstream author.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Migration 34 fails with `error: the server doesn't have a resource
type "rabbitmqs"` when `rabbitmqs.apps.cozystack.io` CRD does not exist
on the cluster
- This happens when RabbitMQ was never installed — the CRD is not
present, `kubectl get` fails, and `set -euo pipefail` terminates the
migration job
- Added a CRD existence check before listing resources; if CRD is
absent, the migration stamps the version and exits cleanly
## Test plan
- [ ] Deploy a cluster without RabbitMQ installed and run migration 34 —
should skip gracefully
- [ ] Deploy a cluster with RabbitMQ instances without `spec.version`
set — should patch them to `v3.13`
- [ ] Deploy a cluster with RabbitMQ instances already having
`spec.version` — should skip patching
## Summary
- Fix Keycloak crashloop caused by misconfigured liveness/readiness
probes
- Add `KC_HEALTH_ENABLED=true` to activate health endpoints on
management port
- Switch probes from application port 8080 (`/`, `/realms/master`) to
management port 9000 (`/health/live`, `/health/ready`)
## Problem
Keycloak 26.x redirects all HTTP requests on port 8080 to the configured
`KC_HOSTNAME` (HTTPS). Since kubelet does not follow redirects, probes
fail with:
```
Probe terminated redirects, Response body:
```
After consecutive failures, kubelet kills the container → restart →
crashloop.
Additionally, `KC_HEALTH_ENABLED` was not set, so the dedicated health
endpoints on the management port (9000) returned 404 even though the
management interface was active (via `KC_METRICS_ENABLED=true`).
## Changes
- `packages/system/keycloak/templates/sts.yaml`:
- Add `KC_HEALTH_ENABLED=true` env var to activate `/health/live` and
`/health/ready`
- Expose management port 9000 in container ports
- Liveness probe: `GET /health/live` on port 9000 (was `GET /` on 8080)
- Readiness probe: `GET /health/ready` on port 9000 (was `GET
/realms/master` on 8080)
- Increase failure thresholds for better startup tolerance
## Test plan
- [x] Verified `/health/live` returns `{"status":"UP"}` (HTTP 200) on
port 9000
- [x] Verified `/health/ready` returns
`{"status":"UP","checks":[{"name":"Keycloak database connections async
health check","status":"UP"}]}` (HTTP 200)
- [x] Confirmed 0 restarts after 10+ minutes
- [x] Confirmed no more `ProbeWarning` or `Killing` events
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[docs] Fixed docs for managed apps
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Documentation
* Updated FoundationDB README title to "Managed FoundationDB Service"
* Improved Harbor README text formatting for consistency
* Corrected spelling and terminology errors in MariaDB README
* Enhanced MariaDB recovery documentation with additional command
example
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace deprecated single-node VLogs CR with VLCluster (cluster mode)
for reliability and horizontal scalability.
Changes:
- Replace VLogs (v1beta1) with VLCluster (v1) using vlinsert/vlselect/vlstorage
- Update fluent-bit outputs to vlinsert-generic:9481
- Update Grafana datasource to vlselect:9471
- Update ExternalName service from vlogs-generic to vlinsert-generic
- Add VPA for all VLCluster components
- Update WorkloadMonitors for three-component architecture
- Add migration 35 to preserve old VLogs resources with keep annotation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Upgrade from v0.55.0 to v0.68.1 to add VLCluster CRD support,
which is required for migrating VictoriaLogs from single-node
to cluster mode.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Migration 34 fails when rabbitmqs.apps.cozystack.io CRD does not exist,
which happens when RabbitMQ was never installed on the cluster. Add a
check for CRD presence before attempting to list resources.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
This PR adds the changelog for release `v1.1.0`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.1.0.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Published v1.1.0 changelog documenting major features: managed secrets
service, tiered storage pools, per-user bucket credentials with S3 UI
updates, RabbitMQ version selection, and monitoring dashboards
* Included breaking changes and upgrade notes for v1.1.0
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.0.3`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.3.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed migration script to correctly apply configuration prefixes
during v0.41 to v1.0 upgrade.
* **Documentation**
* Added white labeling guide covering branding customization and SVG
handling.
* Updated backup and recovery documentation with improved operator and
tenant workflow guidance and administration resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Use a startupProbe to defer liveness/readiness checks until Keycloak
has fully started, instead of relying on initialDelaySeconds. This is
more robust for applications with variable startup times.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Keycloak 26.x exposes dedicated health endpoints on the management
port (9000) via /health/live and /health/ready. The previous probes
used GET / on port 8080 which redirects to the configured KC_HOSTNAME
(HTTPS), causing kubelet to fail the probe with "Probe terminated
redirects" and eventually kill the pod in a crashloop.
Changes:
- Add KC_HEALTH_ENABLED=true to activate health endpoints
- Expose management port 9000 in container ports
- Switch liveness probe to /health/live on port 9000
- Switch readiness probe to /health/ready on port 9000
- Increase failure thresholds for more tolerance during startup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
## What this PR does
Combines and unifies COSI enhancements across seaweedfs and bucket
charts:
**SeaweedFS (extra + system charts):**
- Rename storage pool BucketClass suffix from `-worm` to `-lock`
- Rename parameter `disk` to `diskType` for consistency with COSI driver
- Reduce default object lock retention from 36500 to 365 days
- Add `-lock` BucketClass (COMPLIANCE mode, 365 days) for client and
system topologies
- Add `-readonly` BucketAccessClass with explicit `accessPolicy` for all
topologies
- Add explicit `accessPolicy: readwrite` on default BucketAccessClass
- Update pool name validation to reject `-lock` suffix (was `-worm`)
**Bucket app:**
- Add `locking` parameter: provisions from `-lock` BucketClass
- Add `storagePool` parameter: selects pool-specific BucketClass
- Replace hardcoded BucketAccess with `users` map — each entry creates a
BucketAccess with optional `readonly` flag
- Update dashboard RBAC to dynamically list user credential secrets
- Update ApplicationDefinition schema with new properties
**Breaking change:** empty `users: {}` (default) produces zero
BucketAccess resources. Existing buckets that relied on the implicit
default BucketAccess will need to define users explicitly.
### Release note
```release-note
[apps] Add locking, storagePool, and users configuration to bucket app; rename COSI BucketClass suffix from -worm to -lock
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Bucket locking with a shorter retention option, storage-pool
selectable bucket classes, and per-user access (per-user BucketAccess
and readonly controls)
* S3 Manager login mode: user login/logout, per-session credentials, and
new login UI
* **Behavior Changes**
* Credential handling changed to per-user secrets/label selection;
previously generated secrets removed; Ingress basic auth annotations
removed
* **Documentation**
* Added parameters: locking, storagePool, users (including per-user
readonly)
* **Updates**
* Updated COSI driver and S3 manager images
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Set explicit MTU 1350 for Cilium in KubeVirt-based tenant Kubernetes
clusters to prevent packet drops caused by VXLAN encapsulation overhead
## Problem
Cilium's MTU auto-detection does not account for VXLAN overhead when
running inside KubeVirt VMs. The VM network interface inherits MTU 1400
from the parent cluster's OVN/Geneve overlay (1500 - 100 Geneve
overhead). Cilium detects this MTU and applies it to all tunnel
interfaces without subtracting the 50-byte VXLAN encapsulation overhead.
This results in:
- Large packets (> 1350 bytes) being silently dropped when crossing
VXLAN tunnels between nodes
- Intermittent connectivity issues for services in tenant clusters (TLS
handshakes, HTTP responses with data)
- HTTP 499 errors and timeouts observed under load
## Fix
Explicitly set `MTU: 1350` (1400 - 50 VXLAN overhead) in the default
Cilium values for tenant clusters. This value can still be overridden
via `addons.cilium.valuesOverride` if needed.
## Test plan
- [ ] Deploy a tenant Kubernetes cluster and verify Cilium interfaces
use MTU 1350
- [ ] Verify large packet connectivity from pods inside the tenant
cluster
Add mongodb/mongodb-overview and mongodb/mongodb-inmemory entries
to the monitoring dashboards list so GrafanaDashboard CRDs are
generated and dashboards are served by the grafana-dashboards
HTTP service.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
cozytest.sh executes .bats files as plain shell functions where bats
builtins like `run` are not available. Use `!` negation to assert that
readonly user upload fails.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The mc client requires --insecure on each command when connecting to
SeaweedFS S3 with self-signed certificates via port-forward.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Update bucket E2E test to match the new per-user access model:
- Create bucket with admin (readwrite) and viewer (readonly) users
- Test that readwrite user can upload, list, and download objects
- Test that readonly user can list and download but cannot upload
- Use per-user BucketAccess and credential secret names
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
v0.1.2 ignores accessPolicy parameter from BucketAccessClass,
granting readwrite access to all users regardless of the
readonly flag. v0.3.0 includes support for readonly bucket
access, Object Lock, and improved error handling.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Rebuild s3manager with auth.go login page support and push
to 999669/s3manager registry for testing.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Remove nginx basic auth and credential secret injection from the
bucket Helm chart. s3manager now always starts in login mode and
handles authentication via its own login page with encrypted
session cookies. This eliminates the dependency on the -credentials
and -ui-auth secrets for the UI layer.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
deployment.yaml: use s3._namespace.host for ENDPOINT instead of
secret ref, inject ACCESS_KEY_ID/SECRET_ACCESS_KEY only when users
exist. Without users, s3manager starts in login mode.
ingress.yaml: nginx basic auth annotations only when users exist.
Without users, s3manager handles authentication via its login form.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
When a bucket has no users configured, s3manager previously crashed
due to missing ACCESS_KEY_ID/SECRET_ACCESS_KEY env vars. This adds
a login mode where users enter their S3 credentials via a web form.
New Go code (via cozystack.patch):
- auth.go: session-based auth middleware, login/logout handlers,
per-request S3 client from encrypted cookie session
- login.html.tmpl: Materialize CSS login form
- main.go: LoginMode toggle, conditional route setup
- Dependency: gorilla/sessions for AES-256 encrypted cookies
Dockerfile: add go mod tidy step for new dependency resolution.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Show only per-user credential secrets in the dashboard instead of
both the internal UI secret and per-user secrets.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Fixed migrate-to-version-1.0.sh script to properly convert packages names.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated migration tooling to improve package configuration handling
during version upgrades.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The default BucketAccess was removed in favor of per-user access.
Update secret.yaml to look up the first user's COSI secret instead
of the non-existent default one, with nil-check for race conditions.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Cilium's MTU auto-detection does not account for VXLAN overhead when
running inside KubeVirt VMs. The VM interface inherits MTU 1400 from
the parent OVN/Geneve overlay, and Cilium sets all interfaces
(cilium_vxlan, lxc*, cilium_host/net) to 1400 without subtracting
the 50-byte VXLAN encapsulation overhead.
This causes intermittent packet drops for large packets (TLS
handshakes, HTTP responses with data), resulting in timeouts and
499 errors for services running in tenant clusters.
Set MTU to 1350 (1400 - 50 VXLAN overhead) explicitly in the default
Cilium values for tenant Kubernetes clusters.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Remove the yq strip of properties from Makefile that was clearing
the schema, and run make generate to sync all generated files.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Create labeled secrets in the -system chart using lookup to copy
credentials from COSI-created secrets. The ApplicationDefinition
matchLabels selector exposes them in the dashboard.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add a catch-all include selector so that COSI-created user credential
secrets (dynamically named per user) are visible in the dashboard.
The lineage webhook already verifies ownership via the graph walk
(Secret -> BucketAccess -> HelmRelease -> Bucket), so an empty selector
safely matches only secrets belonging to this application.
This is needed because COSI sidecar creates secrets without custom
labels, making the matchLabels pattern (used by rabbitmq) inapplicable.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The seaweedfs-cosi-driver v0.3.0 expects the parameter key 'disk',
not 'diskType'. Restore the correct key to match the driver's
paramDisk constant.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add COSI resources for object locking and read-only access to both
client topology and system chart:
- BucketClass with -lock suffix (COMPLIANCE mode, 365 days retention)
- BucketAccessClass with -readonly suffix
- Explicit accessPolicy: readwrite on default BucketAccessClass
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- fixed rbac for backup controllers
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated backup controller permissions to focus on core backup
operations.
* Expanded backup strategy controller permissions to support enhanced
backup and restore capabilities, including Velero integration and status
management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.0.2`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.2.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Published v1.0.2 release notes.
* **Bug Fixes**
* Fixed migration script to ensure all upgrade steps execute.
* Improved dashboard functionality for field clearing and secret
copying.
* Restored sidebar navigation on namespace-level pages.
* Updated proxy configurations for enhanced TLS handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Replaces the plain text input for `storageClass` fields with an
API-backed dropdown listing available StorageClasses from the cluster.
Follows the same pattern as the `instanceType` dropdown for VMInstance.
Affected applications:
- **Top-level `spec.storageClass`**: ClickHouse, Harbor, HTTPCache,
Kubernetes, MariaDB, MongoDB, NATS, OpenBAO, Postgres, Qdrant,
RabbitMQ, Redis, VMDisk
- **Nested `spec.storage.storageClass`**: FoundationDB
- **Nested `spec.kafka.storageClass` / `spec.zookeeper.storageClass`**:
Kafka
### Release note
```release-note
[dashboard] storageClass fields in stateful app forms now render as a
dropdown populated with available StorageClasses from the cluster,
instead of a free-text input.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Storage class selection dropdowns now available in configuration forms
for multiple database, messaging, and storage services.
* **Tests**
* Added comprehensive test coverage for storage class configuration
handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Restores `stock-instance-api-form`, `stock-instance-api-table`,
`stock-instance-builtin-form`, and `stock-instance-builtin-table`
sidebar
resources that were removed in #2106, and adds them back to the orphan
cleanup allowlist.
PR #2106 removed these sidebars to fix broken URLs on the main page
before
namespace selection (`default//api-table/...`). However,
`stock-instance-*`
sidebars are required by the frontend for namespace-level
api-table/api-form
pages. Without them and with `CUSTOMIZATION_SIDEBAR_FALLBACK_ID=""`, the
frontend cannot find a sidebar for pages like Backup Plans and renders
an
empty page where no interaction is possible.
The broken-URL bug is already fully fixed by
`CUSTOMIZATION_SIDEBAR_FALLBACK_ID=""`
in `web.yaml`. Re-adding `stock-instance-*` does not reintroduce it,
since
these sidebars are only shown when the user is on a namespace-level page
where the `{namespace}` placeholder is filled.
### Release note
```release-note
[dashboard] fix empty page on Backup Plans and other namespace-level api-table pages
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added four new dashboard sidebar resources for stock instances: API
form, API table, built-in form, and built-in table views. These enable
expanded dashboard customization options for managing stock instance
configurations and data.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Updates the openapi-k8s-toolkit integration in the dashboard to fix two
UX issues:
**1. Allow clearing the instanceType field in VMInstance form**
When `instanceType` has a default value, clearing the field in the form
UI would
silently revert to the default, making it impossible to explicitly send
an empty
value. This blocked use of custom KubeVirt resources without a named
instance type.
Adds `allowEmpty: true` to the instanceType listInput field so the BFF
preserves
an explicit empty value. Also introduces a generic `persistType` prop
(`'str' | 'number' | 'arr' | 'obj'`) to the listInput component, so the
allow-empty behaviour works correctly for any field type, not just
strings.
Updates openapi-k8s-toolkit to release/1.4.0 (d6b9e4ad), which already
includes
the FormListInput layout refactor — the previous
formlistinput-value-binding.diff
patch is no longer needed.
Upstream PR:
https://github.com/PRO-Robotech/openapi-k8s-toolkit/pull/340
**2. Preserve newlines when copying secrets with CMD+C**
Native `input[type=text]` strips newlines on copy. Adds an `onCopy`
handler to
the SecretBase64Plain component that intercepts the copy event and
writes the full
decoded value (including newlines) to the clipboard.
Upstream PR:
https://github.com/PRO-Robotech/openapi-k8s-toolkit/pull/339
### Release note
```release-note
[dashboard] Fix clearing instanceType in VMInstance form: explicit empty value
is now correctly sent to the API instead of falling back to the schema default.
Fix CMD+C copying of secrets stripping newlines.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Dropdown fields now support configuration to allow empty selections
* Enhanced empty value handling for form fields across multiple data
types (string, number, array, object)
* **Bug Fixes**
* Fixed secret field copy functionality to preserve plain-text format
when visible
* **Chores**
* Updated base image dependencies for dashboard build
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
PR #2106 removed stock-instance-* sidebar resources to fix broken URLs
on the main page before namespace selection. However, these sidebars are
required for rendering namespace-level pages (api-table, api-form, etc.)
such as the Backup Plans page.
Without stock-instance-api-table, the frontend cannot find the sidebar
for namespace-scoped api-table pages and renders an empty page instead.
The original bug (broken URLs with empty namespace placeholder) is already
fixed by CUSTOMIZATION_SIDEBAR_FALLBACK_ID="" in web.yaml, so re-adding
stock-instance-* sidebars does not reintroduce it.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Add onCopy handler to SecretBase64Plain inputs to intercept native browser
copy events and explicitly write the full decoded text (including newlines)
to the clipboard. Without this, input[type=text] strips newlines on copy.
Upstream PR: https://github.com/PRO-Robotech/openapi-k8s-toolkit/pull/339
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Replace deprecated `KC_PROXY=edge` with `KC_PROXY_HEADERS=xforwarded`
and `KC_HTTP_ENABLED=true` in the Keycloak StatefulSet template.
`KC_PROXY` was removed in Keycloak 26.x, causing "Non-secure context
detected" warnings and broken cookie handling when running behind a
reverse proxy with TLS termination.
### Release note
```release-note
[system] Fix Keycloak proxy headers configuration for compatibility with Keycloak 26.x
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated system configuration to improve proxy header handling and
enable direct HTTP support for enhanced compatibility with reverse proxy
environments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Update openapi-k8s-toolkit commit to d6b9e4ad (release/1.4.0) which
includes the FormListInput layout refactor, making formlistinput-value-binding.diff
obsolete.
Set allowEmpty: true on the VMInstance instanceType field so users can
explicitly clear the selection and override the default instance type.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Update openapi-k8s-toolkit to release/1.4.0 (d6b9e4ad). The previous
value-binding layout refactor is already included upstream, so drop the
formlistinput-value-binding.diff patch.
Add formlistinput-allow-empty.diff patch which introduces two props to
the listInput component:
- allowEmpty: when set, auto-persists the field so BFF sends an empty
value instead of falling back to the schema default
- persistType: controls the type of empty value ('str' | 'number' | 'arr'
| 'obj'), allowing the feature to work correctly for any field type
Set allowEmpty: true on the VMInstance instanceType field so users can
explicitly clear the selection and override the default instance type.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Fixed off-by-one error where the first required migration was always skipped.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected migration range handling so upgrade steps run for the
intended version window, preventing skipped or duplicated migrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Disabled private key rotation in every CA cert in cozystack system packages to prevent trustchain problems when CA cert reissued
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Disabled private-key rotation (set rotationPolicy: Never) for CA/root
certificates used by multiple system components (ingress-nginx, linstor,
linstor-scheduler, seaweedfs, victoria-metrics-operator,
kubeovn-webhook, lineage-controller-webhook, cozystack-api, etcd,
linstor API/internal, seaweedfs).
* Added patch application steps to relevant update workflows to ensure
the certificate template changes are applied during chart/update
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[rabbitmq] Added version selection to newly created RabbitMQ instances.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Configurable RabbitMQ major.minor version selector (v4.2, v4.1, v4.0,
v3.13), default v4.2; chart validates selection and uses it to pick the
runtime image.
* **Chores**
* Default RabbitMQ image updated to 4.2.4.
* Added an automated version-update helper and a Makefile target to
refresh available versions and regenerate manifests.
* **Migration**
* Migration added to backfill the version field on existing RabbitMQ
resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace plain text input with an API-backed listInput dropdown for
storageClass fields across all applications that expose them.
The dropdown fetches available StorageClasses from the cluster via
/api/clusters/{cluster}/k8s/apis/storage.k8s.io/v1/storageclasses,
following the same pattern as the instanceType dropdown for VMInstance.
Top-level spec.storageClass: ClickHouse, Harbor, HTTPCache, Kubernetes,
MariaDB, MongoDB, NATS, OpenBAO, Postgres, Qdrant, RabbitMQ, Redis, VMDisk.
Nested paths: FoundationDB (spec.storage.storageClass),
Kafka (spec.kafka.storageClass and spec.zookeeper.storageClass).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Adds a check in the migration script to detect and suspend the
`cozy-proxy`
HelmRelease if it has `releaseName: cozystack`, which conflicts with the
installer
release and causes cozystack-operator deletion during upgrade from v0.41
to v1.0.
### Release note
```release-note
[platform] Fix migration script to handle cozy-proxy releaseName conflict during v0.41→v1.0 upgrade.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced the version 1.0 migration process with automatic conflict
detection and interactive guidance, prompting users to resolve issues
during the upgrade for a smoother migration experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
In v0.41.x, cozy-proxy HelmRelease was configured with
releaseName: cozystack, which collides with the installer helm release.
If not suspended before upgrade, the cozy-proxy HR reconciles and
overwrites the installer release, deleting cozystack-operator.
Add a check in the migration script that detects this conflict and
suspends the cozy-proxy HelmRelease before proceeding.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
KC_PROXY=edge was deprecated and removed in Keycloak 26.x, causing
"Non-secure context detected" warnings and broken cookie handling
behind reverse proxy. Replace with KC_PROXY_HEADERS=xforwarded and
KC_HTTP_ENABLED=true.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
This PR adds the changelog for release `v1.0.1`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.1.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Published v1.0.1 release notes with platform, installer, and dashboard
bug fixes
* Updated website documentation: renamed "Bundles" to "Variants," added
new variant options, and updated cross-references
* Added upgrade protection instructions for system components prior to
upgrade
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add FlowSchema `cozy-dashboard-exempt` to exempt the dashboard BFF
service account (`incloud-web-web`) from API Priority and Fairness
throttling
- BFF falls under the default `service-accounts` FlowSchema →
`workload-low` priority level, which causes 429 responses under load
## Test plan
- [ ] Deploy to a cluster with dashboard enabled
- [ ] Verify FlowSchema is created: `kubectl get flowschema
cozy-dashboard-exempt`
- [ ] Verify BFF no longer receives 429 errors under load
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added a new Kubernetes FlowSchema configuration for system resource
access management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds `helm.sh/resource-policy: keep` annotation to the `cozy-system`
Namespace resource
in the installer helm chart. This prevents Helm from deleting the
namespace (and all
HelmReleases within it) when the installer release is removed.
Also updates the v1.0 migration script to annotate the `cozy-system`
namespace and
`cozystack-version` ConfigMap with the same policy before generating the
Package resource.
### Release note
```release-note
[platform] Add helm.sh/resource-policy=keep annotation to cozy-system Namespace in installer chart to prevent namespace deletion on HelmRelease removal. Update migration script to protect namespace and cozystack-version ConfigMap before migration.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced migration process with an interactive step to safeguard
critical resources during system upgrades.
* Added resource protection mechanisms to prevent unintended removal
during Helm operations.
* Improved control flow in the upgrade script with explicit user
confirmation prompts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add helm.sh/resource-policy=keep annotation to the cozy-system Namespace
in the installer helm chart. This prevents Helm from deleting the
namespace when the HelmRelease is removed, which would otherwise destroy
all other HelmReleases within it.
Update the migration script to annotate the cozy-system namespace and
cozystack-version ConfigMap with helm.sh/resource-policy=keep before
generating the Package resource.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The dashboard BFF service account (incloud-web-web) falls under the
default "service-accounts" FlowSchema which maps to the "workload-low"
priority level. Under load, this causes API Priority and Fairness to
return 429 (Too Many Requests) responses to the BFF, resulting in 500
errors for dashboard users.
Add a FlowSchema that maps the BFF service account to the "exempt"
priority level to prevent APF throttling of dashboard API requests.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[ci] Added more debug information to ci tests
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced error handling and diagnostic output in development testing
infrastructure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
- Add `volume.pools` (Simple topology) and `volume.zones[name].pools`
(MultiZone topology) for creating separate Volume StatefulSets per disk
type (SSD/HDD/NVMe)
- Add `nodeSelector`, `storageClass`, and `dataCenter` overrides for
zones in MultiZone topology
- Create per-pool `BucketClass` and `BucketAccessClass` COSI resources
(including WORM and readonly variants)
- Bump seaweedfs-cosi-driver to v0.3.0 (adds `disk` parameter support in
BucketClass)
- Add `volume.diskType` field to tag default volume servers with a disk
type
### How It Works
#### Simple Topology
Each storage pool in `volume.pools` creates an additional Volume
StatefulSet alongside the default one. All pods (default + pool) may run
on the same nodes. SeaweedFS distinguishes storage via the
`-disk=<type>` flag on volume servers.
```yaml
volume:
replicas: 2
size: 10Gi
diskType: ""
pools:
ssd:
diskType: ssd
size: 50Gi
storageClass: local-nvme
```
#### MultiZone Topology
Pools are defined per-zone in `volume.zones[name].pools`. A StatefulSet
is created for each **zone × pool** combination (e.g., `us-east-ssd`,
`us-west-ssd`), inheriting nodeSelector and dataCenter from its parent
zone.
```yaml
volume:
replicas: 2
size: 10Gi
zones:
us-east:
replicas: 2
size: 100Gi
# nodeSelector defaults to: topology.kubernetes.io/zone: us-east
pools:
ssd:
diskType: ssd
size: 50Gi
us-west:
replicas: 3
```
In Simple topology, `volume.pools` is used. In MultiZone,
`volume.zones[name].pools` is used — `volume.pools` is explicitly
blocked to prevent BucketClasses without backing StatefulSets.
### Zone Overrides (MultiZone)
Zones now support:
- `nodeSelector` — YAML string, defaults to
`topology.kubernetes.io/zone: <zoneName>`
- `storageClass` — defaults to `volume.storageClass`
- `dataCenter` — SeaweedFS data center name, defaults to zone name
### COSI Resources
Each unique pool name generates 4 cluster-scoped COSI resources:
- `<namespace>-<pool>` BucketClass (Delete policy, `disk: <type>`)
- `<namespace>-<pool>-worm` BucketClass (Retain policy, object lock)
- `<namespace>-<pool>` BucketAccessClass (readwrite)
- `<namespace>-<pool>-readonly` BucketAccessClass (readonly)
### Validation
- Pool names must be valid DNS labels (no dots)
- Pool names must not end with `-worm` or `-readonly` (reserved COSI
suffixes)
- `diskType` is required and must be lowercase alphanumeric
- Pool `diskType` must differ from `volume.diskType`
- Pool name + zone name composed names must not collide with existing
zone names
- `volume.pools` is blocked in Client and MultiZone topologies
- All replicas have `minimum: 1` in JSON schema
### Inheritance Chain
| Field | Pool fallback (Simple) | Pool fallback (MultiZone) |
| ------------ | -------------------------------- |
---------------------------------------- |
| `replicas` | pool → volume | pool → zone → volume |
| `size` | pool → volume | pool → zone → volume |
| `storageClass` | pool → volume | pool → zone → volume |
| `resources` | pool → volume | pool → volume (zone resources inherited)
|
### Backward Compatibility
- Default `volume.pools: {}` produces identical output to current chart
- Default `volume.diskType: ""` adds no extra flags
- Existing default BucketClass remains unchanged
- No migration needed — pools create new StatefulSets
### Test plan
- [x] `helm template` with empty pools — output identical to current
- [x] `helm template` with Simple + volume.pools — additional volume
StatefulSets, BucketClasses, WorkloadMonitors
- [x] `helm template` with MultiZone + zone.pools — zone × pool
cross-product StatefulSets
- [x] `helm template` with `volume.diskType: hdd` — extraArgs includes
`-disk=hdd`
- [x] `helm template` with Client + volume.pools — fails with validation
error
- [x] `helm template` with MultiZone + volume.pools — fails with
validation error
- [x] `helm template` with reserved pool name suffix — fails with
validation
- [x] Deploy to test cluster and verify volume servers register with
correct disk types
### Release note
```release-note
[seaweedfs] add storage pools support for tiered storage with per-pool COSI resources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for multiple storage pools with configurable disk types
and resource allocation.
* Introduced per-pool bucket and access classes for storage management.
* Added zone-aware pool configurations for multi-zone deployments.
* Enhanced topology-driven resource monitoring and allocation.
* **Documentation**
* Updated service documentation with expanded configuration parameters
and improved formatting.
* **Chores**
* Updated container image to latest version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add a readonly `BucketAccessClass` to the seaweedfs COSI chart with
`accessPolicy: "readonly"` parameter
- Each bucket now automatically creates two sets of S3 credentials:
readWrite (existing, for UI) and readonly
- Update dashboard RBAC and ApplicationDefinition to expose the readonly
credentials secret
## Test plan
- [ ] Verify seaweedfs chart templates render both `BucketAccessClass`
resources (readWrite and readonly)
- [ ] Verify bucket app templates render `BucketClaim` + 2
`BucketAccess` (readWrite + readonly)
- [ ] Deploy a bucket and confirm both credential secrets are created by
COSI driver
- [ ] Confirm readonly credentials can only read/list objects, not
write/delete
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced read-only bucket access capabilities. Users can now
configure read-only permissions for bucket storage resources,
complementing existing access control options. New read-only access
classes and configurations provide enhanced security controls and
finer-grained permission management. This enables improved data
protection while maintaining flexibility for various access requirements
across applications and storage infrastructure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cert-manager] Updated cert-manager to v1.19.3
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Global nodeSelector and hostUsers (pod user-namespace isolation) added
* New/updated CRDs for cert-manager resources (Certificate,
CertificateRequest, Order, etc.)
* **Documentation**
* Revised chart docs and installation guidance; added deprecation/notice
about private-key rotation
* Removed legacy CRD README and schema files from the CRD package
(documentation consolidated)
* **Chores**
* Upgraded cert-manager to v1.19.3
* Moved CRDs into a dedicated CRD package; ServiceMonitor targetPort
default renamed to "http-metrics"
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Prevent cozystack-version configmap from deletion
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated deployment resource configuration to improve system
reliability by ensuring critical components are properly retained and
protected during system operations and maintenance activities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds OpenBAO (open-source Vault fork) as a new managed PaaS application
in Cozystack.
**Structure follows existing app patterns (qdrant, nats):**
- System chart with vendored upstream `openbao/openbao` (chart v0.25.3,
appVersion v2.5.0)
- App chart with standalone/HA mode switching based on replicas count
- TLS via cert-manager self-signed certificates per instance
- ApplicationDefinition, PackageSource, PaaS bundle entry
- E2E test with init/unseal workflow
**Key design decisions:**
- `replicas: 1` → standalone mode with file storage; `replicas > 1` → HA
with Raft integrated storage and retry_join with TLS peer verification
- TLS enabled by default — each instance gets a self-signed Certificate
with DNS SANs covering services and pod addresses
- `disable_mlock = true` in HCL config since default security context
drops IPC_LOCK capability
- Injector and CSI provider disabled (cluster-scoped components, not
safe per-tenant)
- No auto-init/unseal — OpenBAO requires manual initialization by design
- E2E test performs full lifecycle: deploy, wait for certificate + API,
init, unseal, verify readiness, cleanup
### Release note
```release-note
[apps] Add OpenBAO as a managed secrets management service with standalone and HA Raft modes, TLS enabled by default
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added OpenBAO managed secrets management service with
high-availability and standalone deployment options
* Integrated monitoring and dashboards for operational visibility
* Enabled configurable external access and web UI
* Added automated snapshot backup capability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- added dropdown for selection backupClasses in Plan/BackupJob creation form
```
Redesign storage pools architecture:
- Move storagePools map from top-level into volume.pools (Simple topology)
and volume.zones[name].pools (MultiZone topology)
- Add nodeSelector, storageClass, dataCenter overrides for zones
- Add reserved suffix validation (-worm, -readonly) for pool names
- Block volume.pools usage in MultiZone (must use zone.pools instead)
- Use ternary/hasKey pattern for all optional replicas to handle 0 correctly
- Fix nodeSelector rendering for multiline values using indent
- Use disk: parameter (not diskType:) for COSI driver v0.3.0 BucketClass
- Bump seaweedfs-cosi-driver tag to v0.3.0
- Add minimum: 1 constraint for volume/zone/pool replicas in schema
- Regenerate README, CRD, and openAPISchema
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
- Remove dots from pool name regex (K8s resources don't allow dots)
- Add zone×pool name collision validation for MultiZone topology
- Use conditional storageClass rendering to omit empty values
- Fix README resourcesPreset default value
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
- Document MultiZone fallback chain for pool replicas and size
- Move `-volume` WorkloadMonitor reference inside Simple topology block in dashboard-resourcemap.yaml (it is only created for Simple)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Make storageClass optional in storagePools — pools inherit from
volume.storageClass when not explicitly set. Add full COSI resource set
per storage pool: BucketClass, BucketClass-worm (Retain + object lock),
BucketAccessClass readwrite, and BucketAccessClass readonly.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Add optional storagePools configuration that creates separate Volume
StatefulSets per disk type (SSD/HDD/NVMe), enabling tiered storage
within a single SeaweedFS tenant. Each pool gets its own BucketClass
and BucketAccessClass to prepare infrastructure for COSI driver
integration.
Supported in both Simple and MultiZone topologies:
- Simple: one StatefulSet per pool
- MultiZone: one StatefulSet per zone×pool combination
Also adds volume.diskType field for tagging default volume servers.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
This PR adds the changelog for release `v1.0.0`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.0.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added comprehensive v1.0.0 release notes documenting feature
highlights, improvements, and fixes across all platform components
* Included breaking changes and step-by-step upgrade guide for v0.x to
v1.0.0 migration
* Listed 33 incremental migrations and contributor credits
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR prepares the release `v1.0.0`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container image versions from pre-release candidate tags to
stable v1.0.0 releases across core, system, and extra packages.
* Updated all associated container image digests to reflect the stable
release builds.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
In the current version, the sidebar incorrectly shows namespace-scoped
menu items on cluster-level pages (before a tenant is selected).
Clicking these items produces broken URLs with double `//` (e.g.
`default//api-table/backups.cozystack.io/...`) because the `{namespace}`
placeholder resolves to an empty string.
This PR fixes the issue by:
- Removing stock-instance-* sidebar resources that were populated with
the same namespace-scoped menu as stock-project-* sidebars
- Clearing the `CUSTOMIZATION_SIDEBAR_FALLBACK_ID` env var so the
frontend renders no sidebar when no matching sidebar resource exists
- Removing stock-instance-* from the expected resource set so orphan
cleanup removes stale instances on upgrade
Screenshot after changes
<img width="2560" height="1327" alt="dashboard screenshot with no tenant
selected"
src="https://github.com/user-attachments/assets/e0d795f7-55e9-471b-99b8-593b6fc145d8"
/>
### Test plan
- [x] On cluster list page (no tenant selected), sidebar is empty
- [x] After selecting a tenant, sidebar shows full menu
- [x] No double `//` in sidebar URLs
- [x] Existing tests pass: `go test ./internal/controller/dashboard/...`
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] fix: hide sidebar on cluster-level pages when no tenant selected
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Streamlined sidebar resource management by reducing the number of
static sidebar configurations generated by the system.
* Removed sidebar fallback behavior, resulting in simplified sidebar
customization defaults.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove stock-instance-* sidebars that were populated with namespace-scoped
menu items, causing the sidebar to incorrectly appear on cluster-level pages.
Clear the sidebar fallback ID so the frontend gracefully renders no sidebar
when no matching sidebar resource exists.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Hide Ingresses, Services, and Secrets tabs in the dashboard when the
ApplicationDefinition has no resource selectors (Include/Exclude) for
the corresponding type. Previously all tabs were always hardcoded as
visible.
### Release note
```release-note
[dashboard] Hide Ingresses/Services/Secrets tabs when ApplicationDefinition has no resource selectors defined
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Dashboard tab visibility refined: Workloads tab remains always
visible; Ingresses, Services, and Secrets tabs now appear only when
corresponding resource selectors are configured, reducing clutter and
showing relevant tabs based on configured resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add a readonly BucketAccessClass to the seaweedfs COSI chart and a
second fixed BucketAccess per bucket so each bucket automatically
gets both readWrite and readonly S3 credentials.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
This PR adds the changelog for release `v1.0.0-rc.2`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.0-rc.2.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added release notes for v1.0.0-rc.2, documenting features,
improvements, bug fixes, and contributor acknowledgments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR prepares the release `v1.0.0-rc.2`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated numerous container images to v1.0.0-rc.2 across platform
services, controllers, dashboard components, migration/e2e tooling,
storage and networking components.
* Refreshed several image digests (including kubevirt CSI, s3manager and
Linstor components) and other image references.
* Updated default tenant branding text used by the dashboard.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
… in sidebar
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- fixed dashboard backupjobs creation form
- fixed dashboard sidebar backup category id
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Updates**
* Updated dashboard backups menu structure for improved organization
* Enhanced backup job management interface with new form configuration
including Name, Namespace, Plan Name, Application details, and Backup
Class selection
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Add `ingress.host` field to cozy-keycloak values, allowing users to
override the default
`keycloak.<root-host>` Ingress hostname. The custom hostname is applied
to both the Ingress
resource and the `KC_HOSTNAME` environment variable in the StatefulSet.
When left empty,
behavior is unchanged (backward compatible).
### Release note
```release-note
[system] Add `ingress.host` option to cozy-keycloak for custom Ingress hostname override
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Keycloak ingress hostname is now configurable and automatically
defaults to "keycloak.<root-host>" when not explicitly specified.
* **Chores**
* Refactored hostname configuration across Keycloak templates to use
dynamic variable resolution instead of hard-coded values for improved
consistency and flexibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
After application renames (ferretdb→mongodb, mysql→mariadb,
virtual-machine→vm-disk+vm-instance),
the system-level `-rd` HelmReleases in `cozy-system` were left orphaned.
They reference
ExternalArtifacts that no longer exist, causing persistent
reconciliation failures:
- `ferretdb-rd` → no longer exists (replaced by `mongodb-rd`)
- `mysql-rd` → no longer exists (replaced by `mariadb-rd`)
- `virtual-machine-rd` → no longer exists (replaced by `vm-disk-rd` +
`vm-instance-rd`)
Migrations 28 and 29 handled user-facing HelmReleases but missed the
system-level `-rd` ones.
**Changes:**
- Add cleanup of `mysql-rd` to migration 28
- Add cleanup of `virtual-machine-rd` to migration 29
- Add migration 33 as a safety net for users who already passed
migrations 28/29
- Bump `targetVersion` from 33 to 34
### Release note
```release-note
[platform] Fix orphaned ferretdb-rd, mysql-rd, and virtual-machine-rd HelmReleases
that persist after upgrading, referencing non-existent ExternalArtifacts.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved cleanup procedures in platform migrations to properly remove
orphaned system resources. This enhancement helps maintain system
stability and prevents potential resource conflicts during platform
updates.
* **Chores**
* Updated migration version target to latest.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add a safety-net migration for users who already passed migrations 28/29
and still have orphaned -rd HelmReleases in cozy-system:
- ferretdb-rd (replaced by mongodb-rd, never had a dedicated migration)
- mysql-rd (migration 28 only handled user HRs)
- virtual-machine-rd (migration 29 only handled user HRs)
These HRs reference ExternalArtifacts that no longer exist after the
application renames (ferretdb→mongodb, mysql→mariadb,
virtual-machine→vm-disk+vm-instance), causing persistent errors.
Bump targetVersion from 33 to 34.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migrations 28 (mysql→mariadb) and 29 (virtual-machine→vm-disk+vm-instance)
only handled user-facing HelmReleases but left the system-level -rd
HelmReleases (mysql-rd, virtual-machine-rd) orphaned in cozy-system.
These HRs reference ExternalArtifacts that no longer exist, causing
persistent reconciliation failures.
Add cleanup steps to delete the orphaned -rd HRs and their Helm secrets.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add ingress.host field to values.yaml for cozy-keycloak. When set,
it overrides the default "keycloak.<root-host>" hostname in both the
Ingress resource and the KC_HOSTNAME environment variable. When left
empty, behavior is unchanged for backward compatibility.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fixed dashboard sidebar links to Backups and External IPs
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Standardized cluster identifier usage across dashboard menu links,
administration links, and API request paths for consistent link
generation.
* **Bug Fixes**
* Resolved issues causing incorrect or inconsistent link targets and
ensured backup-class options load correctly in the UI.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes multiple upgrade issues discovered during v0.41.1 → v1.0 upgrade
testing.
**Migration 26 (monitoring → monitoring-system):**
- Use `cozystack.io/ui=true` label with
`--field-selector=metadata.name=monitoring` instead of
`apps.cozystack.io/application.kind=Monitoring` to find monitoring
HelmReleases — the old label is guaranteed to exist on v0.41.1 clusters,
while the new one depends on migration 22 having run
- Add `delete_helm_secrets` function with fallback deletion by secret
name pattern and post-deletion verification
**Migrations 28 and 29 (mysql→mariadb, virtual-machine split):**
- Wrap `grep` in pipes with `{ ... || true; }` to prevent `pipefail`
exit when grep filters out all lines
- Fix reconcile annotation in migration 29 to use RFC3339 timestamp
format instead of Unix epoch
- Remove protection-webhook handling from migration 29 — it is an
external component and should not be managed by cozystack migrations
**Migration 27 (piraeus CRD ownership):**
- Skip CRDs that don't exist instead of failing the entire migration
- Add name-pattern fallback for helm secret deletion
**etcd HelmRelease:**
- Increase timeout from 10m to 30m to accommodate TLS cert rotation hook
**migrate-to-version-1.0.sh:**
- Add missing ConfigMap → Package field mappings: `bundle-disable`,
`bundle-enable`, `expose-ingress`, `expose-services`
- Remove redundant bundle enabled flags — the variant already determines
them via its values file
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Package generation now supports disabled/enabled package lists,
ingress name, and exposed services for customized publishing.
* **Bug Fixes**
* More robust secret cleanup with fallback deletions and post-deletion
verification.
* Guarded pipelines to avoid failures when no resources match.
* Reconciliation timestamps now use RFC3339 UTC.
* Suspension failures are no longer silently suppressed.
* **Chores**
* Increased etcd upgrade timeout; improved namespace discovery,
relabeling behavior, and user-facing messaging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add @sircthulhu to the global CODEOWNERS list
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated repository maintenance configuration.
---
**Note:** This release contains only internal repository updates with no
user-facing changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The protection-webhook is not part of the cozystack platform and should
not be managed by the migration script. Old services are now deleted
directly instead of being batched through the webhook disable/enable cycle.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migration 26 was using apps.cozystack.io/application.kind=Monitoring label
which is added by migration 22 and may not be present on v0.41.1 clusters.
Switch to cozystack.io/ui=true (guaranteed on old HRs) with field-selector
for exact name match.
Also remove redundant bundle enabled flags from migrate-to-version-1.0.sh
since the variant already determines them via its values file.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add ConfigMap fields that were not converted to Package values:
- bundle-disable → bundles.disabledPackages
- bundle-enable → bundles.enabledPackages
- expose-ingress → publishing.ingressName
- expose-services → publishing.exposedServices
Remove incorrect bundles.system.type field that is not part of the
Package values schema.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The post-upgrade hook deletes TLS certificates and etcd pods to trigger
cert-manager regeneration. With 3 replicas and startup probes allowing
up to 25 minutes per pod, the previous 10m timeout was insufficient.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migration 27 failed with set -e when Piraeus CRDs did not exist on
clusters without linstor. Add existence check before annotating CRDs.
Also add name-pattern fallback for helm secret deletion, consistent
with migration 26.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
grep returns exit code 1 when no lines match. With set -euo pipefail,
this kills the script when all secrets are helm-release secrets or when
no matching resources exist. Wrap grep calls with { ... || true; }.
Also fix reconcile annotation in migration 29 to use RFC3339 timestamp
format instead of Unix epoch, which Flux v2 expects.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migration 26 silently skipped namespace processing when kubectl
queries failed, leaving helm release secrets intact. This caused
helm to diff old vs new chart manifests during upgrade, deleting
VLogs/CNPG resources and their PVCs.
- Remove silent error suppression (2>/dev/null || true) from
namespace discovery and HR suspend commands
- Add fallback secret deletion by name pattern when label selector
does not match
- Add verification that all helm release secrets are deleted
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Only show Ingresses/Services/Secrets tabs when Include selectors are
defined. Exclude selectors alone don't make resources visible as tenant
resources, so tabs would be empty.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
PR #2075 introduced `_cluster.cluster-domain` references in
monitoring-agents `values.yaml` for FQDN resolution in tenant clusters.
This broke the fluent-bit subchart because `_cluster` values are not
accessible from the Helm subchart context — only `global` values are
shared with subcharts.
This PR replaces `_cluster` references with a new `global.clusterDomain`
variable:
- Empty by default (management cluster uses short DNS names like
`service.namespace.svc`)
- Set to the management cluster domain (e.g. `cozy.local`) for tenant
clusters, enabling FQDN resolution for cross-cluster service discovery
Fixes#2084
### Release note
```release-note
[system] Fix monitoring-agents installation failure caused by inaccessible _cluster values in fluent-bit subchart context. Introduce global.clusterDomain for proper FQDN resolution in tenant workload clusters.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Monitoring agent configuration updated to support configurable cluster
domain names for greater flexibility.
* Remote write and log-forwarding endpoints adjusted to align with
cluster domain handling, improving compatibility when deploying across
different cluster DNS setups.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Show these tabs only when the ApplicationDefinition has non-empty
Include or Exclude resource selectors for the corresponding type.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
PR #2075 added _cluster.cluster-domain references to monitoring-agents
values.yaml for FQDN resolution. This broke fluent-bit because it is
a subchart where _cluster values are not accessible (only global
values are shared with subcharts), causing "index of untyped nil".
Revert values.yaml to short DNS names (.svc) for the management
cluster where they resolve locally, and add FQDN with cluster-domain
suffix in the tenant kubernetes HelmRelease where _cluster values are
available and cross-cluster DNS resolution is needed.
Closes#2084
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
This PR adds the changelog for release `v1.0.0-rc.1`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.0-rc.1.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added comprehensive release notes for v1.0.0-rc.1, documenting new
features, improvements, bug fixes, breaking changes, and upgrade
guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Add 9 missing patch release changelogs:
- **v0.40.x**: v0.40.5, v0.40.6, v0.40.7
- **v0.41.x**: v0.41.4, v0.41.5, v0.41.6, v0.41.7, v0.41.8, v0.41.9
All PR authors were verified via `gh pr view` (not from commit authors).
Format matches existing changelog files in the repository.
### Release note
```release-note
[docs] Add missing changelogs for v0.40.5-v0.40.7 and v0.41.4-v0.41.9
```
This PR prepares the release `v1.0.0-rc.1`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Upgraded container image versions from v1.0.0-beta.6 to v1.0.0-rc.1
across the entire platform. Updates span multiple critical components
including the core operator, platform migrations, dashboard and API
services, backup and storage controllers, networking components, and
various additional system infrastructure services. All updated images
include new SHA256 digests for ensuring integrity and verification.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add 9 missing patch release changelogs covering changes from
January to February 2026. All PR authors verified via GitHub CLI.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Updates kubevirt to v1.6.3 and CDI to v1.64.0.
Please note that VMs would be live-migrated as a part of the update.
### Release note
```release-note
Updated kubevirt to v1.6.3 and CDI to v1.64.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* VM synchronization support with synchronization ports and address
reporting
* Cluster profiler and synchronization controller developer options
* **Updates**
* CDI operator bumped to v1.64.0; filesystem overhead default increased
* KubeVirt operator bumped to v1.6.3
* Added liveness/readiness probes and health/metrics ports
* Expanded operator tolerations for control-plane/master nodes
* Expanded operator permissions for synchronization and webhook-related
resources
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
KubeVirt v1.6.x has a known issue (#15989) where the guest-console-log
init container blocks virt-launcher pods from starting. The container
runs virt-tail as a long-running sidecar but fails to properly function
as a Kubernetes native sidecar, causing all VM pods to get stuck in
PodInitializing state indefinitely.
Disable serial console logging globally via the KubeVirt CR to prevent
the problematic init container from being created.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Reverts #2079 — the Kamaji update was merged prematurely while marked
with `do-not-merge` label. The upstream commit (`309d9889`) has not been
released as an edge tag yet.
### Release note
```release-note
NONE
```
Upstream PR clastix/kamaji#1087 (refactor!: datastore conditions) removed
the startup datastore existence check that our disable-datastore-check.diff
patch was working around. Update to the merge commit and drop the now
redundant patch.
Remaining patches:
- fix-kubelet-config-compat.diff (pending upstream PR #1084)
- increase-startup-probe-threshold.diff (no upstream fix)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Previously `_cluster.clusterissuer` controlled the ACME solver type
using
values `http01` / `cloudflare`, and every ingress template hardcoded
`cert-manager.io/cluster-issuer: letsencrypt-prod` with no way to
override it.
This PR adds new parameters in platform chart:
- `publishing.certificates.solver` (default `http01`)
- `publishing.certificates.issuerName` (default: `letsencrypt-prod`)
instead of single parameter before
- `publishing.certificates.issuerType`
Previous `certificates.issuerType` was renamed to `certificates.solver`;
Also its possible value
`cloudflare` was renamed to `dns01` to use standard ACME terminology.
New `certificates.issuerName` (default: `letsencrypt-prod`) — propagated
as
`_cluster.issuer-name` to all packages via `cozystack-values` then its
value appears in
`cert-manager.io/cluster-issuer` annotation across 8 templates of
ingresses in system applications.
`publishing.certificates.solver` can be set empty to clearly support
`selfsigned-cluster-issuer`,
or have any value, but it can be a bit confusing.
Operators can now point ingresses at any ClusterIssuer (custom ACME,
self-signed, internal CA) by setting `certificates.issuerName` without
touching individual package templates.
## Breaking changes
| What changed | Before | After |
|---|---|---|
| Solver key | `certificates.issuerType` | `certificates.solver` |
| Cloudflare solver value | `issuerType: cloudflare` | `solver: dns01` |
This changes handled by migration when upgrading cozystack from v1
or by `migration-to-v1.0.sh` script (also checked by migration later)
No actions from user needed.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Added publishing.certificates.solver (http01/dns01) and
publishing.certificates.issuerName fields to allow configuring ACME challenge
type and ClusterIssuer per installation, replacing the old implicit issuerType field
[platform] Migration script and upgrade hook (migration 32) convert old
clusterissuer/issuerType fields to the new solver/issuerName fields
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Migrated certificate issuer configuration from legacy `issuerType`
field to new `solver` and `issuerName` fields system-wide.
* Automated migration script converts existing configurations, mapping
legacy values (cloudflare, http01) to new format.
* Updated all certificate-related templates to use new configurable
solver and issuer settings with sensible defaults.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Monitoring agents (vmagent, fluent-bit) in tenant workload clusters
failed to deliver metrics and logs because service addresses used short
DNS names (e.g. `vminsert-longterm.tenant-root.svc`) without the cluster
domain suffix. Tenant CoreDNS could not resolve these names across
cluster boundaries.
This PR appends the configured cluster domain from
`_cluster.cluster-domain` to all vmagent remoteWrite URLs and fluent-bit
output hosts, with a fallback to `cluster.local` when not set.
### Release note
```release-note
[system] Fix monitoring-agents endpoints to use FQDN with configured cluster domain, resolving metrics and logs delivery failures in tenant workload clusters.
```
## What this PR does
Updates Kubernetes version support to match current release landscape
and Talos 1.12 compatibility:
- Update Kamaji from `edge-25.4.1` to `edge-26.2.4` (adds K8s 1.35
support)
- Update Kubernetes version matrix: v1.30, v1.31, v1.32, v1.33, v1.34,
v1.35
- Drop EOL versions v1.28 and v1.29
- Remove merged-upstream patch (992.diff — label preservation fix)
- Regenerate disable-datastore-check.diff for new Kamaji version
Changes:
- Default Kubernetes version is now v1.35
- E2E tests will validate v1.35 (latest) and v1.34 (previous)
- Patch versions updated to latest available (v1.35.0, v1.34.4, v1.33.8,
v1.32.12, v1.31.14, v1.30.14)
### Release note
```release-note
[kubernetes] Update supported Kubernetes versions to v1.30-v1.35
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Kamaji CRDs Helm chart with DataStore and KubeconfigGenerator
resources, plus deployment templates and configurable
kubeconfigGenerator settings
* DataStore now supports multiple backends (etcd, MySQL, PostgreSQL,
NATS) with TLS/auth validations and status tracking (observedGeneration)
* **Chores**
* Bumped default Kubernetes version from v1.33 to v1.35 (added v1.34;
removed v1.28–v1.29)
* Updated charts, packaging metadata, README/docs and helm
ignore/Makefile entries; updated builder base image and chart
dependencies
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Kamaji edge-26.2.4 is compiled against Kubernetes 1.35 libraries.
SetDefaults_KubeletConfiguration() from 1.35 injects two fields
gated by feature gates that are not enabled in earlier versions:
- crashLoopBackOff.maxContainerRestartPeriod (KubeletCrashLoopBackOffMax)
- imagePullCredentialsVerificationPolicy (KubeletEnsureSecretPulledImages)
Kubelets < 1.35 reject these fields during configuration validation,
causing worker nodes to fail to join the tenant cluster.
Add a Go patch that clears these fields from the kubelet-config
ConfigMap when the target Kubernetes version is below 1.35.
See: https://github.com/clastix/kamaji/issues/1062
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Fixed cozy:tenant:admin:base ClusterRole to deny deletion of tenant ResourceQuotas for the tenant admin and superadmin
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Removed resource quota management permissions from tenant admin role
to reduce unnecessary administrative access.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Kill stale port-forward processes before starting a new one;
on retries, the previous attempt's port-forward still holds the
port, causing all kubectl commands to get "connection refused"
- Use -ge 2 instead of -eq 2 for node count check; MachineHealthCheck
may create a 3rd VM, leading to 3 nodes joining the tenant cluster
which would never satisfy the exact equality check
- Increase node join timeout from 5m to 8m; QEMU VMs with v1.34 need
more time to boot and join when running after kubernetes-latest
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When running kubernetes-latest and kubernetes-previous E2E tests
simultaneously, worker VMs compete for resources in the sandbox
environment. 3 minutes was insufficient for nodes to boot and
join the tenant cluster under load. Increase to 5 minutes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Kamaji auto-derives the konnectivity proxy image tag as v0.{minor}.0
from the Kubernetes version. For K8s v1.35, this produces v0.35.0,
but the kas-network-proxy/proxy-server:v0.35.0 image does not exist
in registry.k8s.io yet, causing ImagePullBackOff on new TCP pods.
Add konnectivity-versions.yaml mapping to explicitly override the
konnectivity version when the auto-derived tag is unavailable.
For v1.35, pin to v0.34.0 (latest available).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Go 1.26 was released on 2026-02-10 and is fully compatible with
Kamaji edge-26.2.4 (which requires go 1.25.0 in go.mod).
Verified by local build.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Add API-backed dropdown selects for VMInstance form: `instanceType`
fetches from `VirtualMachineClusterInstancetype` resources,
`disks[].name` fetches from `VMDisk`
resources in the same namespace
- Default value for `instanceType` is read dynamically from the
ApplicationDefinition's OpenAPI schema
- Fix a bug in the upstream `FormListInput` component where Ant Design's
`Form.Item` couldn't pass `value`/`onChange` to `Select` because of an
intermediate `Flex` wrapper
Details
The dashboard renders forms from OpenAPI schemas using the
openapi-k8s-toolkit library. To turn a plain text field into an
API-backed dropdown, the CustomFormsOverride resource's schema field is
used with type: "listInput" and customProps containing the API endpoint
URL.
Controller changes (customformsoverride.go):
- applyListInputOverrides() — injects listInput schema overrides for
VMInstance kind
- parseOpenAPIProperties() — extracts top-level properties from OpenAPI
schema to read defaults
- ensureSchemaPath() / ensureArrayItemProps() — helpers to build nested
schema structures safely
Frontend patch (formlistinput-value-binding.diff):
- Moves `<Flex>` outside `<ResetedFormItem>` so `<Select>` becomes the
direct child — required for Ant Design's `Form.Item` to inject
`value`/`onChange` via `React.cloneElement`
Instance type example:
<img width="1143" height="1091" alt="instance type example"
src="https://github.com/user-attachments/assets/6c401916-b531-4da6-ae27-ca54e6b0bd04"
/>
VMDisks example:
<img width="875" height="323" alt="vmdisks example"
src="https://github.com/user-attachments/assets/18918115-c08a-40bb-b932-536419d6f2c1"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vminstance] Add dropdowns for `instanceType` and `disk[].name` using `VirtualMachineClusterInstancetype` cluster resources and `VMDisk` from current namespace
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Custom form overrides now support auto-population of dropdown field
defaults from API schemas for enhanced user workflows.
* Improved layout and visual alignment of form list input controls for
better usability and responsiveness.
* **Tests**
* Added comprehensive test coverage for custom form override
functionality and API schema integration, including edge cases and
default value handling scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
CAPI Kamaji provider v0.15.0 is incompatible with Kamaji edge-26.2.4
due to the new dataStoreUsername field with XValidation rule. The
provider's CreateOrUpdate drops the field (not in its Go types),
triggering "unsetting the dataStoreUsername is not supported" error.
This results in KamajiControlPlane staying INITIALIZED=false even
though the underlying TenantControlPlane reaches Ready.
v0.16.0 includes support for DataStoreUsername (PR #243 in v0.15.4)
and updated Kamaji types compatible with edge-26.2.4.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds the `skip-adjust-when-device-inaccessible.diff` patch (upstream
[LINBIT/linstor-server#477](https://github.com/LINBIT/linstor-server/pull/477))
which:
- Skips DRBD adjust and .res file regeneration when child layer devices
are inaccessible (fixes encrypted resource deletion)
- Skips lsblk when device path doesn't physically exist yet (fixes race
condition after drbdadm adjust)
- Only checks child devices when disk access is actually needed (allows
network reconnect from StandAlone)
- Fixes missing `setExists(true)` in `LuksLayer` — the root cause of all
new DRBD+LUKS+STORAGE resources failing with "not defined in your
config"
### Release note
```release-note
[linstor] Fix DRBD+LUKS+STORAGE resource creation: all new encrypted volumes were failing because the DRBD .res file was never written due to a missing exists flag in the LUKS layer
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved handling of inaccessible storage devices by adding pre-checks
before performing operations
* Operations are now skipped when underlying storage devices are
unavailable, preventing unnecessary failures
* Enhanced error recovery during storage adjustments when devices are
temporarily inaccessible
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes CRDs from the cozy-installer Helm chart `crds/` directory and
delegates
CRD lifecycle management entirely to the operator via `--install-crds`
flag.
The operator already applies embedded CRDs via server-side apply on
every startup,
making the Helm `crds/` directory redundant. Helm only installs CRDs on
initial
`helm install` and never updates or deletes them on upgrade/uninstall,
which causes
CRDs to become stale over time.
Changes:
- Remove `packages/core/installer/crds/` (Packages and PackageSources
CRDs)
- Remove `templates/packagesource.yaml` Helm template — PackageSource is
now
created by the operator at startup using server-side apply
- Add `installPlatformPackageSource()` function to operator with SSA
- Move variant validation from deleted template to
`cozystack-operator.yaml`
- Simplify `update-codegen.sh` to use single CRD destination
- Update Makefile to source CRDs from `internal/crdinstall/manifests/`
- Update E2E tests to wait for operator-managed CRDs and PackageSource
- Add unit tests for PackageSource creation
### Release note
```release-note
[installer] CRDs are no longer shipped in the Helm chart crds/ directory. The operator now manages CRD lifecycle via --install-crds flag, ensuring CRDs stay up to date on every startup.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Operator now installs CRDs on startup and auto-creates the platform
PackageSource.
* **Improvements**
* Installer waits for CRDs to be established and for the platform
PackageSource to be present before proceeding.
* Deployment variant selection now has stricter validation to prevent
invalid choices.
* **Chores**
* Cleaned up legacy CRD templates and updated build/install scripting
paths.
* **Tests**
* Added comprehensive tests covering platform PackageSource installation
and URL/ref parsing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
kubectl wait fails immediately with NotFound if the CRD does not exist
yet. The operator creates CRDs asynchronously on startup, so wrap the
wait in a retry loop that tolerates the initial absence.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Monitoring agents in tenant workload clusters failed to deliver metrics
and logs because service addresses used short DNS names without the
cluster domain suffix. Tenant CoreDNS could not resolve these names
across cluster boundaries.
Append the configured cluster domain from _cluster.cluster-domain to
all vmagent remoteWrite URLs and fluent-bit output hosts, falling back
to cluster.local when not set.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
When platformSourceURL is empty no Flux source resource is created, so
creating a PackageSource that references it would leave a dangling
SourceRef in a permanent error state. Guard the creation block on
platformSourceURL != "".
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace repetitive Variant struct literals with a loop over variant
data, making it easier to add new variants and reducing copy-paste
errors.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Kamaji edge-26.2.4 requires Go >= 1.25.0, update base image accordingly.
Also remove unused "context" import from disable-datastore-check patch,
since removing the CheckExists call was the only usage of that package.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Change default --platform-source-name from "cozystack-packages" to
"cozystack-platform" to match the value passed by the Helm template.
Gate PackageSource creation on --install-crds flag: when the operator
does not manage CRDs, the PackageSource CRD may not exist yet, so
skip creation and let an external process handle it.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Explicitly check error from parsePlatformSourceURL instead of relying on
the implicit guarantee that installPlatformSourceResource already checked
it. This prevents latent bugs if startup order is ever restructured.
Add wait for platform PackageSource existence in E2E test before creating
Package resource, preventing flaky failures when operator startup is slow.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Switch installPlatformPackageSource to server-side apply (SSA) with
field manager, matching the pattern used in crdinstall. SSA is idempotent
and preserves metadata fields managed by other controllers.
Create PackageSource unconditionally (not only when platformSourceURL is
set), matching the previous Helm template behavior where PackageSource
was always created regardless of source URL configuration.
Use a dedicated context with its own 2-minute timeout for PackageSource
creation, separate from the platform source resource installation.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add explicit CRD wait (kubectl wait crd --for=condition=Established) in
E2E test before creating Package resources, preventing race condition
between operator CRD installation and resource creation.
Add unit tests for installPlatformPackageSource covering create, update,
and GitRepository sourceRef kind scenarios.
Document that hardcoded variant list is an intentional design choice
matching the previous Helm template behavior.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Check error from parsePlatformSourceURL instead of discarding it, preventing
silent fallback to OCIRepository on invalid URLs.
Move variant validation from deleted packagesource.yaml to
cozystack-operator.yaml template so invalid cozystackOperator.variant
values still fail at helm template/install time.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace the Helm hook approach with programmatic PackageSource creation
in the operator startup sequence. Helm hooks are unsuitable for persistent
resources like PackageSource because before-hook-creation policy causes
cascade deletion of owned ArtifactGenerators during upgrades.
The operator now creates the platform PackageSource after installing CRDs
and the Flux source resource, using the same create-or-update pattern as
installPlatformSourceResource(). The sourceRef.kind is derived from the
platform source URL (OCIRepository for oci://, GitRepository for git).
Also fix stale comment in e2e test referencing deleted crds/ directory.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Remove the crds/ directory from the cozy-installer Helm chart. The operator
already installs embedded CRDs via server-side apply on every startup with
the --install-crds=true flag, making the Helm crds/ directory redundant.
Convert templates/packagesource.yaml to a Helm post-install/post-upgrade
hook so it is applied after the operator has started and installed CRDs.
Update codegen to write CRDs only to internal/crdinstall/manifests/ (single
source of truth) and update the Makefile to source build assets from there.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update Kubernetes version matrix to match Talos 1.12 support range:
- Add v1.35.0 (latest) and v1.34.4
- Update existing patch versions (v1.33.8, v1.32.12)
- Drop EOL versions v1.28 and v1.29
- Set default version to v1.35
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update Kamaji from edge-25.4.1 to edge-26.2.4 which adds support for
Kubernetes 1.35 (KubeadmVersion bumped from v1.33.0 to v1.35.0).
- Update Dockerfile VERSION to edge-26.2.4
- Update vendored Helm charts from upstream
- Remove 992.diff patch (label preservation fix merged upstream)
- Regenerate disable-datastore-check.diff for new version
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add skip-adjust-when-device-inaccessible.diff patch (upstream PR #477)
which prevents DRBD adjust and res file regeneration when child layer
devices are inaccessible (e.g. during encrypted resource deletion).
This patch also includes a fix for a missing setExists(true) call in
LuksLayer, which caused all new DRBD+LUKS+STORAGE resources to fail
with "not defined in your config" errors because the DRBD .res file
was never written.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add API-backed listInput dropdown for disks[].name in VMInstance form,
listing available VMDisk resources from the same namespace.
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The Flex wrapper between ResetedFormItem and Select prevented Ant
Design's Form.Item from injecting value/onChange into the Select,
causing the dropdown to appear empty even when the form store had a
value. Move Flex outside ResetedFormItem so Select is its direct child.
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Override spec.instanceType field with listInput type in schema so the
dashboard renders it as a select dropdown populated from
VirtualMachineClusterInstancetype resources. Default value is read
dynamically from the ApplicationDefinition's OpenAPI schema.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Bug: Changes to Tenant `tenant-root` will be dropped on next force (or
upgrade) reconciliation of `cosystack-basics` HelmRelease. This may lead
to data loss and outage of service
This PR fixes such behavior preserving values, applied by the user
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-basics] Preserve existing `HelmRelease` values of `tenant-root` during reconciliations
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed cluster configuration to preserve existing settings during
updates instead of overwriting them. The system now properly merges new
configuration with prior values, ensuring no settings are unexpectedly
lost.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds Harbor v2.14.2 as a managed tenant-level container registry service
in the PaaS bundle.
**Architecture:**
- Wrapper chart (`apps/harbor`) — HelmRelease, Ingress,
WorkloadMonitors, BucketClaim, dashboard RBAC
- Vendored upstream chart (`system/harbor`) from helm.goharbor.io
v1.18.2
- System chart (`system/harbor`) provisions PostgreSQL via CloudNativePG
and Redis via redis-operator
- ApplicationDefinition (`system/harbor-rd`) for dynamic `Harbor` CRD
registration
- PackageSource and paas.yaml bundle entry for platform integration
**Key design decisions:**
- Database and Redis provisioned via CPNG and redis-operator (not
internal Helm-based instances) for reliable day-2 operations
- Registry image storage uses S3 via COSI BucketClaim/BucketAccess from
namespace SeaweedFS
- Trivy vulnerability scanner cache uses PVC (S3 not supported by
vendored chart)
- Token CA key/cert persisted across upgrades via Secret lookup
- Per-component resource configuration (core, registry, jobservice,
trivy)
- Ingress with TLS via cert-manager, cloudflare issuer type handling,
proxy timeouts for large image pushes
- Auto-generated admin credentials persisted across upgrades
**E2E test:** Creates Harbor instance, verifies HelmRelease readiness,
deployment availability, credentials secret, service port, then cleans
up.
### Release note
```release-note
[harbor] Add managed Harbor container registry as a tenant-level service
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Harbor container registry deployment with integrated Kubernetes
support, including database and cache layers.
* Enabled metrics monitoring via Prometheus integration.
* Configured dashboard management interface for Harbor administration.
* **Tests**
* Added end-to-end testing for Harbor deployment and verification.
* **Chores**
* Integrated Harbor into the platform's application package bundle.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add encoding, localeCollate, and localeCType to initdb bootstrap
configuration to ensure fulltext search (ilike) works correctly.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
This PR adds the changelog for release `v1.0.0-beta.6`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.0-beta.6.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Cilium-Kilo networking variant for enhanced multi-location setup
* NATS monitoring dashboards with Prometheus integration
* DNS validation for Application names
* Operator auto-installation of CRDs at startup
* **Bug Fixes**
* Fixed HelmRelease adoption during platform migration
* Improved test reliability and timeout handling
* **Documentation**
* Enhanced Azure autoscaling troubleshooting guidance
* Updated multi-location configuration docs
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR prepares the release `v1.0.0-beta.6`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container image references from v1.0.0-beta.5 to v1.0.0-beta.6
across the platform, including updates to operators, controllers,
dashboards, storage components, and other services with corresponding
digest updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Specify initdb bootstrap with database and owner names explicitly
instead of relying on CNPG defaults which may change between versions.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Follow the same hostname pattern as kubernetes and vpn apps: use
Release.Name directly (which already includes the harbor- prefix)
instead of adding an extra harbor. subdomain.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Change sslmode from disable to require for CNPG PostgreSQL connection,
as CNPG supports TLS out of the box. Fix token key/cert preservation to
verify both values are present before passing to Harbor core.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Use tenant base domain in default hostname construction (harbor.RELEASE.DOMAIN)
to match the pattern used by other apps (kubernetes, vpn). Remove unused $ingress
variable from harbor.yaml. Add cleanup of stale resources from previous failed
E2E runs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## Summary
- Replace pre-rendered static YAML application (`kubectl apply`) with
direct `helm upgrade --install` of the `packages/core/installer` chart
in E2E tests
- Remove CRD/operator artifact upload/download from CI workflow — the
chart with correct values is already present in the sandbox via
workspace copy and `pr.patch`
- Remove `copy-installer-manifest` Makefile target and its dependencies
## Test plan
- [ ] CI build job completes without uploading CRD/operator artifacts
- [ ] E2E `install-cozystack` step succeeds with `helm upgrade
--install`
- [ ] All existing E2E app tests pass
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* PR workflows now only keep the primary disk asset; publishing/fetching
of auxiliary operator and CRD artifacts removed.
* CRD manifests are produced by concatenation and a verify-crds check
was added to unit tests; file-write permissions for embedded manifests
tightened.
* **New Features**
* Operator can install CRDs at startup to ensure resources exist before
reconcile.
* E2E install now uses the chart-based installer flow.
* **Tests**
* Added comprehensive tests for CRD-install handling and manifest
writing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace PVC-based registry storage with S3 via COSI BucketClaim/BucketAccess.
The system chart parses BucketInfo secret and creates a registry-s3 Secret
with REGISTRY_STORAGE_S3_* env vars that override Harbor's ConfigMap values.
- Add bucket-secret.yaml to system chart (BucketInfo parser)
- Remove storageType/size from registry config (S3 is now the only option)
- Use Harbor's existingSecret support for S3 credentials injection
- Add objectstorage-controller to PackageSource dependencies
- Update E2E test with COSI bucket provisioning waits and diagnostics
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[nats] add monitoring
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added Grafana dashboards for NATS JetStream and Server monitoring
* Added support for specifying container image digests and full image
names
* **Documentation**
* Enhanced NATS Helm chart documentation with container resource
configuration guidance
* **Chores**
* Updated NATS application version and component image versions
* Improved Kubernetes graceful shutdown and Prometheus exporter
configuration
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds migration 31 to adopt existing `tenant-root` Namespace and
HelmRelease
into the `cozystack-basics` Helm release during upgrade from v0.41.x to
v1.0.
In v0.41.x these resources were applied via `kubectl apply` (from the
platform
chart `apps.yaml`) with no Helm release tracking. In v1.0 they are
created by
the `cozystack-basics` chart. Without Helm ownership annotations, `helm
install`
of `cozystack-basics` fails because the resources already exist.
The migration adds:
- `meta.helm.sh/release-name` and `meta.helm.sh/release-namespace`
annotations
for Helm adoption
- `app.kubernetes.io/managed-by: Helm` label
- `helm.sh/resource-policy: keep` annotation on the HelmRelease to
prevent
accidental deletion
- `sharding.fluxcd.io/key: tenants` label required by flux sharding in
v1.0
This follows the same pattern as migrations 22 and 27 (CRD adoption).
Related: #2063
### Release note
```release-note
[platform] adopt tenant-root resources into cozystack-basics Helm release during migration
```
In v0.41.x the tenant-root Namespace and HelmRelease were applied via
kubectl apply with no Helm release tracking. In v1.0 these resources
are managed by the cozystack-basics Helm release. Without proper Helm
ownership annotations the install of cozystack-basics fails because
the resources already exist.
Add migration 31 that annotates and labels both the Namespace and
HelmRelease so Helm can adopt them, matching the pattern established
in migrations 22 and 27.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Multiple Harbor instances in the same namespace would get the same
default hostname when derived from namespace host. Use Release.Name
instead for unique hostnames per instance.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The vendored Harbor chart does an unsafe `lookup` of the Redis auth
Secret at template rendering time to extract the password. On first
install, the Secret doesn't exist yet (created by the same chart),
causing a nil pointer error. Failed installs are rolled back, deleting
the Secret, so retries also fail — creating an infinite failure loop.
Fix by generating the Redis password in the wrapper chart (same
pattern as admin password), storing it in the credentials Secret,
and injecting it via HelmRelease valuesFrom with targetPath. This
bypasses the vendored chart's lookup entirely — it uses the password
value directly instead of looking up the Secret.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Dump HelmRelease status, pods, events, and ExternalArtifact info
when harbor-test-system fails to become ready, to diagnose the
root cause of the persistent timeout.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add postgres-operator and redis-operator to PackageSource dependsOn
to ensure CRDs are available before Harbor system chart deploys.
Make persistentVolumeClaim conditional to avoid empty YAML mapping
when using S3 storage without Trivy.
Increase E2E system HelmRelease timeout from 300s to 600s to account
for CPNG + Redis + Harbor bootstrap time on QEMU.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add registry.storageType parameter (pvc/s3) to let users choose
between PVC storage and S3 via COSI BucketClaim. Default is pvc,
which works without SeaweedFS in the tenant namespace.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
ApplicationDefinition has prefix "harbor-", so CR name "harbor" produces
HelmRelease "harbor-harbor". Use name="test" and release="harbor-test"
to correctly reference all resources.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add | quote to host values in ingress for proper YAML escaping.
Add cloudflare issuer type handling following bucket/dashboard pattern.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Move Harbor from packages/extra/ to packages/apps/ as it is a
self-sufficient end-user application, not a singleton tenant module.
Update bundle entry from system to paas accordingly.
Replace registry PVC storage with S3 via COSI BucketClaim/BucketAccess,
provisioned from the namespace's SeaweedFS instance. S3 credentials are
injected into the HelmRelease via valuesFrom with targetPath.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace Harbor's internal PostgreSQL with CloudNativePG operator and
internal Redis with redis-operator (RedisFailover), following established
Cozystack patterns from seaweedfs and redis apps.
Additional fixes from code review:
- Fix registry resources nesting level (registry.registry/controller)
- Persist token CA across upgrades to prevent JWT invalidation
- Update values schema and ApplicationDefinition
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Quote resourceNames values starting with {{ to prevent YAML parser
from interpreting them as flow mappings.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add Harbor v2.14.2 as a tenant-level managed service with per-component
resource configuration, ingress with TLS termination, and internal
PostgreSQL/Redis.
Includes:
- extra/harbor wrapper chart with HelmRelease, WorkloadMonitors, Ingress
- system/harbor with vendored upstream chart (helm.goharbor.io v1.18.2)
- harbor-rd ApplicationDefinition for dynamic CRD registration
- PackageSource and system.yaml bundle entry
- E2E test with Secret and Service verification
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
CollectCRDNames now requires both apiVersion "apiextensions.k8s.io/v1"
and kind "CustomResourceDefinition", consistent with the validation in
crdinstall.Install.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Remove duplicate "Starting controller manager" log before install
phases, keep only the one before mgr.Start()
- Rename misleading test "document without kind returns error" to
"decoder rejects document without kind" to match actual behavior
- Document Helm uninstall CRD behavior in deployment template comment
- Use --health-probe-bind-address=0 consistently with metrics-bind
- Exclude all dotfiles in verify-crds diff, not just .gitattributes
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The workload reconciler was refactored to use the label
workloads.cozystack.io/monitor but the test still used the old
workloadmonitor.cozystack.io/name label, causing the reconciler to
delete the workload instead of keeping it.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Check both apiVersion and kind when validating embedded CRD manifests
to prevent applying objects with wrong API group
- Move ctrl.SetupSignalHandler() before install phases so CRD and Flux
installs respect SIGTERM instead of blocking for up to 2 minutes
- Replace custom contains/searchString helpers with strings.Contains
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Validate that all parsed objects are CustomResourceDefinition before
applying with force server-side apply. This prevents accidental
application of arbitrary resources if a non-CRD file is placed in
the manifests directory.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Move duplicated YAML parsing (ReadYAMLObjects, ParseManifestFile) and
CRD readiness check (WaitForCRDsEstablished, CollectCRDNames) into a
shared internal/manifestutil package. Both crdinstall and fluxinstall
now import from manifestutil instead of maintaining identical copies.
Replace fluxinstall's time.Sleep(2s) after CRD apply with proper
WaitForCRDsEstablished polling, matching the crdinstall behavior.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Use explicit apiextensions.k8s.io/v1 CRD GVK in waitForCRDsEstablished
instead of fragile objects[0].GroupVersionKind()
- Add TestInstall_crdNotEstablished for context timeout path
- Add --recursive to diff in verify-crds Makefile target
- Document why both crds/ and --install-crds exist in deployment template
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add verify-crds target that diffs packages/core/installer/crds/ and
internal/crdinstall/manifests/ to catch accidental divergence. Include
it in unit-tests target so it runs in CI.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Wait for CRDs to have Established condition after server-side apply,
instead of returning immediately
- Add TestInstall with fake client and interceptor to simulate CRD
establishment
- Add TestInstall_noManifests and TestInstall_writeManifestsFails for
error paths
- Fix fluxinstall/manifests.embed.go: use filepath.Join for OS paths
and restrict permissions from 0666 to 0600 (same fix as crdinstall)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Use filepath.Join instead of path.Join for OS file paths
- Restrict extracted manifest permissions from 0666 to 0600
- Add unit tests for readYAMLObjects, parseManifests, and
WriteEmbeddedManifests including permission verification
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
After generating CRDs to packages/core/installer/crds/, copy them to
internal/crdinstall/manifests/ so the operator binary embeds the latest
CRD definitions.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add --install-crds=true to cozystack-operator container args so the
operator applies embedded CRD manifests on startup via server-side apply.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Embed Package and PackageSource CRDs in the operator binary using Go
embed, following the same pattern as --install-flux. The operator applies
CRDs at startup using server-side apply, ensuring they are updated on
every operator restart/upgrade.
This addresses the CRD lifecycle concern: Helm crds/ directory handles
initial install, while the operator manages updates on subsequent
deployments.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The test-cluster target references non-existent hack/e2e-cluster.bats
file. Remove it and its dependency from the test target.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Helm installs crds/ contents before processing templates, resolving the
chicken-and-egg problem where PackageSource CR validation fails because
its CRD hasn't been registered yet.
- Move definitions/ to crds/ in the installer chart
- Remove templates/crds.yaml (Helm auto-installs from crds/)
- Update codegen script to write CRDs to crds/
- Replace helm template with cat for static CRD manifest generation
- Remove pre-apply CRD workaround from e2e test
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Helm cannot validate PackageSource CR during install because the CRD
is part of the same chart. Pre-apply CRDs via helm template + kubectl
apply --server-side before running helm upgrade --install.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace pre-rendered static YAML application with direct helm chart
installation in e2e tests. The chart directory with correct values is
already present in the sandbox after pr.patch application.
- Remove CRD/operator artifact upload/download from CI workflow
- Remove copy-installer-manifest target from testing Makefile
- Use helm upgrade --install from local chart in e2e-install-cozystack.bats
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## Summary
Add a new `cilium-kilo` networking variant that combines Cilium as the
CNI with Kilo as the WireGuard mesh overlay. This replaces the
standalone kilo PackageSource with a unified variant under the
networking source.
## Changes
- Add `cilium-kilo` variant to `networking.yaml` PackageSource with
proper component ordering and dependencies
- Add `values-kilo.yaml` for Cilium to disable host firewall when used
with Kilo
- Remove standalone `kilo.yaml` PackageSource (now integrated into
networking source)
- Switch Kilo image to official
`ghcr.io/cozystack/cozystack/kilo:v0.8.2`
- Remove unused `podCIDR`/`serviceCIDR` options and `--service-cidr`
flag from Kilo chart
Switch kilo image to official ghcr.io/cozystack/cozystack/kilo:v0.8.2,
remove unnecessary enable-ipip-termination from cilium-kilo values,
and update platform source digest.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The podCIDR was incorrectly passed as --service-cidr to prevent
masquerading on pod traffic. This is unnecessary for multi-location
mesh and was a leftover from single-cluster assumptions.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The --service-cidr flag prevents masquerading for service IPs, but
service CIDRs are cluster-local and not useful for multi-location
mesh routing. Remove the serviceCIDR value and its template usage.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add a new networking variant that integrates Kilo with Cilium
pre-configured. Cilium is deployed with host firewall disabled and
enable-ipip-termination enabled, which are required for correct IPIP
encapsulation through Cilium's overlay.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Kilo is now integrated into the cilium-kilo networking variant instead
of being a separate package that users install manually.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Fix build error in `pkg/generated/applyconfiguration/utils.go` caused by
a reference to `testing.TypeConverter` which was removed in client-go
v0.34.1.
The root cause was that `hack/update-codegen.sh` called `gen_helpers`
and
`gen_openapi` but never called `gen_client`, so the applyconfiguration
code
was never regenerated after the client-go upgrade.
Changes:
- Fix `THIS_PKG` from `k8s.io/sample-apiserver` template leftover to
correct module path
- Add `kube::codegen::gen_client` call with `--with-applyconfig` flag
- Regenerate applyconfiguration (now uses `managedfields.TypeConverter`)
- Add tests for `ForKind` and `NewTypeConverter` functions
### Release note
```release-note
[maintenance] Regenerate applyconfiguration code for client-go v0.34.1 compatibility
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated backup class definitions example to reference MariaDB instead
of MySQL.
* **Chores**
* Updated code generation tooling and module dependencies to support
enhanced functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add pre-creation cleanup of backend deployment/service and NFS pod/PVC
in `run-kubernetes.sh`, so E2E test retries start fresh instead of
reusing stuck resources from a failed attempt
- Increase the tenant deployment wait timeout from 90s to 300s to handle
CI resource pressure, aligning with other timeouts in the same function
(control plane 4m, TCP 5m, NFS pod 5m)
## Context
When `kubernetes-previous` (or `kubernetes-latest`) E2E test fails at
the `kubectl wait deployment --timeout=90s` step, `set -eu` causes
immediate exit before cleanup. On retry, `kubectl apply` sees the stuck
deployment as "unchanged", making retries 2 and 3 guaranteed to fail
against the same stuck pod.
This was observed in [run
22076291555](https://github.com/cozystack/cozystack/actions/runs/22076291555)
where `kubernetes-previous` failed 3/3 attempts while passing in 5/6
other recent runs.
## Test plan
- [ ] E2E tests pass (kubernetes-latest and kubernetes-previous)
- [ ] First clean run: `--ignore-not-found` cleanup commands are no-ops,
no side effects
- [ ] Retry after failure: stale deployment is deleted before
re-creation, fresh pod is scheduled
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved test deployment reliability with more robust pre/post-test
resource cleanup (including storage-related resources) to prevent
leftovers from prior runs.
* Made environment configuration handling more consistent to avoid
misreads of cluster credentials.
* Increased initialization and load balancer wait timeouts to reduce
flaky failures during provisioning and validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Add DNS-1035 validation for Application names to prevent creation of
resources with invalid names that would fail Kubernetes resource
creation.
### Changes
- Add `ValidateApplicationName()` function using standard
`IsDNS1035Label` from `k8s.io/apimachinery`
- Call validation in REST API `Create()` method (skipped on `Update` —
names are immutable)
- Add name length validation: `prefix + name` must fit within Helm
release name limit (53 chars)
- Remove rootHost-based length validation — the underlying label limit
issue is tracked in #2002
- Remove `parseRootHostFromSecret` and `cozystack-values` Secret reading
at API server startup
- Add unit tests for both DNS-1035 format and length validation
### DNS-1035 Requirements
- Start with a lowercase letter `[a-z]`
- Contain only lowercase alphanumeric or hyphens `[-a-z0-9]`
- End with an alphanumeric character `[a-z0-9]`
Fixes#1538Closes#2001
### Release note
```release-note
[platform] Add DNS-1035 validation for Application names to prevent invalid tenant names
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enforced DNS-1035 name format and Helm-compatible length limits for
applications during creation; name constraints surfaced early and
reflected in API validation.
* **Documentation**
* Updated tenant naming rules to DNS-1035, clarified hyphen guidance,
and noted maximum length considerations.
* **Tests**
* Added comprehensive tests covering format and length validation,
including many invalid and boundary cases.
* **API**
* OpenAPI/Swagger schemas updated to include name pattern and max-length
validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Quote all tenantkubeconfig-${test_name} references in run-kubernetes.sh
for consistent shell scripting style. The only exception is line 195
inside a sh -ec "..." double-quoted string where inner quotes would
break the outer quoting.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When the kubernetes E2E test fails at the deployment wait step, set -eu
causes immediate exit before cleanup. On retry, kubectl apply outputs
"unchanged" for the stuck deployment, making retries 2 and 3 guaranteed
to fail against the same stuck pod.
Add pre-creation cleanup of backend deployment/service and NFS test
resources using --ignore-not-found, so retries start fresh. Also
increase the deployment wait timeout from 90s to 300s to handle CI
resource pressure, aligning with other timeouts in the same function.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add operator and application PackageSource CRs to expose OpenSearch
in the Cozystack platform:
- opensearch-operator: operator deployment in cozy-opensearch-operator namespace
- opensearch-application: app + resource definition with cozy-lib integration
This enables OpenSearch to appear in the platform dashboard and be
deployed by tenants.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Matthieu <matthieu@hidora.com>
The applyconfiguration code referenced testing.TypeConverter from
k8s.io/client-go/testing, which was removed in client-go v0.34.1.
Root cause: hack/update-codegen.sh called gen_helpers and gen_openapi
but not gen_client, so applyconfiguration was never regenerated after
the client-go upgrade.
Changes:
- Fix THIS_PKG from sample-apiserver template leftover to correct
module path
- Add kube::codegen::gen_client call with --with-applyconfig flag
- Regenerate applyconfiguration (now uses managedfields.TypeConverter)
- Add tests for ForKind and NewTypeConverter functions
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Run make generate to bring generated files up to date with current
API types. This was pre-existing staleness unrelated to any code change.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
This PR prepares the release `v1.0.0-beta.5`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated all platform and system components to v1.0.0-beta.5, including
core services, controllers, dashboard, and networking utilities. Changes
include refreshed container image references across installer, platform
migrations, API services, backup systems, and object storage components
for improved stability and consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds the changelog for release `v1.0.0-beta.5`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.0-beta.5.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added generic Kubernetes deployment support
* Added Cilium-compatible kilo variant
* **Improvements**
* Enhanced cluster autoscaler with enforced node group minimum sizes
* Upgraded dashboard to version 1.4.0
* **Breaking Changes**
* Updated VPC subnet configuration structure; automated migration
provided
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Extend the installer chart to support generic and hosted Kubernetes
deployments via the existing `cozystackOperator.variant` parameter
(introduced in #2034)
- For `variant=generic`: render ConfigMaps (`cozystack`,
`cozystack-operator-config`) and an optional Platform Package CR —
resources previously required to be created manually before deploying on
non-Talos clusters
- Add variant validation in `packagesource.yaml` to fail fast on typos
- Publish the installer chart as an OCI Helm artifact
## Motivation
Deploying Cozystack on generic Kubernetes (k3s, kubeadm, RKE2) currently
requires manually creating ConfigMaps before applying the rendered
operator manifest. This change makes the installer chart variant-aware
so that:
1. `helm template -s` workflow continues to produce correct rendered
manifests
2. `helm install --set cozystackOperator.variant=generic` becomes a
viable single-command deployment path for generic clusters
3. Required ConfigMaps and optional Platform Package CR are generated
from values, eliminating manual steps
## OCI Helm chart publishing
The installer chart is now packaged and pushed to the OCI registry as
part of the `image` build target via `make chart`. A `.helmignore` file
ensures only chart-relevant files are included in the published
artifact.
## Test plan
- [ ] `helm template` with `variant=talos` (default) renders: operator +
PackageSource
- [ ] `helm template` with `variant=generic` renders: operator + 2
ConfigMaps + PackageSource
- [ ] `helm template` with `variant=generic` + `platform.enabled=true`
renders: + Package CR
- [ ] `helm template` with `variant=hosted` renders: operator +
PackageSource
- [ ] Invalid variant value produces a clear error message
- [ ] `make manifests` generates all asset files
- [ ] `helm package` produces a clean chart without build artifacts
## Summary
- Add a new `cilium` variant to the kilo PackageSource
- When selected, kilo is deployed with `--compatibility=cilium` flag
- This enables Cilium-aware IPIP encapsulation where outer packets are
routed through Cilium's VxLAN overlay instead of the host network
## Changes
- `packages/core/platform/sources/kilo.yaml` — new `cilium` variant with
`values-cilium.yaml`
- `packages/system/kilo/values-cilium.yaml` — sets `kilo.compatibility:
cilium`
- `packages/system/kilo/templates/kilo.yaml` — conditional
`--compatibility` flag
## Test plan
- [x] Tested on a cluster with Cilium networking and kilo
`--compatibility=cilium`
- [x] Verified IPIP tunneling works through Cilium's VxLAN overlay
- [x] Confirmed `kubectl exec` connectivity to worker nodes across
WireGuard mesh
Add a new "cilium" variant to the kilo PackageSource that deploys kilo
with --compatibility=cilium flag. This enables Cilium-aware IPIP
encapsulation, routing outer packets through Cilium's VxLAN overlay
instead of the host network.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Upgrade dashboard to version 1.4.0.
- Upgrade CRDs in CozyStack dashboard controller
- Add new CRD for CFOMapping
- Increase Ingress proxy timeouts, so that websocket won't be terminated
after 10 seconds of idle
- Add patch in Dockerfile to fix temporary 404 "Factory not found" error
while loading factories on page open
<img width="2560" height="1333" alt="dashboard 1 4 0"
src="https://github.com/user-attachments/assets/4bc19d0e-58a6-4506-b571-07cc6bf4880a"
/>
Notable changes:
- cluster selector is not in the sidebar
- namespace selector moved from navigation to table block
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Upgrade dashboard to version 1.4.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a new customization-mapping resource and integrated it into
dashboard navigation and static bootstrapping.
* **Improvements**
* Better routing between factory and detail pages; navigation sync
added.
* Unified dashboard link/URL placeholders for consistency.
* YAML editor now includes explicit API group/version context.
* Increased ingress/proxy timeouts for stability.
* Expanded frontend configuration for navigation, factories, and
customization behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Migrates VPC subnets definition from map format (`map[string]Subnet`) to
array format (`[]Subnet`) with an explicit `name` field. This aligns VPC
subnet definitions with the vm-instance format.
Before:
```yaml
subnets:
mysubnet0:
cidr: "172.16.0.0/24"
```
After:
```yaml
subnets:
- name: mysubnet0
cidr: "172.16.0.0/24"
```
Subnet ID generation remains unchanged — the sha256 input
(`namespace/vpcId/subnetName`) is the same, so existing resources are
not affected.
Includes a migration script (migration 30) that automatically converts
existing VPC HelmRelease values Secrets from map to array format. The
migration is idempotent and skips subnets that are already arrays or
null.
### Release note
```release-note
[vpc] Migrate VPC subnets from map to array format with automatic data migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Breaking Changes**
* VPC subnet configuration format changed from object-based to
array-based structure
* **Migration**
* Automatic migration included to convert existing VPC configurations to
new format
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Kilo container version from v0.7.1 to v0.8.0 with
corresponding digest update.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add migration script that converts VPC HelmRelease values from map
format to array format. The script discovers all VirtualPrivateCloud
HelmReleases, reads their values Secrets, and converts subnets using
yq. Idempotent: skips if subnets are already an array or null.
Bumps migration targetVersion from 30 to 31.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Change VPC subnets from map[string]Subnet to []Subnet with explicit
name field, aligning with the vm-instance subnet format.
Map format: subnets: {mysubnet: {cidr: "x"}}
Array format: subnets: [{name: mysubnet, cidr: "x"}]
Subnet ID generation (sha256 of namespace/vpcId/subnetName) remains
unchanged — subnetName now comes from .name field instead of map key.
ConfigMap output format stays the same.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
- Upgrade CRDs in CozyStack dashboard controller
- Add Ingress proxy timeouts for WebSocket to work without terminations
- Add CFOMapping custom resource
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## Summary
- Enable `enforce-node-group-min-size` option for the system
cluster-autoscaler chart
- This ensures node groups are always scaled up to their configured
minimum size, even when current workload doesn't require it
## Test plan
- [ ] Deploy cluster-autoscaler with updated values and verify the flag
is passed to the container
- [ ] Verify node groups scale up to min-size when below minimum
This PR adds the changelog for release `v1.0.0-beta.4`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.0-beta.4.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added v1.0.0-beta.4 release changelog documenting major features,
improvements in VM architecture, storage/CSI, platform infrastructure,
RBAC, and backups, along with fixes, dependency updates, and development
enhancements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR prepares the release `v1.0.0-beta.4`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container image references across system components from
v1.0.0-beta.3 to v1.0.0-beta.4, including backup controllers, API
services, dashboard components, and networking modules.
* Updated image digests to reflect latest builds.
* Fixed KubeVirt CSI driver image reference to a specific version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Add migration script (#28) that converts virtual-machine HelmReleases
into separate vm-disk + vm-instance HelmReleases, preserving disk data,
kube-ovn IPs, and LoadBalancer IPs.
## Changes
- Add migration 28 that discovers all VirtualMachine HelmReleases and
for each:
- Extracts values, saves kube-ovn IP and LoadBalancer IP
- Switches PV reclaim to Retain, stops and deletes the VM
- Creates new PVC bound to existing PV (with volumeMode detection)
- Creates vm-disk and vm-instance HelmReleases with proper values
- Pre-creates LoadBalancer service with metallb annotation if
external=true
- Cleans up old resources including protected services
- Temporarily disables CDI operator and apiserver webhooks during
DataVolume creation to bypass PVC existence validation
- Script is fully idempotent (safe to re-run if interrupted)
Replace `|| echo ""` with `|| true` to avoid newline bugs in variable
assignments. Switch `for x in $(cmd)` loops to `while read` for safer
iteration over kubectl output.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The virtual-machine application is replaced by separate vm-disk and
vm-instance applications. Migration 28 handles the conversion of
existing VirtualMachine HelmReleases.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Disable both cdi-operator and cdi-apiserver during DataVolume creation
to prevent CDI webhook from rejecting PVC adoption. Also delete mutating
webhook configurations alongside validating ones. Fix cloud-init value
serialization to avoid spurious newline. Add volumeMode detection from PV.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Splits virtual-machine HelmReleases into separate vm-disk and vm-instance
components. Handles PV data preservation, kube-ovn IP migration, and
LoadBalancer IP pinning via metallb annotation for external VMs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Port three features from virtual-machine to vm-instance:
- Add cpuModel field to specify CPU model without instanceType
- Allow switching between instancetype and custom resources via update hook
- Migrate from running boolean to runStrategy enum
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Etcd operator depends on vertical pod autoscaler in its helm chart. This
PR adds vertical pod autoscaler to the operator dependencies
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[etcd-operator] fix dependencies: add vertical-pod-autoscaler
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated etcd-operator configuration with an additional dependency
entry to enhance operational capabilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes a race condition where multiple Packages sharing the same
namespace could overwrite each other's PodSecurity Admission labels via
Server-Side Apply.
When two PackageSources deploy to the same namespace (e.g.
`cozystack.linstor` and `cozystack.linstor-scheduler` both use
`cozy-linstor`), each Package only knew about its own components. If a
Package without `privileged: true` reconciled the namespace, SSA would
remove the `pod-security.kubernetes.io/enforce=privileged` label set by
the other Package.
On Talos clusters (which default to `baseline` PodSecurity enforcement),
this caused privileged pods like LINSTOR satellites to be rejected.
The fix makes each Package check ALL PackageSources and their active
Packages when reconciling a namespace. A namespace is set to
`privileged` if ANY Package has a component with `privileged: true` in
it. This ensures a consistent, holistic decision regardless of
reconciliation order.
### Release note
```release-note
[platform] Fixed namespace PodSecurity label race condition when multiple Packages share a namespace
```
## What this PR does
This PR removes the `cozystackAPIKind` toggle and makes the controller
always reconcile `cozystack-api` as a `Deployment`.
Why:
- `cozystack-api` was switched to `Deployment` in #2041.
- Keeping a DaemonSet/Deployment switch in `cozystack-controller` made
reconciliation mode configuration-dependent.
- In CI this could leave `cozystack-api` without the expected restart on
`ApplicationDefinition` changes, causing dynamic app resources (for
example `tenants`) to be missing temporarily.
Changes:
- Remove `cozystackAPIKind` from `cozystack-controller` chart values.
- Remove conditional `--reconcile-deployment` arg from controller
Deployment template.
- Remove `--reconcile-deployment` flag handling from
`cmd/cozystack-controller`.
- Simplify `ApplicationDefinitionReconciler` to always patch
`cozystack-api` `Deployment` pod template.
### Release note
```release-note
[platform] Remove cozystackAPIKind switch and always reconcile cozystack-api as Deployment
```
Instead of using per-Package SSA field owners (which is a workaround relying
on SSA mechanics), properly resolve whether a namespace should be privileged
by iterating all PackageSources and their active Packages. A namespace gets
the privileged PodSecurity label if ANY Package has a component with
privileged: true installed in it.
This fixes the race condition where Packages sharing a namespace (e.g.
linstor and linstor-scheduler in cozy-linstor) would overwrite each other's
labels depending on reconciliation order.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
When multiple Packages share a namespace (e.g. cozystack.linstor and
cozystack.linstor-scheduler both use cozy-linstor), the shared SSA field
owner "cozystack-package-controller" caused a race condition: the last
Package to reconcile would overwrite the namespace labels set by others.
This meant that if cozystack.linstor set pod-security.kubernetes.io/enforce=
privileged and then cozystack.linstor-scheduler reconciled without the
privileged flag, SSA would remove the label. On Talos clusters (which
default to "baseline" PodSecurity enforcement), this caused privileged
satellite pods to be rejected with PodSecurity violations.
Fix by using per-Package field owners (cozystack-package-{name}), so each
Package independently manages its own namespace labels without interfering
with other Packages sharing the same namespace.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Replace DaemonSet-based deployment of cozystack-api with a regular
Deployment
using Service `trafficDistribution: PreferClose`. Previously,
cozystack-api ran
as a DaemonSet on control-plane nodes with `internalTrafficPolicy:
Local` and
direct host kube-apiserver access, providing strict node-local routing
with no
fallback. The new approach prefers routing to the nearest pod but
gracefully
falls back to remote pods when the local one is unavailable.
Key changes:
- Always deploy as Deployment instead of conditional
DaemonSet/Deployment
- Use `trafficDistribution: PreferClose` on the Service (matching
seaweedfs pattern)
- Remove `KUBERNETES_SERVICE_HOST`/`KUBERNETES_SERVICE_PORT` override
- Replace hard `nodeSelector` with soft node affinity for control-plane
nodes
- Simplify the migration hook to always clean up leftover DaemonSet
### Release note
```release-note
[platform] Switch cozystack-api from DaemonSet to Deployment with trafficDistribution: PreferClose for improved resilience
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Breaking Changes**
* Removed localK8sAPIEndpoint configuration option; existing deployments
relying on this setting require reconfiguration.
* **Infrastructure Updates**
* Simplified deployment resource strategy and updated service traffic
distribution behavior to fixed configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add a wrapper CSI driver that imports upstream kubevirt-csi-driver as
a Go module, delegates RWO volumes to upstream, and handles RWX
Filesystem volumes via LINSTOR NFS-Ganesha
- Add managed NFS application (`apps/nfs`) for provisioning per-tenant
NFS volumes with 1:1 PVC mapping (enabling snapshots, expansion, and
cloning per-volume)
- Add NFS driver system package with extra StorageClass support for
tenant clusters
- Adapt e2e tests to use native RWX support through the standard
`kubevirt` StorageClass
## How it works
RWX Filesystem PVCs in tenant clusters are handled by the wrapper CSI
driver which:
1. Creates a DataVolume with RWX accessMode in the infra cluster
2. Extracts the NFS endpoint from the LINSTOR PV
(`linstor.csi.linbit.com/nfs-export`)
3. Creates CiliumNetworkPolicy for NFS egress with VMI ownerReferences
4. Mounts NFS (`nfsvers=4.2`) in tenant VMs at the node level
RWO volumes are fully delegated to the upstream kubevirt-csi-driver
(hotplug SCSI).
## Test plan
- [x] `go build` compiles wrapper CSI driver without errors
- [x] RWO volume provisioning still works (regression)
- [x] RWX Filesystem PVC creates DataVolume with RWX in infra cluster
- [x] NFS endpoint correctly extracted from LINSTOR PV
- [x] CiliumNetworkPolicy created with correct VMI ownerReferences
- [x] NFS mount works in tenant VMs (write + read data)
- [ ] e2e test passes (`run-kubernetes.sh`)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* ReadWriteMany (RWX) NFS storage support via KubeVirt CSI driver.
* Dynamic CiliumNetworkPolicy management to allow NFS server egress.
* NFS volume mounting at the node level with automatic mount detection.
* **Tests**
* End-to-end test sequence validating RWX NFS PVC binding, I/O, and
cleanup.
* **Chores**
* Updated CSI driver image reference and runtime image deps (nfs-utils,
fs tools).
* Added RBAC for network policy management.
* New runtime flags to toggle controller/node services; updated Helm
ignore patterns.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Now CozyStack operator removes `suspend` field on all changes, so
suspending resource is impossible. This breaks local development for
packages using `make apply`
This PR preserves `suspend` field for HelmRelease from existing
manifest.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] fix: preserve `suspend` field for `HelmRelease` in package reconciler
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Bug Fixes**
* Fixed issue where the suspend state of an existing HelmRelease was not
preserved during updates, ensuring manual suspension settings remain
intact across reconciliation operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Move nil check before req dereference in CreateVolume
- Scope CiliumNetworkPolicy endpointSelector to specific VMI
- Use vmNamespace from NodeId for VMI lookup instead of infraNamespace
- Log PVC lookup errors in ControllerExpandVolume
- Wrap CNP ownerReference updates in retry.RetryOnConflict
- Fix infraClusterLabels validation to check runControllerService flag
- Dereference nodeName pointer in error message
- Replace panic with klog.Fatal for consistent error handling
- Honor CSI readonly flag in NFS NodePublishVolume
- Log mount list errors in isNFSMount
- Reorder Dockerfile ENTRYPOINT after COPY for better layer caching
- Add cleanup on e2e test failure and --wait on pod deletion
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Remove separate NFS Application dependency from e2e test. The kubevirt
CSI driver wrapper now handles RWX Filesystem volumes natively - PVCs
with ReadWriteMany accessMode use the standard kubevirt StorageClass.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Implement a wrapper around upstream kubevirt-csi-driver that adds RWX
Filesystem volume support via LINSTOR NFS exports:
- CreateVolume: intercepts RWX+Filesystem requests, creates DataVolume
with explicit AccessMode=RWX and VolumeMode=Filesystem
- ControllerPublishVolume: waits for PVC bound, extracts NFS export URL
from infra PV, creates CiliumNetworkPolicy with VMI ownerReferences
- ControllerUnpublishVolume: manages CNP ownerReferences per-VMI
- ControllerExpandVolume: delegates to upstream, disables node expansion
for NFS volumes
- NodeStageVolume/NodePublishVolume: mounts NFS at target path
- NodeExpandVolume: no-op for NFS (LINSTOR handles resize)
Also includes:
- RBAC for CiliumNetworkPolicy management in infra cluster
- Explicit --run-node-service=false for controller deployment
- Explicit --run-controller-service=false for node DaemonSet
- nfs-utils in container image for NFS mount support
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Fixes Helm upgrade failure for `monitoring-agents` caused by unrendered
`{{ .Release.Namespace }}` template expressions in `values.yaml`.
Introduces `global.target` parameter to control the target namespace for
monitoring services (vminsert, vlogs). Default is `cozy-system`,
platform bundle passes `tenant-root`. Uses `tpl` in vmagent template to
render URLs containing template expressions.
### Release note
```release-note
[monitoring] Fix YAML parse error in monitoring-agents vmagent template
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced monitoring agents configuration to use centralized
target-based values, replacing namespace-based identifiers for improved
consistency across multiple environments.
* Updated monitoring component labeling and URL handling to leverage
parametrized configuration approach for greater flexibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Renames the `mysql` application to `mariadb` across the codebase. The
chart has always deployed MariaDB (via mariadb-operator), but was
incorrectly named "mysql", causing confusion.
- Rename `packages/apps/mysql` → `packages/apps/mariadb` with all
internal references updated
- Rename `packages/system/mysql-rd` → `packages/system/mariadb-rd` with
CRD kind `MySQL` → `MariaDB`
- Update platform source, bundle references, RBAC, e2e tests, and backup
controller tests
- Add platform migration 27 to auto-discover and rename all deployed
MySQL resources to mariadb
- Preserves PVC data via PV claimRef rebinding
- Handles protection-webhook and mariadb-operator webhook during
migration
- Idempotent: safe to re-run
Real MySQL CLI/config tool names (`mysqldump`, `[mysqld]`, `MYSQL_*` env
vars) are intentionally left unchanged.
### Release note
```release-note
[apps] Rename mysql application to mariadb. A platform migration automatically renames all deployed MySQL resources. No user action required.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automated migration capability from MySQL to MariaDB with data
persistence.
* **Tests**
* Added end-to-end tests for MariaDB deployment and validation.
* **Chores**
* Updated platform components and configuration to use MariaDB as the
default database solution.
* Migrated all internal references, documentation, and build
configurations from MySQL to MariaDB.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace DaemonSet with direct host API access in favor of a regular
Deployment using Service trafficDistribution: PreferClose. This provides
prefer-local routing to the nearest cozystack-api pod with fallback
to remote pods when local one is unavailable.
- Replace DaemonSet/Deployment toggle with always-Deployment
- Replace internalTrafficPolicy: Local with trafficDistribution: PreferClose
- Remove KUBERNETES_SERVICE_HOST/PORT override (use default kubernetes service)
- Replace hard nodeSelector with soft nodeAffinity for control-plane nodes
- Simplify migration hook to always clean up DaemonSet if present
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Migration 23 already removes the `cozystack-resource-definition-crd`
HelmRelease, but existing Helm release secrets
for `*-rd` applications still reference the deleted
`CozystackResourceDefinition` CRD, causing upgrade failures.
This adds a step to delete those stale Helm secrets so that Flux can
recreate the releases cleanly.
### Release note
```release-note
[platform] Clean up stale Helm release secrets for *-rd applications during migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced system cleanup routine to remove deprecated Kubernetes
secrets during version updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add platform migration that auto-discovers all deployed MySQL
HelmReleases and renames their resources to use the mariadb prefix.
The migration handles PVC data preservation via PV claimRef rebinding,
temporarily disables protection-webhook for protected resource deletion,
and scales mariadb-operator once for all instances.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The mysql chart actually deploys MariaDB via mariadb-operator, but was
incorrectly named "mysql". Rename all references to use the correct
"mariadb" name across the codebase.
Changes:
- Rename packages/apps/mysql -> packages/apps/mariadb
- Rename packages/system/mysql-rd -> packages/system/mariadb-rd
- Rename platform source and bundle references
- Update CRD kind from MySQL to MariaDB
- Update RBAC, e2e tests, backup controller tests
- Keep real MySQL CLI/config tool names unchanged (mysqldump, [mysqld], etc.)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Root-host validation for Tenant names is no longer needed here.
The underlying issue (namespace.cozystack.io/host label exceeding
63-char limit) will be addressed in #2002 by moving the label
to an annotation.
Name length validation now only checks the Helm release name
limit (53 - prefix length), which applies uniformly to all
application types.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Separates Piraeus CRD management from the `piraeus-operator` chart into
the dedicated `piraeus-operator-crds` chart. This prevents
`piraeus-operator` from deleting CRDs during upgrades.
Changes:
- Add `installCRDs` toggle to `piraeus-operator-crds` chart
- Update `piraeus-operator` Makefile to move CRDs template on chart
update
- Add migration 27 to reassign Helm ownership on existing Piraeus CRDs
and clean up stale Helm secrets
### Release note
```release-note
[linstor] Move CRDs installation logic to piraeus-operator-crds chart
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* CRD installation can now be controlled via configuration settings
(enabled by default).
* System version upgraded to 28.
* **Chores**
* Added migration script to handle system upgrade from version 27 to 28,
including metadata updates and configuration management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Remove the FerretDB managed application from Cozystack. This includes
the application Helm chart, resource definition, platform source, PaaS
bundle entry, RBAC clusterrole entry, and e2e test. Historical migration
scripts are left intact for upgrade compatibility.
### Release note
```release-note
[ferretdb] Removed FerretDB managed application
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Removed FerretDB managed database service and associated Helm chart,
documentation, and test components from the platform.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Merges three separate operator deployment templates
(`cozystack-operator.yaml`,
`cozystack-operator-generic.yaml`, `cozystack-operator-hosted.yaml`)
into a single
template with variant selection via `cozystackOperator.variant`
parameter in `values.yaml`.
This resolves conflicts when running `make apply` from
`packages/core/installer/`, which
previously rendered all three Deployment manifests simultaneously. Now
only the selected
variant is rendered (default: `talos`).
The root `Makefile` manifests target passes `--set
cozystackOperator.variant=<variant>`
to generate the separate asset files as before.
### Release note
```release-note
[installer] Merge operator deployment templates into a single variant-based template with `cozystackOperator.variant` parameter (talos/generic/hosted)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Consolidated operator deployment templates into a unified
configuration system
* Implemented variant-specific handling for Talos, Generic, and Hosted
deployment environments
* Updated operator asset references and naming conventions across build,
release, and testing workflows
* Improved deployment configuration consistency and maintainability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace unrendered Helm template expressions in values.yaml with
global.target parameter. Pass tenant-root from platform bundle,
default to cozy-monitoring. Use tpl in vmagent template to render URLs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Migrate Helm ownership labels and annotations on Piraeus CRDs from
piraeus-operator to piraeus-operator-crds, and delete stale Helm
secrets to prevent piraeus-operator from removing CRDs on upgrade.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Update piraeus-operator Makefile to move CRDs template into
piraeus-operator-crds chart on update, and add installCRDs toggle.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add -talos suffix to the default variant output file for consistency
with -generic and -hosted variants. Update all references in CI
workflows, e2e tests, upload scripts, and testing Makefile.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Consolidate three separate operator deployment templates (talos, generic,
hosted) into a single template with variant selection via values.yaml
parameter. This resolves conflicts when running `make apply` from the
installer package, which previously rendered all three deployments.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Previously the debug log collection script that fired when CI failed
treated Packages and PackageSources as namespaced resources and as a
result of incorrect parsing failed to correctly kubectl describe and
kubectl get -oyaml them. Additionally, the script did not read the logs
of init containers. These issues are fixed with this patch.
### Release note
```release-note
[ci] Improvements to cozyreport.sh (ci log collection script): fix
retrieval of Package and PackageSource details, consider initContainers
as well as containers, when fetching logs of errored pods.
```
## What this PR does
This patch improves the naming conventions used in Cozystack's RBAC
resources. It follows the k8s system convention of using colons as
separators in RBAC resource names (e.g. system:nodes:<nodename>) and
renames some default tenant clusterroles to a scheme like
cozy:tenant:admin.
### Release note
```release-note
[rbac] Use a more structured naming convention for Cozystack's RBAC
resources.
```
## What this PR does
This patch adds cluster roles that get deployed with the backup
controller and which follow aggregation rules to automatically let users
work with resources from the backups.cozystack.io API group as soon as
the CRDs and controller are installed.
### Release note
```release-note
[backups] Add RBAC resources to let users work with backups.
```
## What this PR does
This patch improves the naming conventions used in Cozystack's RBAC
resources. It follows the k8s system convention of using colons as
separators in RBAC resource names (e.g. system:nodes:<nodename>) and
renames some default tenant clusterroles to a scheme like
cozy:tenant:admin.
### Release note
```release-note
[rbac] Use a more structured naming convention for Cozystack's RBAC
resources.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Keep the 1-minute timeout for other components (cilium, coredns, csi,
vsnap-crd) to preserve fast failure detection, and apply the 5-minute
timeout specifically to ingress-nginx which needs it after the
hostNetwork to NodePort migration.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The 1-minute timeout for waiting on HelmRelease readiness is too short
for ingress-nginx after its migration from hostNetwork to NodePort
Service, causing consistent E2E failures on kubernetes-latest.
Increase the timeout to 5 minutes to allow sufficient time for all
components to become ready.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Previously the debug log collection script that fired when CI failed
treated Packages and PackageSources as namespaced resources and as a
result of incorrect parsing failed to correctly kubectl describe and
kubectl get -oyaml them. Additionally, the script did not read the logs
of init containers. These issues are fixed with this patch.
### Release note
```release-note
[ci] Improvements to cozyreport.sh (ci log collection script): fix
retrieval of Package and PackageSource details, consider initContainers
as well as containers, when fetching logs of errored pods.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch adds cluster roles that get deployed with the backup
controller and which follow aggregation rules to automatically let users
work with resources from the backups.cozystack.io API group as soon as
the CRDs and controller are installed.
### Release note
```release-note
[backups] Add RBAC resources to let users work with backups.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This PR adds the changelog for release `v1.0.0-beta.3`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.0.0-beta.3.md`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added comprehensive changelog for v1.0.0-beta.3 release documenting
new applications, system components, networking features, virtual
machine functionality, backup capabilities, platform improvements, bug
fixes, updated dependencies, development enhancements, CI/CD changes,
and documentation updates with full contributor credits and detailed
comparison reference.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR prepares the release `v1.0.0-beta.3`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated container image versions from v1.0.0-beta.2 to v1.0.0-beta.3
across multiple system and application components.
* Refreshed image digests for cluster-autoscaler, kubevirt-csi-driver,
dashboard services, and backup controllers.
* Updated supporting infrastructure component references including
SeaweedFS, KubeOVN, Linstor, and Grafana components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Fix goimports order: duration before validation/field
- Show rootHost in error messages only for Tenant kind where it
actually affects the length calculation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Return field.ErrorList from validateNameLength for consistent
apierrors.NewInvalid error shape (was NewBadRequest)
- Add klog warning when YAML parsing fails in parseRootHostFromSecret
- Fix maxHelmReleaseName comment to accurately describe Helm convention
- Add note that root-host changes require API server restart
- Replace interface{} with any throughout openapi.go and rest.go
- Remove trailing blank line in const block
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Remove patchObjectMetaNameValidation and patchObjectMetaNameValidationV2
functions that were modifying the global ObjectMeta schema. This patching
affected ALL resources served by the API server, not just Application
resources. Backend validation in Create() is sufficient for enforcing
name constraints.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace custom DNS-1035 regex with k8s.io/apimachinery IsDNS1035Label.
Remove hardcoded maxApplicationNameLength=40 from both validation and
OpenAPI — length validation is now handled entirely by validateNameLength
which computes dynamic limits based on Helm release prefix and root-host.
Fix README to reflect that max length depends on cluster configuration.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Skip DNS-1035 and length validation on Update since Kubernetes names
are immutable — validating would block updates to pre-existing resources
with non-conforming names. Replace fmt.Printf with klog for structured
logging consistency.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Add DNS-1035 format validation to Update path (was only in Create)
- Simplify Secret reading by reusing existing scheme instead of
creating a separate client
- Add nil secret test case for parseRootHostFromSecret
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add protection against negative or zero maxLen when release prefix or
root host are too long, returning a clear configuration error instead of
a confusing "name too long" message. Add corresponding test cases.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Read root-host from cozystack-values secret at API server startup
and use it to compute maximum allowed name length for applications.
For all apps: validates prefix + name fits within the Helm release
name limit (53 chars). For Tenants: additionally checks that the
host label (name + "." + rootHost) fits within the Kubernetes label
value limit (63 chars).
This replaces the static 40-char limit with a dynamic calculation
that accounts for the actual cluster root host length.
Ref: https://github.com/cozystack/cozystack/issues/2001
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add pattern and maxLength constraints to ObjectMeta.name in OpenAPI schema.
This enables UI form validation when openapi-k8s-toolkit supports it.
- Pattern: ^[a-z]([-a-z0-9]*[a-z0-9])?$ (DNS-1035)
- MaxLength: 40
Depends on: cozystack/openapi-k8s-toolkit#1
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Clarify DNS-1035 naming rules:
- Must start with lowercase letter
- Allowed characters: a-z, 0-9, hyphen
- Must end with letter or number
- Maximum 40 characters
Change wording from "not allowed" to "discouraged" for dashes
since the validation technically permits them.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add validation to ensure Application names (including Tenants) conform
to DNS-1035 format. This prevents creation of resources with names
starting with digits, which would cause Kubernetes resource creation
failures (e.g., Services, Namespaces).
DNS-1035 requires names to:
- Start with a lowercase letter [a-z]
- Contain only lowercase alphanumeric or hyphens [-a-z0-9]
- End with an alphanumeric character [a-z0-9]
Also fixes broken validation.go that referenced non-existent internal
types (apps.Application, apps.ApplicationSpec).
Fixes: https://github.com/cozystack/cozystack/issues/1538
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Update Kube-OVN to v1.15.3
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* DNS name resolver support for improved service discovery.
* Non-primary CNI mode configuration.
* **Improvements**
* Enhanced RBAC and tighter secret-access scoping.
* Added ephemeral-storage resource limits across components.
* Better dual-stack and single-stack networking handling.
* Expanded CRD schemas with richer descriptions and status fields.
* **Updates**
* KubeOVN chart and app versions updated to v1.15.3.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Update the MongoDB application logo to the official MongoDB icon with
the leaf symbol on a green radial gradient background, replacing the
previous simplified version.
### Release note
```release-note
[mongodb] Updated MongoDB logo to the official icon
```
Replace the old MongoDB logo with the official one featuring
the MongoDB leaf icon on a green radial gradient background.
Co-authored-by: Viktoriia Kvapil <159528100+kvapsova@users.noreply.github.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Remove the FerretDB managed application, its resource definition,
platform source, RBAC entry, and e2e test. Historical migration
scripts are left intact for upgrade compatibility.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Add a new system package `cluster-autoscaler` that supports multiple
cloud providers (Hetzner and Azure) for automatic node scaling of
Cozystack management cluster.
Key features:
- Single base package with provider-specific variants via PackageSource
- Auto-update capability from upstream Helm chart
- RBAC rules for leader election (coordination.k8s.io/leases)
- Comprehensive documentation for Hetzner setup with Talos Linux
The Hetzner documentation covers:
- Talos image creation via rescue mode snapshot
- vSwitch (private network) configuration
- Correct Talos machine config structure (nodeLabels, nodeIP,
cloud-provider)
- Testing with pod anti-affinity
- Troubleshooting common issues
- Kilo mesh networking integration
### Release note
```release-note
[system] Add cluster-autoscaler package with support for Hetzner Cloud and Azure providers
```
Documentation moved to the website repository as a separate PR.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Cluster-autoscaler does not require privileged installation.
Remove the unnecessary privileged: true setting from both
Hetzner and Azure PackageSource definitions.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This patch installs the kubevirt-velero-plugin, so that backing up a
kubevirt resource such as a VMI will automatically include related
items, such as datavolumes, and, eventually, all the way down to pods
and volumes.
### Release note
```release-note
[backups] Include the kubevirt-velero-plugin with the Velero package.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added KubeVirt plugin integration: Velero now provides comprehensive
backup and restore support for KubeVirt-managed virtual machines and
associated resources. The plugin is automatically installed and
configured during deployment, enabling unified disaster recovery for
both containerized applications and virtualized workloads.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Enables the installation of the backupstrategy controller by default.
### Release note
```release-note
[backups] Install the backupstrategy controller by default.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Backup strategy component is now included by default in all system
installations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch adds another label selector for resources being backed up by
the default Velero strategy for virtual machines, ensuring that the
actual VM and VMI are captured by selector.
### Release note
```release-note
[backups] Refined the label selector in the Velero VM backup strategy to
capture resources previously missed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded backup scope to include additional virtual machine and
storage resources.
* Added configurable timeout for snapshot operations (10 minutes).
* Enhanced resource selection logic for more flexible backup filtering.
* **Configuration Changes**
* Updated backup strategy configuration for improved resource coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Remove duplicate `dashboards.list` from `extra/monitoring`
- Dashboard files should only exist in `system/monitoring`
- Dashboards are created by `system/monitoring` when deployed via
HelmRelease
## Test plan
- [ ] Deploy extra/monitoring and verify GrafanaDashboard resources are
created by system/monitoring
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Removed monitoring dashboard configuration entries from the
deployment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds optional node-lifecycle-controller component to local-ccm package
that automatically deletes unreachable NotReady nodes from the cluster.
This solves the problem of "zombie" node objects left behind when
cluster autoscaler deletes cloud instances but the node object remains
in Kubernetes.
**Features:**
- Monitors nodes matching a configurable label selector
- Deletes nodes that are NotReady for configurable duration (default:
5m)
- Verifies unreachability via ICMP ping before deletion
- Supports protected labels to prevent deletion of specific nodes
- Leader election for HA deployment
**Usage:**
```yaml
nodeLifecycleController:
enabled: true
nodeSelector: "node.kubernetes.io/instance-type" # optional
protectedLabels: "kilo.squat.ai/leader" # optional
```
Disabled by default.
### Release note
```release-note
[local-ccm] Add optional node-lifecycle-controller to automatically cleanup unreachable NotReady nodes
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced optional Node Lifecycle Controller component for managing
node readiness states with configurable timeouts, health monitoring, and
reconciliation intervals. Supports leader election, dry-run modes, and
customizable node selectors and protected labels for operational
flexibility. Disabled by default.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Updates kilo to v0.7.0 from the cozystack fork, replacing the local
image build (with patches applied via Dockerfile) with a pre-built image
from `ghcr.io/cozystack/cozystack/kilo`. This simplifies the build
process and removes the need for maintaining local patches.
Additionally, exposes the `--mtu` flag in kilo's `values.yaml` to allow
overriding the WireGuard interface MTU. The default value is set to
`auto`, letting kilo determine the optimal MTU automatically.
### Release note
```release-note
[kilo] Update kilo to v0.7.0 from cozystack fork; add configurable `mtu` parameter (default: auto) for WireGuard interface
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable MTU support with automatic detection
* Added internal CIDR filtering capability for network mesh
configuration
* **Updates**
* Updated kilo image version to v0.7.0
* Transitioned to pre-built remote images
* **Improvements**
* Refined IP overlap and containment warning logic for better
diagnostics
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cilium] change cilium-operator replicas to 1
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated operator configuration settings. Added support for configuring
the number of operator replicas, enabling adjustment of deployment
specifications based on infrastructure requirements and resource
availability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Adds system package for
[clustersecret-operator](https://github.com/sap/clustersecret-operator)
using Helm Chart v0.3.68
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[clustersecret-operator] add option to deploy clustersecret-operator v0.3.68
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added ClusterSecret operator for managing cluster-scoped secrets
across multiple namespaces
* Secrets can be distributed to selected namespaces using namespace
selectors
* Supports template-based secret creation with customizable data
* Includes validation webhooks and admission control for ClusterSecret
resources
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The cozystack-values secret is created as part of the cozystack-basics
chart and is required by cozystack to install properly, however there is
a race condition between it and the lineage-controller-webhook, where it
may be blocked from being created if the mutating webhook configuration
is already set up. By adding a system label to this secret it is dropped
from consideration by the webhook.
### Release note
```release-note
[platform] Resolve race condition between the system cozystack-values
secret and the lineage webhook by adding a label that causes the webhook
to ignore this secret.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated system manifest metadata configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Enable insecure TLS verification for Keycloak communication to support
environments with self-signed certificates:
- **keycloak-configure**: Switch ClusterKeycloak to use internal service
URL (`http://keycloak-http.cozy-keycloak.svc:8080`) and enable
`insecureSkipVerify: true`
- **dashboard**: Add `--ssl-insecure-skip-verify=true` flag to
oauth2-proxy
### Release note
```release-note
[keycloak-configure,dashboard] Enable insecure TLS verification for Keycloak by default to support self-signed certificates
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added an option to skip OIDC TLS certificate verification (disabled by
default).
* Authentication proxy can now be configured to omit SSL verification
when enabled.
* Keycloak connection switched to a non-TLS endpoint and TLS
verification handling updated.
* Removed the Keycloak authorization-services toggle from configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The cozystack-values secret is created as part of the cozystack-basics
chart and is required by cozystack to install properly, however there is
a race condition between it and the lineage-controller-webhook, where it
may be blocked from being created if the mutating webhook configuration
is already set up. By adding a system label to this secret it is dropped
from consideration by the webhook.
### Release note
```release-note
[platform] Resolve race condition between the system cozystack-values
secret and the lineage webhook by adding a label that causes the webhook
to ignore this secret.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch adds another label selector for resources being backed up by
the default Velero strategy for virtual machines, ensuring that the
actual VM and VMI are captured by selector.
### Release note
```release-note
[backups] Refined the label selector in the Velero VM backup strategy to
capture resources previously missed.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Enables the installation of the backupstrategy controller by default.
### Release note
```release-note
[backups] Install the backupstrategy controller by default.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch installs the kubevirt-velero-plugin, so that backing up a
kubevirt resource such as a VMI will automatically include related
items, such as datavolumes, and, eventually, all the way down to pods
and volumes.
### Release note
```release-note
[backups] Include the kubevirt-velero-plugin with the Velero package.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add field `cpuModel` to specify VirtualMachine CPU model without usage
of `InstanceType`
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vm] add field `cpuModel` to specify CPU model without usage of KubeVirt InstanceType
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added CPU model configuration parameter for virtual machines, allowing
users to specify the desired CPU model for their VM instances.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Dashboard files should only be in system/monitoring.
The dashboards are created by system/monitoring when deployed
via HelmRelease from extra/monitoring.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated code ownership governance (no user-facing impact).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add authentication.oidc.insecureSkipVerify option to platform chart
with default=false. The flag is now conditionally included in
oauth2-proxy args only when explicitly enabled.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## Summary
- Create new `piraeus-operator-crds` package containing all piraeus CRDs
- Add `piraeus-operator-crds` as dependency for `piraeus-operator` in
linstor PackageSource
- Set `privileged: true` for CRDs package to ensure namespace gets
correct PodSecurity labels
- Disable `installCRDs` in piraeus-operator values
## Motivation
Helm does not reliably install all CRDs from the `templates/` directory
when the crds.yaml file is large. This causes
`linstorsatellites.piraeus.io` CRD to be missing, which breaks satellite
pod creation.
Separating CRDs into a dedicated package (similar to cert-manager-crds,
prometheus-operator-crds) ensures reliable CRD installation.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Introduced a separate component for managing Piraeus operator CRDs
with explicit dependency ordering.
* Updated operator configuration to disable built-in CRD installation
and depend on the new dedicated CRD component.
* Established installation dependency chain to ensure components
initialize in the correct sequence.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Increase TLS certificate duration from 90 days to 10 years
- Adjust renewBefore from 15 to 30 days
- Prevents certificate expiration issues in SeaweedFS
## Test plan
- [ ] Deploy SeaweedFS with `global.enableSecurity=true`
- [ ] Verify certificates are issued with 10 year duration
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added RestoreJob resource and end-to-end restore workflows with Velero
integration, including polling and status tracking.
* Introduced Velero restoreSpec to strategy templates and a
virtual-machine backup strategy template.
* Enhanced backup plans UI with backup class and schedule fields.
* **Bug Fixes**
* Enforced non-empty backupClassName and immutability for BackupJob.
* **Chores**
* Removed BackupJob webhook implementation and its tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds `startupProbe` to both `bff` and `web` containers in the dashboard
deployment. On slow hardware, kubelet kills containers because the
`livenessProbe` only allows ~33 seconds for startup. The `startupProbe`
gives containers up to 60 seconds to start before `livenessProbe` kicks
in.
### Release note
```release-note
[dashboard] Add startupProbe to bff and web containers to prevent restarts on slow hardware
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added health monitoring during service startup for core application
services. Services now perform health checks to verify proper
initialization and readiness before handling requests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Implemented by upgrade hook atomically patching VM resource
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Allow switching between `instancetype` and custom resources for VM
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vm] allow switching between instancetype and custom resources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved virtual machine resource configuration with more flexible CPU
and memory management
* Added support for transitioning between instancetype and custom
resource configurations
* Enhanced VM update process to automatically detect and handle
resource-related changes, including removal of outdated configurations
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Automatically enable `enableGatewayAPI` in cert-manager when the
Gateway API addon is enabled
- Uses the same `define` + `mergeOverwrite` pattern as Cilium for
consistency
- User can still override via `valuesOverride`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Improved certificate manager configuration handling by enhancing the
integration of default settings with custom user overrides in Kubernetes
deployments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove authorizationServicesEnabled as it's incompatible with public
clients (requires service account which public clients don't have).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Switch ClusterKeycloak to use internal service URL instead of external
ingress, and enable insecureSkipVerify for environments with self-signed
certificates.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Migrates usage of deprecated field `running` to `runStrategy`
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vm] migrate usage of deprecated field `running` to `runStrategy`
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* VM running states now support Always, Halted, Manual, RerunOnFailure,
and Once strategies
* Configure CPU, memory, and socket allocation for virtual machines
* Add GPU support to virtual machines
* Define network subnets for virtual machines
* Configure storage classes and options for system disks
* Set SSH keys and cloud-init parameters for initialization
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Remove `cozystack.cozystack-controller` from monitoring package
dependencies
- Allows monitoring to work without cozystack-engine being enabled
## Motivation
The monitoring package depends on `cozystack.cozystack-controller` which
is only installed as part of `cozystack-engine`. This prevents
monitoring from becoming ready when cozystack-engine is not enabled.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Simplified monitoring default: reduced installed sub-components and
top-level dependencies for a leaner deployment.
* Removed generation of several automated workload-monitoring manifests
to streamline resource set.
* **New Features**
* Added a curated list of monitoring dashboards for cluster, networking,
storage, databases, control-plane, ingress, observability tooling, and
key services.
* Added three ExternalName services to expose tenant monitoring
endpoints.
* **Improvements**
* Made monitoring agents namespace-aware by templating
tenant/multi-namespace references for metrics and logging endpoints.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Expose the --mtu flag in values.yaml to allow overriding the default
WireGuard interface MTU (1420). This is needed for environments where
the underlying network has a lower MTU, such as Azure VMs (eth0 MTU
1400), to avoid packet fragmentation in the WireGuard tunnel.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Add Qdrant vector database as a new managed application, following the
HelmRelease-based vendoring pattern (same as nats, ingress, seaweedfs).
Changes:
- `packages/system/qdrant/` — vendored upstream Qdrant Helm chart
- `packages/apps/qdrant/` — wrapper chart creating Flux HelmRelease CR
- `packages/system/qdrant-rd/` — ApplicationDefinition for
`apps.cozystack.io/v1alpha1/Qdrant` CRD
- `hack/e2e-apps/qdrant.bats` — E2E test
Features:
- Single replica or clustered mode (automatic based on replica count)
- Persistent storage with configurable size and storage class
- Resource presets (nano to 2xlarge)
- API key authentication (auto-generated)
- Optional external LoadBalancer access
- Dashboard integration with WorkloadMonitor and ServiceMonitor
### Release note
```release-note
[qdrant] Add Qdrant vector database as a managed application
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added Qdrant vector database as a managed service, enabling users to
deploy and manage Qdrant instances with configurable replicas, storage,
and resource presets.
* Introduced support for external access, persistent storage, API key
management, and metrics monitoring integration.
* Added end-to-end testing for Qdrant deployments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add PackageSource and PaaS bundle entry so the platform deploys
qdrant-rd, which provides the ApplicationDefinition that registers
the Qdrant Kind in the API server.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Previously a namespaced role was created per tenant and access level.
Since these roles are all identical, it's sufficient to create a cluster
role per access level and just create namespaced rolebindings to these
cluster roles per tenant. This will enable aggregation rules. E.g. if a
new API group is installed, such as backups.cozystack.io, a new
clusterrole can be created for managing this API group with a label like
rbac.cozystack.io/aggregate-to-admin. Smart use of aggregation rules
will enable automatic granting of access rights not just to admin, but
to super-admin too, and there will be no need to update every single
tenant.
### Release note
```release-note
[tenant,rbac] Use ClusterRoles with aggregationRules instead of roles
per every tenant.
```
## What this PR does
Previously a namespaced role was created per tenant and access level.
Since these roles are all identical, it's sufficient to create a cluster
role per access level and just create namespaced rolebindings to these
cluster roles per tenant. This will enable aggregation rules. E.g. if a
new API group is installed, such as backups.cozystack.io, a new
clusterrole can be created for managing this API group with a label like
rbac.cozystack.io/aggregate-to-admin. Smart use of aggregation rules
will enable automatic granting of access rights not just to admin, but
to super-admin too, and there will be no need to update every single
tenant.
### Release note
```release-note
[tenant,rbac] Use ClusterRoles with aggregationRules instead of roles
per every tenant.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Add topology.kubernetes.io/zone node label alongside kilo.squat.ai/location
for standard Kubernetes topology awareness across Hetzner and Azure zones.
Update Azure docs with correct Package resource format (spec.components),
replace hardcoded values with placeholders, and remove environment-specific
infrastructure references.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This patch adds a conditional for running on a statically defined VM the
maintainers have SSH access to if the pull request has a `debug` label.
This is useful for debugging failing workflows when the diagnostic info
from the pipeline is insufficient.
### Release note
```release-note
[ci] Run builds on a static VM with SSH access if the PR has a debug
label.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
This release contains no user-facing changes. Internal workflow
adjustments have been made to optimize development infrastructure.
* **Chores**
* Updated CI/CD pipeline configuration for improved test execution
efficiency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
WorkloadMonitor resources are only needed when extra/monitoring is
deployed. Move them from system/monitoring to extra/monitoring package.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
This patch adds a conditional for running on a statically defined VM the
maintainers have SSH access to if the pull request has a `debug` label.
This is useful for debugging failing workflows when the diagnostic info
from the pipeline is insufficient.
### Release note
```release-note
[ci] Run builds on a static VM with SSH access if the PR has a debug
label.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
- Replace hardcoded tenant-root with {{ .Release.Namespace }} in vmagent
and fluent-bit configs
- Add ExternalName services in cozystack-basics to redirect monitoring
traffic from cozy-monitoring to tenant-root when engine is deployed
- Add missing components to monitoring PackageSource
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
When the Gateway API addon is enabled, automatically configure
cert-manager with enableGatewayAPI: true. Uses the same default
values + mergeOverwrite pattern as Cilium for consistency.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Kubelet kills bff and web containers on slow hardware because the
livenessProbe only allows 33 seconds for startup. Add startupProbe
with failureThreshold=30 and periodSeconds=2, giving containers up
to 60 seconds to start before livenessProbe kicks in.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add dashboards.list that defines which Grafana dashboards to create.
Without this file, no GrafanaDashboard resources were being generated.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add optional node-lifecycle-controller that automatically deletes
unreachable NotReady nodes from the cluster. This solves the problem
of "zombie" node objects left behind when cluster autoscaler deletes
cloud instances.
Features:
- Monitors nodes matching a label selector
- Deletes nodes that are NotReady for configurable duration
- Verifies unreachability via ICMP ping before deletion
- Supports protected labels to prevent deletion of specific nodes
- Leader election for HA deployment
Disabled by default. Enable with:
nodeLifecycleController:
enabled: true
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Remove components that are already defined in separate PackageSources
to avoid race conditions and HelmRelease conflicts:
- vertical-pod-autoscaler
- postgres-operator
- grafana-operator
- victoria-metrics-operator
- prometheus-operator-crds
These components should be installed via their dedicated PackageSources.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Helm does not reliably install all CRDs from templates/ directory,
particularly when the crds.yaml file is large. This causes
linstorsatellites.piraeus.io CRD to be missing, breaking satellite pod
creation.
Changes:
- Create new piraeus-operator-crds package with all piraeus CRDs
- Add piraeus-operator-crds as dependency for piraeus-operator
- Set privileged: true for CRDs package to ensure namespace has correct
PodSecurity labels from the start
- Disable installCRDs in piraeus-operator since CRDs come from separate
package
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The monitoring package depends on cozystack.cozystack-controller which
is only installed as part of cozystack-engine. This prevents monitoring
from becoming ready when cozystack-engine is not enabled.
Remove this dependency to allow monitoring to work independently.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Use GitHub Copilot CLI for automatic changelog generation on tagged
releases.
### How it works
When a new version tag is pushed, the `generate-changelog` job in
`tags.yaml`:
1. Checks out the `main` branch with full history and tags
2. Verifies that a changelog file doesn't already exist in
`docs/changelogs/`
3. Installs GitHub Copilot CLI (`@github/copilot` npm package)
4. Runs Copilot CLI in non-interactive mode (`-p` flag) with
`--allow-all-tools --allow-all-paths` to generate the changelog
following the instructions in `docs/agents/changelog.md`
5. Commits the generated file to a `changelog-vX.Y.Z` branch and opens a
PR to `main`
### Authentication
- `COPILOT_GITHUB_TOKEN` secret — fine-grained PAT with **"Copilot
Requests: Read"** account permission, used to authenticate Copilot CLI
- `GH_PAT` secret — used by `gh` CLI inside the Copilot session to query
PR authors via GitHub API
### Release note
```release-note
[ci] Replaced Gemini with GitHub Copilot CLI for automatic changelog generation on release tags
```
## Summary
- Fix invalid PromQL expression in `CNPGClusterOffline` alert rule that
causes vmalert pods to crash
- Fix PromQL logic so the alert fires correctly when cluster is offline
## Problem
The `CNPGClusterOffline` alert in
`packages/system/postgres-operator/alerts/cnpg-default-alerts.yaml` had
two issues:
1. **Syntax error** - Extra closing parenthesis causing vmalert pods to
crash:
```promql
sum by (namespace, pod) (cnpg_collector_up)) OR on() vector(0) == 0
^^-- extra )
```
2. **Logic error** - After removing the extra parenthesis, the
expression still had incorrect logic:
```promql
sum by (namespace, pod) (cnpg_collector_up) OR on() vector(0) == 0
```
This evaluates as `cnpg_collector_up OR (vector(0) == 0)` which fires
when the cluster is **online** (wrong behavior).
## Solution
Fix both issues by properly wrapping the OR expression in parentheses:
```promql
(sum by (namespace, pod) (cnpg_collector_up) OR on() vector(0)) == 0
```
This correctly fires when `cnpg_collector_up` is 0 or absent (cluster
actually offline).
## Test plan
- [ ] Verify vmalert pods start successfully after applying the fix
- [ ] Verify the alert rule is properly loaded
- [ ] Verify alert only fires when CNPG cluster is actually offline
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Change TLS certificate duration from 90 days to 10 years to prevent
certificate expiration issues. Also adjust renewBefore from 15 to 30 days.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
When OIDC is disabled, the dashboard's token-proxy now properly
validates bearer tokens against the k8s API's JWKS url.
### Release note
```release-note
[dashboard] Verify bearer tokens against the issuer's JWKS url.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Implemented JWKS-based JWT token verification for enhanced security.
* Added conditional role-based access control for non-OIDC deployments.
* **Chores**
* Updated authentication dependencies and modernized token validation.
* Removed legacy external token validation mechanism.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When OIDC is disabled, the dashboard's token-proxy now properly
validates bearer tokens against the k8s API's JWKS url.
### Release note
```release-note
[dashboard] Verify bearer tokens against the issuer's JWKS url.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Remove extra closing parenthesis in the CNPGClusterOffline alert expression
that causes vmalert pods to crash with "bad prometheus expr" error.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- Add monitoring as a system chart
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced comprehensive monitoring stack with Grafana, Victoria
Metrics operators, Prometheus operator, and vertical pod autoscaler for
enhanced system observability and performance metrics.
* **Chores**
* System upgraded to version 27 with automatic monitoring resource
reorganization for improved infrastructure management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add migration 26 that re-labels monitoring resources to be owned by
monitoring-system HelmRelease instead of monitoring. This allows
seamless transition from direct resource management in extra/monitoring
to delegated management via system/monitoring chart.
Migration steps:
- Find all monitoring HelmReleases in tenant namespaces
- Suspend HelmRelease to prevent reconciliation
- Delete helm secrets to orphan resources
- Relabel all resources to monitoring-system
- Delete suspended HelmRelease
Also updates:
- monitoring-application.yaml: add monitoring-system component
- helmrelease.yaml: reference monitoring-system ExternalArtifact
- values.yaml: bump targetVersion to 25
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Configure CoreDNS chart to create `kube-dns` ServiceAccount matching
Kubernetes bootstrap ClusterRoleBinding
- Fixes RBAC errors (`Failed to watch`) when CoreDNS pods restart
## Problem
Kubernetes bootstrap creates a `ClusterRoleBinding: system:kube-dns`
that references `ServiceAccount: kube-dns` in `kube-system`. However,
the coredns chart was using the `default` ServiceAccount because
`serviceAccount.create` was not enabled.
This caused CoreDNS pods to fail with `[ERROR] plugin/kubernetes: Failed
to watch` errors after restarts, as they lacked RBAC permissions to
watch the Kubernetes API. Old pods worked due to cached data, but new
pods failed after rollout.
## Solution
Add ServiceAccount configuration to
`packages/system/coredns/values.yaml`:
```yaml
serviceAccount:
create: true
name: kube-dns
```
## Test plan
- [x] Verify ServiceAccount `kube-dns` is created: `kubectl get sa -n
kube-system kube-dns`
- [x] Verify deployment uses correct ServiceAccount: `kubectl get
deployment -n kube-system coredns -o
jsonpath='{.spec.template.spec.serviceAccountName}'` → `kube-dns`
- [x] Restart CoreDNS pods and verify all pods are Ready
- [x] Check logs show no RBAC errors
- [x] Test DNS resolution works
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated DNS service configuration with service account settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add migration 25 for v1.0 upgrade cleanup:
- Remove legacy `cozystack` installer deployment
- Remove legacy `cozystack-assets` statefulset
- Remove `bootbox` and `bootbox-rd` HelmReleases (will be recreated by
new platform chart if enabled)
- Revert migration 23 to original state (only removes
cozystack-resource-definition-crd)
- Update targetVersion from 24 to 26
- Fix jq syntax error in `hack/migrate-to-version-1.0.sh`
Migration 23 was already executed on existing clusters at version 23+,
so adding cleanup there would never run. The new migration 25 ensures
proper cleanup during v1.0 upgrade.
## Test plan
- [ ] Upgrade from v0.4x to v1.0
- [ ] Verify old `cozystack` deployment is deleted
- [ ] Verify old `cozystack-assets` statefulset is deleted
- [ ] Verify old `bootbox` and `bootbox-rd` HelmReleases are deleted
- [ ] Verify platform installs successfully after migration
- [ ] Run `./hack/migrate-to-version-1.0.sh` without jq errors
Migration 23 was already executed on existing clusters, so the legacy
installer removal never ran. This moves the cleanup to a new migration 25
which will execute during upgrade.
Changes:
- Revert migration 23 to original state (only removes cozystack-resource-definition-crd)
- Add migration 25 for v1.0 upgrade cleanup:
- Remove legacy cozystack installer deployment
- Remove legacy cozystack-assets statefulset
- Remove bootbox and bootbox-rd HelmReleases
- Update targetVersion from 24 to 26
- Fix jq syntax error in migrate-to-version-1.0.sh
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
- Remove duplicate `bootbox-system` component from `bootbox-application`
PackageSource (was duplicating system/bootbox installation)
- Auto-create `cozystack.bootbox-application` Package when bootbox is
enabled in `bundles.enabledPackages`
- This ensures `bootbox-rd` (ApplicationDefinition) is properly
installed as a dependency before `cozystack.bootbox`
## Test plan
- [ ] Enable bootbox in `bundles.enabledPackages`
- [ ] Verify both `cozystack.bootbox-application` and
`cozystack.bootbox` Packages are created
- [ ] Verify `bootbox-rd` (ApplicationDefinition) is installed
- Fix docker build context for migrations image
- Fix yq flags for compatibility with Mike Farah's yq v4
- Add platform image to main build target
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add cleanup of old installer components during migration:
- Delete cozystack deployment
- Delete cozystack-assets statefulset
This ensures no conflict between old installer and new operator
when upgrading to v1.0 architecture.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
- Remove duplicate bootbox-system component from bootbox-application
PackageSource (was duplicating system/bootbox installation)
- Auto-create cozystack.bootbox-application Package when bootbox is
enabled in bundles.enabledPackages
- This ensures bootbox-rd (ApplicationDefinition) is properly installed
as a dependency before cozystack.bootbox
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[
- add BackupClass Velero and strategy for VM
- dashboard fixes for BackupClass and simplifed Plan/BackupJob API
]
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Implement restoreJob controller and velero strategy.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a RestoreJob controller with manager registration to handle
restore operations.
* Added Velero-based and Job-based restore workflows with lifecycle
handling, requeueing, and polling.
* Velero templates now include a RestoreSpec field to configure
restores.
* **Bug Fixes / Reliability**
* New helpers to mark BackupJob and RestoreJob failures and reliably
update status.
* **Documentation / CRD**
* RestoreJob CRD gains a status subresource and a Phase print column.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Removes `hostNetwork` for Ingress NGINX Controller for Managed
kubernetes cluster
- Adds Service of type NodePort for load balancing
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] use NodePort service instead of hostNetwork for ingress-nginx
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Configuration**
* Updated ingress-nginx controller to use NodePort exposure method
(ports 30000 for HTTP, 30001 for HTTPS) when configured as "Proxied."
* Removed unnecessary hostNetwork configuration for the controller.
* Service port mappings updated to align with new NodePort settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds separate values to keycloak branding.
### Release note
```release-note
Added separate values to keycloak branding
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Enhanced branding configuration application with improved conditional
logic to ensure branding is only applied when appropriate configuration
values are present.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Aligns MongoDB configuration with postgres and mysql patterns by
introducing a separate `databases` section.
**Before:**
```yaml
users:
myuser:
db: mydb
roles:
- name: readWrite
db: mydb
- name: dbAdmin
db: mydb
```
**After:**
```yaml
users:
myuser: {}
databases:
mydb:
roles:
admin:
- myuser
readonly:
- reader
```
Changes:
- Add separate `databases` section with `roles.admin` and
`roles.readonly`
- Simplify `users` to only contain optional password field
- All users authenticate via `admin` database (MongoDB production best
practice)
- `admin` role maps to `readWrite` + `dbAdmin` permissions
- `readonly` role maps to `read` permission
## Migration
Includes migration script (`migrations/24`) that automatically converts
existing MongoDB HelmReleases:
- Detects old format users (with `db` and `roles` fields)
- Converts `readWrite`/`dbAdmin` roles → `databases.{db}.roles.admin`
- Converts `read` role → `databases.{db}.roles.readonly`
- Preserves user passwords in new `users` format
- Skips HelmReleases that don't have old format
### Release note
```release-note
[mongodb] Unified users and databases configuration to match postgres and mysql patterns. Users now defined separately from databases, with role assignments in the databases section. Includes automatic migration for existing deployments.
```
This PR prepares the release `v1.0.0-beta.2`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated multiple system component images from alpha to beta releases
(v1.0.0-beta.2)
* Updated container image digests across various services for
consistency and stability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Service will be recreated
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Allow changing field `external` after creation.
Service will be deleted and then created with correct type.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[vm] recreate service on `external` change to allow changing this field after creation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Detects Service type changes during virtual machine updates and
automatically recreates the Service as needed; update workflow now runs
a job to remove and recreate Services before reconciliation.
* **Chores**
* Updated RBAC to grant the update workflow permissions to manage
(delete) the targeted Service.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add detailed documentation for setting up cluster-autoscaler with Hetzner
Cloud and Talos Linux, including:
- Talos image creation via rescue mode
- vSwitch (private network) configuration
- Correct Talos machine config structure for nodeLabels and nodeIP
- Package deployment with RBAC rules for leader election
- Testing with pod anti-affinity
- Configuration reference tables (env vars, server types, regions)
- Troubleshooting section for common issues
- Kilo mesh networking integration
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add coordination.k8s.io/leases permissions to Role via additionalRules.
This fixes leader election failures in the cluster-autoscaler.
Also add documentation for Hetzner Cloud setup with Talos Linux.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add cluster-autoscaler system package with support for multiple cloud
providers. Each provider has its own PackageSource and values file,
allowing simultaneous deployment in multi-cloud setups.
Supported providers:
- Hetzner Cloud
- Azure
Each instance uses a unique leader-elect-resource-name to prevent
conflicts when running multiple autoscalers in the same cluster.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The Kubernetes bootstrap creates a ClusterRoleBinding 'system:kube-dns'
that references ServiceAccount 'kube-dns' in 'kube-system'. However,
the coredns chart was using the 'default' ServiceAccount because
serviceAccount.create was not enabled.
This caused CoreDNS pods to fail with 'Failed to watch' errors after
restarts, as they lacked RBAC permissions to watch the Kubernetes API.
Configure the chart to create the 'kube-dns' ServiceAccount, which
matches the expected binding from Kubernetes bootstrap.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
Add instance profile metadata to workload monitor
Currently getWorkloadMetadata only extracts the
kubevirt.io/cluster-instancetype-name annotation from VMI objects. This
misses the instance profile information
(kubevirt.io/cluster-preference-name), which is needed to track
workload sizing/configuration alongside the instance type.
This change adds extraction of the cluster instance profile annotation
and exposes it as the
workloads.cozystack.io/kubevirt-vmi-instance-profile label on collected
metrics.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Workload monitoring now detects kubevirt instance profile annotations
and adds a matching workload label
(workloads.cozystack.io/kubevirt-vmi-instance-profile) when present.
* Enriched workload metadata for virtualized workloads.
* No other behavior or control-flow changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Configured cluster for high availability with multiple replicas
* Implemented rolling update strategy to ensure safer deployments
* Enhanced health constraints to maintain cluster stability during
updates
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Enable Cozystack deployment on generic Kubernetes clusters (kubeadm,
k3s, RKE2, etc.) by making Talos-specific configurations conditional and
adding a new `isp-full-generic` bundle variant.
## Problem 1: Cilium uses hardcoded Talos API endpoint
**Issue**: Cilium is configured to connect to Kubernetes API at
`localhost:7445` (Talos KubePrism). On non-Talos clusters, the API
server runs on standard `<node-ip>:6443`.
**Solution**: Add `networking.apiServer.host` and
`networking.apiServer.port` to platform values, passed to Cilium via
bundle template.
**Why this approach**: Allows per-deployment configuration while keeping
Talos defaults for backwards compatibility.
## Problem 2: Cilium cgroup automount disabled
**Issue**: `values-talos.yaml` sets `cgroup.autoMount.enabled: false`
because Talos mounts cgroups. On Ubuntu/Debian, Cilium needs to mount
cgroups itself.
**Solution**: Add `networking.cilium.cgroup.autoMount` configuration and
create `cilium-generic` / `kubeovn-cilium-generic` networking variants
without `values-talos.yaml`.
**Why this approach**: Separate variants avoid complex conditional logic
in templates and make the difference explicit.
## Problem 3: KubeOVN helm lookup fails on fresh clusters
**Issue**: KubeOVN chart uses `lookup` to find control-plane nodes.
During initial deployment, lookup returns empty results causing
installation failure.
**Solution**: Add `networking.kubeovn.MASTER_NODES` parameter to
explicitly pass node IPs when needed.
**Why this approach**: Helm lookup is unreliable during initial
deployment; explicit configuration is more predictable.
## Problem 4: LinstorSatelliteConfiguration breaks non-Talos nodes
**Issue**: `cozystack-talos` LinstorSatelliteConfiguration removes DRBD
init containers (`drbd-module-loader`, `drbd-shutdown-guard`). These are
required on Ubuntu/Debian where DRBD modules aren't pre-loaded.
**Solution**: Add `talos.enabled` value (default: true) and wrap the
configuration in `{{- if .Values.talos.enabled }}`.
**Why this approach**: Conditional rendering is cleaner than node
selectors (Talos has no unique default label) and maintains backwards
compatibility.
## Problem 5: linstor-scheduler image tag invalid for k3s
**Issue**: k3s reports version as `v1.35.0+k3s1`. The `+` character is
invalid in Docker image tags, causing `InvalidImageName` error for
kube-scheduler.
**Solution**: Strip distribution suffix using `regexReplaceAll "\\+.*$"`
in the helper template.
**Why this approach**: k3s version `v1.35.0+k3s1` is based on upstream
`v1.35.0`, so stripping the suffix produces the correct upstream image
tag. This is automatic and doesn't require manual version specification.
## Problem 6: No bundle for non-Talos deployments
**Issue**: `isp-full` bundle always uses Talos-specific networking and
doesn't pass `talos.enabled=false` to linstor.
**Solution**: Add `isp-full-generic` bundle variant that uses
`kubeovn-cilium-generic` networking and passes `talos.enabled=false` to
linstor.
**Why this approach**: Dedicated bundle variant makes deployment
straightforward — users just set `bundle-name: isp-full-generic` without
needing post-install patches.
## Files Changed
| Path | Change |
|------|--------|
| `packages/core/platform/values.yaml` | Add apiServer, cilium.cgroup,
kubeovn.MASTER_NODES config |
| `packages/core/platform/sources/networking.yaml` | Add generic
variants |
| `packages/core/platform/templates/bundles/system.yaml` | Add
isp-full-generic bundle |
| `packages/system/linstor/values.yaml` | Add talos.enabled |
| `packages/system/linstor/templates/satellites-talos.yaml` |
Conditional rendering |
| `packages/system/linstor-scheduler/.../templates/_helpers.tpl` | Strip
version suffix |
| `packages/system/linstor-scheduler/.../templates/deployment.yaml` |
Use helper for image tag |
## Testing
Tested on k3s v1.35.0+k3s1 with Ubuntu 24.04 LTS. All HelmReleases
deploy successfully without post-install patches.
## Breaking Changes
None. Existing Talos deployments continue to work unchanged with default
values.
---
Closes#1933
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added support for generic (non-Talos) Kubernetes clusters with new
isp-full-generic platform bundle
* Introduced generic and hosted deployment variants for
cozystack-operator
* Added generic networking variants with Cilium and KubeOVN support
* Expanded IAAS, PAAS, and NAAS bundle support for generic deployments
* Improved backup retention configuration
* **Bug Fixes**
* Fixed Kubernetes version handling to work with distribution-specific
suffixes
* **Chores**
* Made LINSTOR Talos satellite support optional
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add external ips (LoadBalancer services with assigned IP address) count
to the Tenant details page
<img width="2560" height="1330" alt="external ip count on tenant details
page"
src="https://github.com/user-attachments/assets/f2256629-3815-4892-b80d-e150d0fd3908"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] add external ips count (count of services of type LoadBalancer with assigned ip) to the Tenant details page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* External IPs count now shown in the Tenant application Details tab — a
left-column item displays the number of external IPs for LoadBalancer
services.
* Application status now exposes an External IPs Count metric for Tenant
applications, so UI and APIs can surface the number of LoadBalancer
external IPs without changing other behaviors.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Fix `Used` column to show correct data
- Removes table from `Info` page
- Add patch to let flatMap work with jsonpath
<img width="1800" height="940" alt="resource quota table example"
src="https://github.com/user-attachments/assets/421d1d32-fdc8-4269-838f-b02db0ed6712"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] fix resource quota table on tenant details page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed resource quota data fetching for tenant resources to use correct
namespace references from status fields.
* **Refactor**
* Refined dashboard layout for resource quota displays and improved
custom field resolution with enhanced dynamic key mapping for complex
data structures.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Removed reference to .Values.networking.apiServer which was removed
from values.yaml. Use empty defaults and let ConfigMap or
apiServerEndpoint parsing override them.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Extract common system packages shared between isp-full and isp-full-generic
into a helper template `cozystack.platform.system.common-packages`.
Packages moved to helper:
- kubeovn-webhook, kubeovn-plunger, cozy-proxy
- multus, metallb, reloader
- linstor-scheduler, snapshot-controller
Packages NOT in helper (differ between variants):
- networking (variant differs: kubeovn-cilium vs kubeovn-cilium-generic)
- linstor (talos.enabled differs)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Per review feedback, create separate manifest files for different
deployment targets instead of using templated values:
- cozystack-operator.yaml (Talos): hardcoded localhost:7445 (KubePrism)
- cozystack-operator-generic.yaml: reads from cozystack-operator-config
ConfigMap (user must create before applying)
- cozystack-operator-hosted.yaml: no env override (uses in-cluster SA)
This keeps installation flow clean - users apply the manifest matching
their deployment target without needing to modify original files.
Build system updated to generate all three variants.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Remove apiServer and cilium.cgroup from user-facing values.yaml
- Hardcode Talos-specific values (localhost:7445, cgroup autoMount: false)
directly in isp-full bundle template
- isp-full-generic already hardcodes its own values
- Improve MASTER_NODES comment explaining helm lookup behavior
This hides implementation details from users as requested in review.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The CRD upgrade pre-install job has image compatibility issues.
CRDs are installed as part of the Helm chart install anyway.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
rancher/kubectl is a minimal image without shell.
alpine/k8s includes kubectl and shell for running upgrade jobs.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Bitnami images are forbidden and bitnamilegacy/kubectl:1.35 doesn't exist.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The enabledPackages list must use fully qualified package names
(e.g., cozystack.velero) to match the helper template check.
Also add backupstrategy-controller which is required by backup-controller.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add velero to enabledPackages list for isp-full-generic to enable
backup functionality by default.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Allow iaas, paas, naas bundles when using isp-full-generic variant
- Enable all bundles by default in values-isp-full-generic.yaml
- Update bundle templates to accept isp-full-generic alongside isp-full
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The cozystack-values secret was using the default publishing.host value
from values.yaml (example.org) instead of reading from the cozystack
ConfigMap where the actual root-host is configured.
Add ConfigMap lookup to apps.yaml to read root-host from cozystack
ConfigMap, falling back to chart values if not present.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Same issue as lineage-controller-webhook: DaemonSet uses hardcoded
nodeSelector with empty value, but k3s/kubeadm use value "true".
Changes:
- Add nodeSelector to cozystack-api values (default: empty for Talos)
- Update deployment template to use configurable nodeSelector
- Pass nodeSelector for both cozystack-api and lineage-controller-webhook
in isp-full-generic bundle variant
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
On Talos, control-plane nodes have label node-role.kubernetes.io/control-plane
with empty value. On generic k8s (k3s, kubeadm), the same label has value "true".
The lineage-controller-webhook DaemonSet was hardcoded to use empty value,
causing 0 pods scheduled on k3s clusters.
Changes:
- Add nodeSelector to lineage-controller-webhook values (default: empty for Talos)
- Update DaemonSet template to use configurable nodeSelector
- Pass nodeSelector with value "true" for isp-full-generic bundle variant
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
k3s and kubeadm set node-role.kubernetes.io/control-plane=true,
while Talos uses an empty value. KubeOVN node selector needs
the exact label value to match.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
For isp-full-generic bundle variant, the template now looks up the
cozystack ConfigMap to read cluster-specific settings instead of only
relying on chart default values.
This fixes the issue where Cilium was trying to connect to localhost:7445
(Talos KubePrism default) instead of the actual API server endpoint.
Values read from ConfigMap:
- api-server-endpoint: parsed to extract host/port for Cilium
- ipv4-pod-cidr, ipv4-pod-gateway, ipv4-svc-cidr, ipv4-join-cidr: for KubeOVN
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Remove trailing dash from {{- end -}} constructs in _helpers.tpl to
preserve trailing newlines after each rendered package YAML document.
Without this fix, consecutive packages are rendered without proper
separators:
variant: default---
apiVersion: cozystack.io/v1alpha1
This causes YAML parse errors: "mapping key 'apiVersion' already
defined" because the YAML parser sees a single document with duplicate
keys instead of two separate documents.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The isp-full-generic variant should have iaas, paas, and naas bundles
disabled by default since these bundles require 'isp-full' or 'isp-hosted'
variants as per validation rules in their respective templates.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Create dedicated values file for isp-full-generic bundle variant with
bundles.system.variant set to "isp-full-generic" instead of "isp-full".
This fixes the ArtifactGenerator generating identical digests for both
isp-full and isp-full-generic variants, which caused the wrong bundle
template to be selected.
Also adds isp-full-generic variant to installer PackageSource definition.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The isp-full-generic bundle now automatically extracts host and port
from publishing.apiServerEndpoint URL for Cilium and KubeOVN
configuration. This eliminates the need for manual networking.apiServer
values when api-server-endpoint is set in the ConfigMap.
Changes:
- Parse apiServerEndpoint URL to extract host and port for Cilium
- Auto-detect MASTER_NODES for KubeOVN from apiServerEndpoint if not
explicitly set
- Hardcode cgroup.autoMount=true for generic k8s (non-Talos)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add isp-full-generic bundle variant for deploying Cozystack on non-Talos
Kubernetes clusters (kubeadm, k3s, RKE2, etc.).
This variant:
- Uses kubeovn-cilium-generic networking (no values-talos.yaml)
- Passes talos.enabled=false to linstor (keeps DRBD init containers)
- Supports all standard networking configuration via platform values
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add cilium-generic and kubeovn-cilium-generic variants that exclude
values-talos.yaml. These variants are suitable for kubeadm, k3s, RKE2
and other non-Talos Kubernetes distributions.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
k3s and RKE2 include distribution suffixes in version string
(e.g., v1.35.0+k3s1) which are not valid container image tags.
Strip everything after '+' using regexReplaceAll to produce clean
version tags like v1.35.0.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add talos.enabled value (default: true) to control whether
LinstorSatelliteConfiguration cozystack-talos is created.
This allows deploying linstor on non-Talos clusters (Ubuntu, Debian, etc.)
where DRBD init containers are required.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Add networking.kubeovn.MASTER_NODES to platform values.yaml
- Conditionally include MASTER_NODES in kubeovn values when set
- Enables KubeOVN deployment on k3s/kubeadm without control-plane node labels
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add networking.apiServer.host and networking.apiServer.port to platform values
to allow overriding the default Talos KubePrism settings (localhost:7445).
Also add networking.cilium.cgroup.autoMount for non-Talos clusters that need
Cilium to mount cgroups automatically.
Changes:
- packages/core/platform/values.yaml: Add apiServer and cilium.cgroup settings
- packages/core/platform/templates/bundles/system.yaml: Pass cilium values
- packages/core/installer/values.yaml: Add kubernetesServiceHost/Port
- packages/core/installer/templates/cozystack-operator.yaml: Template env vars
Closes: #1933
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Fixes v1.0 migration script
### Release note
```release-note
V1.0 migration script fixed after an upgrade was performed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated bundle naming convention (paas → isp).
* Restructured configuration schema: flattened branding, renamed
scheduling field, and moved authentication redirect setting.
* Minor output text and prompt wording adjustments.
* **New Features**
* Added support for external IPs in generated configuration with safe
empty fallback.
* Consolidated branding and scheduling entries for simpler output.
* **Bug Fixes**
* Migration script now validates prerequisites and produces improved,
YAML-friendly output.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add migration script to convert existing MongoDB HelmReleases
from old users format to new users + databases format.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Add Tenant details page to dashboard
- Add Resource Quota usage to Tenant page in dashboard
<img width="1800" height="939" alt="resource quota in tenant"
src="https://github.com/user-attachments/assets/7f4f6ed6-e9b8-4258-a60b-15f203443a1f"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Add Tenant details page and resource quota usage to tenant page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Resource Quotas section to the Details tab to show allocation
limits and usage.
* Info and Tenant types now display quota data in an interactive table
with columns for resource, limits, and current usage.
* Enhanced quota table presentation with additional columns for
flattened resource keys and values for clearer reporting.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add "Edit" button to all resource pages in dashboard
fixes https://github.com/cozystack/cozystack/issues/1911
<img width="1974" height="1327" alt="edit button"
src="https://github.com/user-attachments/assets/8e28dd2d-b474-488b-a7ae-a6c60d23619f"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Add "Edit" button to all resources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Edit action links to dashboard headers for quick access to edit
functionality.
* **UI/Layout**
* Improved dashboard header layout with enhanced spacing and center
alignment for better visual consistency.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes v1.0 packagesources.
### Release note
```release-note
V1.0 packagesources fixed after an upgrade was performed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated platform component configurations for cozy-proxy and
metrics-server deployments.
* Reorganized monitoring component namespace allocation.
* Added metrics server as a dependency in the monitoring pipeline.
* Integrated new system components (linstor-scheduler and
metrics-server) into the platform templates.
* Refined HelmRelease chart references for vertical pod autoscaler.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Align MongoDB configuration with postgres and mysql patterns:
- Add separate `databases` section with `roles.admin` and `roles.readonly`
- Simplify `users` to only contain optional password field
- All users now authenticate via `admin` database (MongoDB best practice)
- Admin role maps to readWrite + dbAdmin permissions
- Readonly role maps to read permission
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Add missing changelog documentation for recent patch releases that were
released without changelog entries.
### Release note
```release-note
[docs] Add changelog documentation for v0.40.3, v0.41.1, v0.41.2, v0.41.3
```
Add missing changelog documentation for recent patch releases.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Fixes telemetry collection in cozystack-operator which was failing
because the manager's cache is filtered and doesn't include resources
needed for telemetry (kube-system namespace, nodes, services, PVs).
Switches to using `mgr.GetAPIReader()` which bypasses the cache.
### Release note
```release-note
[system] Fix telemetry collection in cozystack-operator
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated Docker image versions from 1.5.0 to 1.6.1 for platform
migrations and testing environments.
* Enhanced telemetry collection system with improved data access
patterns.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch enables the `offline_access` scope for the dashbord keycloak
client, so that users get a refresh token which gatekeeper can use to
automatically refresh an expiring access token. Also session timeouts
were increased.
### Release note
```release-note
[dashboard] Increase session timeouts, add the offline_access scope,
enable refresh tokens to improve the overall user experience when
working with the dashboard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated authentication configuration with additional OAuth2 scope
parameters to enhance security credentials handling.
* Enhanced Keycloak client configuration with optional client scopes and
session management attributes, including logout redirect settings and
session timeout controls for improved session lifecycle management.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cozystack-operator's manager cache is filtered to only include
specific secrets and namespaces with certain labels. This caused
telemetry collection to fail because resources like kube-system
namespace, nodes, services, and PVs were not in the cache.
Switch to using mgr.GetAPIReader() which bypasses the cache and
queries the API server directly. This is appropriate for telemetry
since it only runs every 15 minutes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Introduce a new "External IPs" sidebar and associated dashboard factory
to display services of type LoadBalancer.
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add External IPs tab to sidebar with all `Service` resources of type
`LoadBalancer` in tenant
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] add External IPs tab with `Service` resources of type `LoadBalancer`
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added External IPs menu item in the Administration section for quick
access to load balancer services
* Introduced a new dashboard view displaying external IPs with tabbed
interface for enhanced visibility
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add labels and selectors for Service and Ingress resources to correctly
show them in corresponding tabs for Kubernetes app in dashboard
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] show Ingress and Service resources in dashboard
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Introduced standardized metadata labels across Kubernetes application
resources including cloud configuration, ingress, and service components
to improve resource organization, tracking, identification, and
multi-tenant environment management
* Extended the system's resource management framework to include and
manage additional Kubernetes infrastructure components, such as
ingress-nginx, enhancing overall cluster orchestration and service
networking capabilities
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Fixes filtering pods on Pods tab of Service details page
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Fix filtering pods on Pods tab for Service
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected label selector field name in pod details table configuration
to ensure proper label filtering functionality.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This patch enables the `offline_access` scope for the dashbord keycloak
client, so that users get a refresh token which gatekeeper can use to
automatically refresh an expiring access token. Also session timeouts
were increased.
### Release note
```release-note
[dashboard] Increase session timeouts, add the offline_access scope,
enable refresh tokens to improve the overall user experience when
working with the dashboard.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Introduce a new "External IPs" sidebar and associated dashboard factory to display services of type LoadBalancer.
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Set minReplicas for VPA for VMAgent to work correctly in the default
case when replicas = 1
Fixes https://github.com/cozystack/cozystack/issues/1893
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[monitoring-agents] set minReplicas=1 in VPA for cluster-wide VMAgent to allow recrating single replica
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Changes
* **Chores**
* Updated system monitoring autoscaler configuration to enforce minimum
replica constraints, enhancing service resilience and availability
during scaling operations.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This reverts commit 73d6e3013e.
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Revert wrong type change according review
[request](https://github.com/cozystack/cozystack/pull/1873#discussion_r2717088708)
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[fix strategyRef type in backups]
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Breaking Changes**
* Backup strategy references have been updated from cluster-scoped to
namespace-scoped references. This change enables better namespace
isolation and control. Update your backup configurations to reference
strategies within the appropriate namespace.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes the ability for users to specify custom container images
(`images.pmm` and `images.backup`) in the MongoDB application values.
This is a security hardening measure - allowing users to specify
arbitrary container images could lead to:
- Running malicious or compromised images
- Supply chain attacks through untrusted image registries
- Privilege escalation if malicious images are designed to exploit the
cluster
The images are now hardcoded in the template:
- `percona/pmm-client:2.44.1`
- `percona/percona-backup-mongodb:2.11.0`
### Release note
```release-note
[apps] Remove user-configurable images from MongoDB chart for security hardening
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Removed container image customization options for MongoDB's monitoring
and backup components. The system now uses fixed, pre-determined image
versions (PMM Client 2.44.1 and Percona Backup MongoDB 2.11.0) to ensure
deployment consistency and reduce configuration complexity.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add documentation for the resourceQuotas parameter explaining
supported resource types and their usage.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
Closes#1880
```release-note
Added BackupClass into BackupJob and BackupPlan for simplify UX.
```
The following functions related StorageRef were removed as they are no
longer used after migrating to BackupClass API:
// - resolveBucketStorageRef: Previously resolved S3 credentials from
Bucket storageRef
// - createS3CredsForVelero: Previously created Velero S3 credentials
secrets
// - createBackupStorageLocation: Previously created Velero
BackupStorageLocation resources
// - createVolumeSnapshotLocation: Previously created Velero
VolumeSnapshotLocation resources
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added BackupClass CRD and backupClassName field; webhook enforces and
defaults/validates backupClassName.
* **Refactor**
* Removed inline storageRef/strategyRef from specs; controllers resolve
strategy and parameters from BackupClass. Application apiGroup defaults
to apps.cozystack.io. Velero flow now uses resolved parameters.
* **Tests**
* Added unit tests for BackupClass resolution, BackupJob/Plan factory,
webhook validation, and Velero template rendering.
* **Chores**
* Manager cache/indexing updated to track BackupClass and index
BackupJobs by backupClassName.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Removes Tenant `isolated` flag and enforces isolation using network
policies
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[tenant] Removed Tenant field `isolated` and enforced network isolation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Network policies for tenant namespace isolation are now always
enforced and cannot be disabled.
* Removed the `isolated` configuration parameter from all tenant
settings, schema definitions, and documentation.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove the ability for users to specify custom container images
(images.pmm and images.backup) in the MongoDB application values.
This is a security hardening measure - allowing users to specify
arbitrary container images could lead to running malicious or
compromised images, supply chain attacks, or privilege escalation.
The images are now hardcoded in the template:
- percona/pmm-client:2.44.1
- percona/percona-backup-mongodb:2.11.0
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add enum validation for addon `ingressNginx.exposeMethod` field
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] add enum validation for ingressNginx `exposeMethod` field
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Enhanced validation for Kubernetes ingress configuration to enforce
allowed exposure methods ("Proxied" or "LoadBalancer"). This ensures
configurations are properly restricted to supported options and prevents
invalid settings from being accepted.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[ci] Run e2e tests on shared runners
```
This PR prepares the release `v1.0.0-alpha.2`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container images and system components across all major
platform services. Includes infrastructure upgrades for networking,
storage systems, application APIs, data backup services, system
monitoring, and supporting components. All updates incorporate the
latest security patches, critical stability improvements, and
performance optimizations to enhance overall system reliability,
operational efficiency, and platform stability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- Add Feature Highlights section with MongoDB as the main feature of
v0.41.0
- Add Documentation section with website repository changes to both
v0.41.0 and v1.0.0-alpha.2
- Add @matthieu-robin to v1.0.0-alpha.2 contributors
Fixes#749
## Changes
- Add documentation explaining how to enable Hubble for network
observability in Grafana
- Add four Grafana dashboards for Hubble metrics visualization
(Overview, DNS, L7 HTTP, Network)
- Register Hubble dashboards in the monitoring package
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced Hubble DNS Overview dashboard for namespace-level DNS
metrics visibility.
* Introduced Hubble L7 HTTP Metrics dashboard for HTTP request and
performance monitoring.
* Introduced Hubble Network Overview dashboard for network flows and
policy insights.
* **Documentation**
* Added comprehensive Hubble observability guide covering setup,
configuration, and troubleshooting.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds MongoDB as a managed application to Cozystack using Percona
Operator for MongoDB (Apache 2.0).
**New packages:**
- `packages/system/mongodb-operator/` - Percona psmdb-operator
deployment
- `packages/apps/mongodb/` - MongoDB application chart
- `packages/system/mongodb-rd/` - CozystackResourceDefinition for
dashboard
**Features:**
- Replica set mode with configurable replica count
- Sharded cluster mode (config servers, mongos routers, multiple shards)
- S3 backup integration with cron scheduling
- Point-in-time recovery from backups
- External access via LoadBalancer
- Multiple MongoDB versions (v6, v7, v8)
- Resource presets (nano to 2xlarge)
- User management with role-based access
### Release note
```release-note
[apps] Add MongoDB as a managed application with replica set, sharding, and S3 backup support
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Managed MongoDB service and operator packaging; replica sets with
optional sharding, external access, per-user credentials, workload
monitoring, and version mappings for provider images
* Backup & restore with S3 support and point-in-time recovery
* **Documentation**
* Comprehensive README plus Helm values and schema for configuration
* **Tests**
* Extensive unit/template tests and end-to-end tests covering
credentials, services, backup, restore, CR rendering, and operator
install
* **Packaging**
* Charts and bundle entries added for operator and managed service
installation
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kilo] Introduce kilo
```
## What this PR does
This PR introduces a new flux-plunger controller that automatically
fixes HelmRelease resources experiencing "has no deployed releases"
errors by cleaning up corrupted Helm release secrets.
### Problem
When Helm releases fail with "has no deployed releases" error, FluxCD
cannot recover automatically. This typically happens when:
- Helm release secrets are corrupted
- Failed upgrades leave the release in an inconsistent state
- Manual intervention is required to delete the problematic secret
### Solution
The flux-plunger controller:
1. Watches HelmRelease resources for "has no deployed releases" errors
2. Suspends the HelmRelease (with `flux-client-side-apply` field
manager)
3. Identifies and deletes the latest (problematic) Helm release secret
4. Tracks processed versions via annotation to prevent recursive
deletion
5. Unsuspends the HelmRelease to allow FluxCD to retry
**Key features:**
- Only deletes one secret at a time (prevents cascading deletions)
- Uses version tracking: if `latest+1 == processed`, skips (already
handled)
- Handles suspended HelmReleases: only unsuspends if it was suspended by
flux-plunger itself
- Gracefully handles invalid annotations by treating them as unprocessed
### Components
- **Controller:** `internal/controller/fluxplunger/flux_plunger.go`
- **Main:** `cmd/flux-plunger/main.go`
- **Helm Chart:** `packages/system/flux-plunger/`
- **Platform Integration:** Added to `paas-full.yaml` bundle
### Release note
```release-note
[platform] Add flux-plunger controller to automatically fix HelmRelease resources with "has no deployed releases" error by cleaning up corrupted Helm secrets
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced a new controller that automatically resolves HelmRelease
resource failures with built-in health probes and graceful recovery
procedures.
* Enabled by default in system installations.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds a new package `local-ccm` to the Cozystack platform.
The local-ccm package provides a Cloud Controller Manager for local
development and testing environments. It integrates the upstream Helm
chart from https://github.com/cozystack/local-ccm (version 0.2.0).
**Changes:**
- Created package structure in `packages/system/local-ccm/`
- Added Makefile with automated upstream chart synchronization from
GitHub
- Added PackageSource definition in
`packages/core/platform/sources/local-ccm.yaml`
- Configured dependency on networking components
- Set privileged mode as required for cloud controller operations
**Package details:**
- Namespace: `cozy-local-ccm`
- Release name: `local-ccm`
- Upstream: https://github.com/cozystack/local-ccm
### Release note
```release-note
[local-ccm] Add local-ccm package for cloud controller manager in local environments
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Local Cloud Controller Manager package providing automatic node
IP detection, internal and external IP support, initialization taint
removal, and continuous node reconciliation with configurable intervals.
* **Chores**
* Updated system component tolerations and scheduling policies to
improve compatibility with cloud infrastructure, including better
handling of node readiness, network initialization, and cloud provider
states.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes the redundant `cozystack.io/ui=true` label from HelmReleases.
The existing `apps.cozystack.io/application.kind`,
`apps.cozystack.io/application.group`, and
`apps.cozystack.io/application.name` labels already provide more
specific identification and are sufficient for filtering.
Updates all label selectors to use `apps.cozystack.io/application.kind`
instead.
### Release note
```release-note
[platform] Remove cozystack.io/ui label - use apps.cozystack.io/application.kind for filtering instead
```
## What this PR does
Splits telemetry collection between cozystack-operator and
cozystack-controller for better separation of concerns.
### Release note
```release-note
[platform] Split telemetry collection between cozystack-operator and cozystack-controller
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added telemetry support to the operator with configurable options
(disable, endpoint, interval).
* Enhanced telemetry to track application kinds and UI-managed releases.
* **Bug Fixes**
* Removed the runtime cozystack-version flag; version is now managed at
build time.
* **Chores**
* Removed unused YAML dependency.
* Simplified build process with improved Makefile commands.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
- Fix view of loadbalancer ip in services window
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed LoadBalancer IP column display to correctly retrieve IP
addresses from the updated data source in the dashboard details view.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases the `startupProbe.failureThreshold` for kube-apiserver
container from 3 to 30.
The default 30 seconds (3 failures × 10s period) is often insufficient
for kube-apiserver to complete RBAC bootstrap, especially:
- Under high load conditions
- With slower etcd connections
- During initial cluster provisioning
This change gives kube-apiserver up to 300 seconds to initialize,
preventing unnecessary CrashLoopBackOff cycles.
### Release note
```release-note
[kubernetes] Increase kube-apiserver startup probe threshold to prevent CrashLoopBackOff during slow RBAC initialization
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated startup probe threshold configuration to enhance system
initialization stability across core components.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Changes the default `resourcesPreset` for kube-apiserver from "medium"
to "large" to prevent OOM kills during normal operation. The "medium"
preset provides insufficient memory for clusters with moderate
workloads, leading to repeated crashes and service disruption.
### Release note
```release-note
[kubernetes] Change default apiServer resourcesPreset from "medium" to "large" to prevent OOM kills
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Configuration Changes**
* Updated default API Server resource preset from medium to large for
enhanced resource allocation
* Removed version property from Kubernetes chart configuration schema
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add new package local-ccm from https://github.com/cozystack/local-ccm
- Create package structure in packages/system/local-ccm
- Add Makefile with upstream chart sync from GitHub
- Add PackageSource definition in packages/core/platform/sources/
- Package provides cloud controller manager for local environments
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The apps.cozystack.io/application.kind, apps.cozystack.io/application.group,
and apps.cozystack.io/application.name labels already exist on all HelmReleases
and provide more specific identification. Remove the redundant cozystack.io/ui
label and update all selectors to use apps.cozystack.io/application.kind instead.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add MongoDB managed service based on Percona Operator for MongoDB with:
- Replica set mode (default) and sharded cluster mode
- Configurable replicas, storage, and resource presets
- Custom users with role-based access control
- S3-compatible backup with PITR support
- Bootstrap/restore from backup
- External access support
- WorkloadMonitor integration for dashboard
- Comprehensive helm-unittest test coverage (91 tests)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Increase startupProbe.failureThreshold from 3 to 30 for kube-apiserver
container. The default 30 seconds (3 failures * 10s period) is often
insufficient for apiserver to complete RBAC bootstrap, especially under
load or with slower etcd connections.
This change gives kube-apiserver up to 300 seconds to initialize,
preventing unnecessary CrashLoopBackOff cycles.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add flux-plunger controller to automatically fix HelmRelease resources
with "has no deployed releases" error.
The controller watches HelmRelease resources and performs the following:
- Detects HelmRelease with "has no deployed releases" error
- Suspends the HelmRelease with flux-client-side-apply field manager
- Deletes the latest Helm release secret
- Updates annotation with processed version to prevent recursive deletion
- Unsuspends the HelmRelease to allow Flux to retry
Special handling for suspended HelmRelease:
- If suspend=true and latest+1==processed: removes suspend
- Otherwise: skips processing as suspended by external process
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR updates Talos Linux which includes DRBD fixes:
- https://github.com/LINBIT/drbd/releases/tag/drbd-9.2.15
- https://github.com/LINBIT/drbd/releases/tag/drbd-9.2.16
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Update Talos Linux v1.11.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Bumped Talos profile version from v1.11.3 to v1.12.1 across all
configuration profiles
* Updated baseInstaller and system extensions with latest firmware and
driver images
* Added output format settings to profile configurations
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases etcd probe thresholds to allow more time for cluster
synchronization after pod restarts. This addresses issues where etcd
members were being killed by startup probes before they could fully sync
with the cluster, especially when VPA assigns minimal resources.
Changes:
- `startupProbe.failureThreshold`: 3 → 300 (allows 25 minutes for
initial sync)
- `livenessProbe.failureThreshold`: default → 10 (reduces unnecessary
restarts)
- `readinessProbe.failureThreshold`: default → 3
### Release note
```release-note
[etcd] Increase probe thresholds to prevent premature pod termination during cluster synchronization
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved etcd service stability with automatic health monitoring and
failover detection capabilities in Kubernetes environments.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Change the default resourcesPreset for kube-apiserver from "medium" to
"large" to prevent OOM kills during normal operation. The "medium"
preset provides insufficient memory for clusters with moderate workloads.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Increase startup probe failureThreshold to 300 (25 minutes) to allow
etcd members more time to sync with the cluster after restart or
recovery. This prevents pods from being killed during initial
synchronization when VPA assigns minimal resources.
Also increase liveness probe failureThreshold to 10 to reduce
unnecessary restarts during temporary network issues.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Updates cilium component to version 1.18.6
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cilium] update to v1.18.6
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded Cilium to v1.18.6 and updated container images and checksums
across components (Hubble Relay, Operator, Preflight, Cluster Mesh API
Server, SPIRE, etc.).
* Updated Envoy to v1.35.9 with new image digest.
* **Bug Fixes / Behavior**
* Adjusted preflight/Envoy gating: preflight-enabled now suppresses
certain Envoy bootstrap config and removed the preflight Envoy bootstrap
config mount.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes aggregated API server Watch implementation to properly work with
controller-runtime informers. External controllers watching Tenant
resources via the aggregated API were experiencing issues with cache
sync timeouts and missing reconciliations on startup.
### Changes
**ResourceVersion handling in List:**
- Compute ResourceVersion from items when the cached client doesn't set
it on the list itself
**Bookmark handling:**
- Pass through bookmark events with converted types for proper informer
sync
**ADDED event filtering (main fix):**
- Simplified the filtering logic that was incorrectly skipping initial
events
- Only skip ADDED events when `startingRV > 0` AND `objRV <= startingRV`
(client already has from List)
- When `startingRV == 0`, always send ADDED events (client wants full
state)
- Removed the complex `initialSyncComplete` tracking that had inverted
logic
### Problem
When a controller starts watching resources, controller-runtime may call
Watch with `resourceVersion=""`. The server should send all existing
objects as ADDED events. The previous `initialSyncComplete` logic was
inverted and could skip these events, causing objects (like Tenants with
lock annotations) to not be reconciled on controller startup.
### Release note
```release-note
[apiserver] Fix Watch resourceVersion and bookmark handling for controller-runtime compatibility
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Enhanced resource watching and synchronization with Kubernetes 1.27+
compatibility, including proper handling of initial events and
bookmarks.
* Optimized event filtering and resource version tracking for
applications, tenant modules, tenant namespaces, and tenant secrets to
reduce unnecessary event noise.
* Improved list metadata consistency by deriving accurate resource
versions when unavailable in responses.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Add resourceVersion handling for Watch requests by filtering ADDED events
based on the resourceVersion provided by the client
- Forward bookmark events from underlying HelmRelease watchers to clients
for proper resourceVersion synchronization
- Extract MaxResourceVersion helper using meta.EachListItem for cleaner code
- This ensures clients don't receive duplicate objects they already have
from List+Watch patterns
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Update changelog for v1.0.0-alpha.1 to include missing features:
- **Cozystack Operator**: New operator for Package and PackageSource
management (#1740, #1741, #1755, #1756, #1760, #1761)
- **Backup System**: Comprehensive backup functionality with Velero
integration (#1640, #1685, #1687, #1708, #1719, #1720, #1737, #1762)
- Add @androndo to contributors
- Update Full Changelog link to v0.38.0...v1.0.0-alpha.1
### Release note
```release-note
[docs] Update changelog for v1.0.0-alpha.1: add cozystack-operator and backup system
```
This PR prepares the release `v1.0.0-alpha.1`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated container images for core system components to v1.0.0-alpha.1
* Upgraded Cilium networking from 1.17.8 to 1.18.5
* Updated infrastructure component images including Metallb, Linstor,
and storage services with latest digests for enhanced security and
stability
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add file existence checks to talos asset targets to prevent
rebuilding kernel, initramfs, installer, iso, nocloud, and metal
assets when they already exist from a previous build step.
This fixes duplicate talos builds in the release pipeline where
`make build` and `make assets` both triggered talos image generation.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Renames `CozystackResourceDefinition` CRD to `ApplicationDefinition` for
better clarity and consistency.
Changes include:
- Renamed all Go types (ApplicationDefinition,
ApplicationDefinitionList, ApplicationDefinitionSpec, etc.)
- Renamed controller files and reconciler structs
- Updated all packages that use these types (dashboard,
lineagecontrollerwebhook, crdmem, etc.)
- Renamed CRD Helm chart from `cozystack-resource-definition-crd` to
`application-definition-crd`
- Updated all 25 cozyrds YAML manifests to use `kind:
ApplicationDefinition`
- Added migration 23 to remove old `cozystack-resource-definition-crd`
HelmRelease
### Release note
```release-note
[api] Rename CozystackResourceDefinition CRD to ApplicationDefinition
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Renamed the primary custom resource from CozystackResourceDefinition
to ApplicationDefinition across the platform, dashboards, controllers,
and manifests.
* Updated packaging and Helm charts to use ApplicationDefinition
consistently.
* **Chores**
* Bumped platform migration target to v24 and added a migration to apply
the new resource CRD.
* **Bug Fix**
* Restored deep-copy support for Component-related types.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Rename the CRD and all related types for better clarity:
- CozystackResourceDefinition -> ApplicationDefinition
- CozystackResourceDefinitionList -> ApplicationDefinitionList
- CozystackResourceDefinitionSpec -> ApplicationDefinitionSpec
- All related nested types updated accordingly
Updated components:
- API types and generated deepcopy code
- Controllers and reconcilers
- Dashboard, lineagecontrollerwebhook, crdmem packages
- CRD YAML definition and Helm chart
- All 25 cozyrds YAML manifests
- Migration scripts and documentation
Added migration 23 to remove old cozystack-resource-definition-crd HelmRelease.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
- Add pattern rule for building cozypkg binaries per platform
(`assets-cozypkg-<os>-<arch>`) with checksums generation
- Add Version variable to cozypkg CLI injected via ldflags
- Split manifests into separate `cozystack-crds.yaml` and
`cozystack-operator.yaml` files
- Update CI workflow to handle CRDs and operator as separate artifacts
- Update e2e tests and upload script for new manifest structure
### Release note
```release-note
[ci] Add cross-platform cozypkg build targets with version injection and split installer manifests into CRDs and operator
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Multi-platform binaries (Linux, macOS, Windows; amd64 & arm64)
* Checksum file added for release artifact verification
* **Refactor**
* Installation flow reworked to validate and apply CRDs and Operator
manifests separately
* CLI exposes a configurable build-time version for reporting
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Add pattern rule for building cozypkg binaries per platform
(assets-cozypkg-<os>-<arch>) with checksums generation
- Add Version variable to cozypkg CLI injected via ldflags
- Split manifests into separate cozystack-crds.yaml and
cozystack-operator.yaml files
- Update CI workflow to handle CRDs and operator as separate artifacts
- Update e2e tests and upload script for new manifest structure
- Suppress git describe stderr when no tags exist
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
- Move `common-envs.mk` and `package.mk` from `scripts/` to `hack/`
directory
- Update all Makefile includes to use new paths
- Remove unused `issue-flux-certificates.sh` script
### Release note
```release-note
[refactor] Move build scripts from scripts/ to hack/ directory
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Reorganized build infrastructure by consolidating helper scripts and
configuration paths.
* Removed deprecated script from the build tooling.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Move common-envs.mk and package.mk from scripts/ to hack/ directory.
Update all Makefile includes to use new paths. Remove unused
issue-flux-certificates.sh script.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Replace HelmRelease-based bundle system with Package resources managed
by cozystack-operator. Restructure values.yaml with full configuration
support.
## What this PR does
- Restructure values.yaml with full configuration (networking,
publishing, authentication, scheduling, branding, resources)
- Add values-isp-full.yaml and values-isp-hosted.yaml for bundle
variants
- Create templates/packages/isp-full.yaml with Package resources
- Move PackageSources from sources/ to templates/sources/
- Remove old bundle files and HelmRelease templates
- Add hack/migrate-to-version-1.0.sh migration script for converting
ConfigMaps to Package resource
### Release note
```release-note
[platform] Migrate from HelmRelease bundles to Package-based deployment
```
Add dedicated flux-tenants controller with label selector
--watch-label-selector=sharding.fluxcd.io/key=tenants to handle
tenant workloads separately from platform components.
Update all kubernetes app HelmReleases to:
- Use chartRef with ExternalArtifact instead of OCIRepository sourceRef
- Add sharding.fluxcd.io/key=tenants label
- Add cozystack.io/target-cluster-name label
Update fluxinstall to parse multiple YAML manifest files and
use flux service for storage-adv-addr.
Add cozystack-basics package for core tenant/namespace setup.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Restructure platform bundles from monolithic files to modular
directory structure with separate applicationdefinitions.
Add PackageSources for better dependency management and
migrate from legacy HelmRepositories to new repository format.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Update all CozystackResourceDefinition files to use chartRef
with ExternalArtifact instead of OCIRepository sourceRef.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Remove legacy installer components (cozystack-assets-server,
installer.sh script, cozystack container image) in favor of
cozystack-operator based deployment.
Move migration scripts from scripts/migrations/ to
packages/core/platform/images/migrations/ for containerized execution.
Add grafana-dashboards image for centralized dashboard management.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Replace the chart field with chartRef for referencing Helm charts via
ExternalArtifact resources. This enables the Package controller to
manage chart sources centrally.
Changes:
- Add chartRef field to CozystackResourceDefinition spec
- Remove chart field (deprecated)
- Remove validation (moved to controller)
- Update lineage mapper for new field structure
- Regenerate openapi specs
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Fix getVersion to parse "0.1.4+abcdef" format (with "+" separator)
instead of incorrectly looking for "sha256:" prefix.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
upstream PR https://github.com/piraeusdatastore/linstor-csi/pull/403
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[linstor] Refactor node-level RWX validation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a command-line option to disable strict RWX block volume
validation for specialized deployments.
* Centralized RWX validation into a shared utility used by the driver.
* **Improvements**
* Improved VM identification and pod/PV handling for RWX block
attachments, including hotplug scenarios.
* **Tests**
* Added comprehensive unit tests covering RWX validation and related
pod/PV scenarios.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Removes node-level RWX block validation from linstor-csi as
controller-level check is sufficient. The controller already validates
that all pods attached to RWX block volume belong to the same VM by
extracting vmName from pod owner references (VirtualMachineInstance).
This simplifies the validation logic and fixes VM live migration issues.
### Release note
```release-note
[linstor] Remove node-level RWX block validation to fix VM live migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Enhanced RWX (read-write-many) block validation with VM-aware checks
across node and controller flows, including support for hotplug-disk
pods and stricter prevention of cross-VM block sharing.
* Improved propagation and resolution of VM identity for attachments to
ensure consistent validation.
* **Tests**
* Added comprehensive unit tests covering single/multiple pod scenarios,
VM ownership, hotplug disks, upgrade paths, and legacy volumes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR updates piraeus-server patches to address several critical
production issues with DRBD resources and LUKS encryption:
1. **Add fix-duplicate-tcp-ports.diff** - Prevents duplicate TCP ports
after toggle-disk operations (upstream PR #476)
2. **Update skip-adjust-when-device-inaccessible.diff** - Comprehensive
fix for multiple issues:
- Resources stuck in StandAlone state after node reboot
- Unknown state race condition during satellite restart
- Encrypted LUKS resource deletion failures
- Network reconnect blocked by unavailable child device checks
These patches resolve scenarios where DRBD resources fail to
automatically reconnect after node reboots and improve LUKS resource
lifecycle management.
Upstream PRs:
- https://github.com/LINBIT/linstor-server/pull/476
- https://github.com/LINBIT/linstor-server/pull/477
### Release note
```release-note
[linstor] Fix DRBD resources stuck in StandAlone state after reboot and encrypted resource deletion issues
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Prevents duplicate TCP port conflicts after disk toggle operations
* Fixes resources stuck in StandAlone or Unknown state after reboot
* Resolves issues with encrypted resource deletion
* Improves handling of temporarily inaccessible storage devices
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove node-level RWX block validation from linstor-csi as
controller-level check is sufficient. The controller already validates
that all pods attached to RWX block volume belong to the same VM by
extracting vmName from pod owner references (VirtualMachineInstance).
This simplifies the validation logic and fixes VM live migration issues.
Update linstor-csi image tag with rebuilt image containing the fix.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Update piraeus-server patches to address critical production issues:
- Add fix-duplicate-tcp-ports.diff to prevent duplicate TCP ports
after toggle-disk operations (upstream PR #476)
- Update skip-adjust-when-device-inaccessible.diff with comprehensive
fixes for resources stuck in StandAlone after reboot, Unknown state
race condition, and encrypted LUKS resource deletion (upstream PR #477)
```release-note
[linstor] Fix DRBD resources stuck in StandAlone state after reboot and encrypted resource deletion issues
```
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This PR generates missing changelogs for releases v0.37.10,
v0.38.5-v0.38.8, v0.39.2-v0.39.4, and v0.40.0 following the instructions
from `docs/agents/changelog.md`.
### Release note
```release-note
[docs] Add missing changelogs for releases v0.37.10, v0.38.5-v0.38.8, v0.39.2-v0.39.4, and v0.40.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added release notes for versions v0.37.10 through v0.40.0, documenting
improvements to VM storage resize, dashboard form schema, Windows VM
scheduling, SeaweedFS updates, and various platform enhancements and bug
fixes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Generated changelogs for missing releases following changelog.md instructions:
- v0.37.10: Backports for dashboard schema fix and VM resize improvements
- v0.38.5-v0.38.8: Backports for SeaweedFS, Cilium, VM scheduling, and Multus fixes
- v0.39.2-v0.39.4: Backports for VM services, tenant policies, LINSTOR fixes, and Multus dependencies
- v0.40.0: Minor release with LINSTOR scheduler, SeaweedFS traffic locality, valuesFrom mechanism, auto-diskful, and version management systems
All changelogs include proper PR author attribution using GitHub CLI and user impact descriptions.
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Fixes field selector filtering for registry resources (Applications,
TenantModules, TenantSecrets) when using kubectl with field selectors
like `--field-selector=metadata.namespace=tenant-kvaps` or
`metadata.name=test`.
Controller-runtime cache doesn't support field selectors natively, which
caused incorrect filtering behavior. This PR implements manual filtering
for `metadata.name` and `metadata.namespace` field selectors in List()
and Watch() methods.
Changes:
- Created `pkg/registry/fields` package with `ParseFieldSelector`
utility for common field selector parsing
- Refactored field selector logic in application, tenantmodule, and
tenantsecret registries to use the common implementation
- Implemented manual post-processing filtering after label-based queries
- Removed `Raw` field usage and field selectors from
`client.ListOptions`
### Release note
```release-note
[registry] Fix field selector filtering for kubectl queries with metadata.name and metadata.namespace selectors
```
Controller-runtime cache doesn't support field selectors, causing
incorrect filtering when using kubectl with field selectors like
--field-selector=metadata.namespace=tenant-kvaps or metadata.name=test.
Changes:
- Created pkg/registry/fields package with ParseFieldSelector utility
- Refactored field selector parsing logic in application, tenantmodule,
and tenantsecret registries to use common implementation
- Implemented manual filtering for metadata.name and metadata.namespace
in List() and Watch() methods
- Removed Raw field usage and field selectors from client.ListOptions
- Label selectors passed directly via LabelSelector field
Field selectors now properly filter resources by name and namespace
through manual post-processing after label-based filtering.
See: https://github.com/kubernetes-sigs/controller-runtime/issues/612
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Adds multus as a dependency as other CNIs
### Release note
```release-note
Adds multus as a dependency as other CNIs
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated component initialization ordering to ensure more consistent
and reliable platform startup sequencing.
* Streamlined dependency requirements for several core system components
to improve deployment reliability and reduce startup complexity.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds custom linstor-csi image build to the linstor package with
an important upstream patch for RWX block volume validation.
**Changes:**
- Add linstor-csi Dockerfile based on upstream linstor-csi master
- Import patch from upstream PR piraeusdatastore/linstor-csi#403 for RWX
block volume validation
- Update Makefile to build both piraeus-server and linstor-csi images
- Configure LinstorCluster CR to use custom linstor-csi image in CSI
controller and node pods
**RWX Validation Patch:**
The imported patch prevents misuse of DRBD allow-two-primaries by
ensuring RWX block volumes are only used by pods belonging to the same
KubeVirt VM during live migration. This prevents data corruption when
multiple non-related pods attempt to use the same RWX block volume.
**Upstream PR:**
https://github.com/piraeusdatastore/linstor-csi/pull/403
### Release note
```release-note
[linstor] Add custom linstor-csi image build with RWX block volume validation patch that prevents data corruption during KubeVirt live migration
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Integrated LINSTOR CSI with deployable CSI components and node init
helpers; packaged as a production-ready image.
* **Updates**
* Added build/publish flow for Piraeus server and LINSTOR CSI images;
chart values updated with new image references and tags.
* **Behavior changes**
* Node-side RWX block-volume pre-validation to prevent conflicting
multi-node/multi-writer access.
* **Tests**
* Comprehensive RWX validation tests covering multi-pod and VM
scenarios.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add custom linstor-csi image build to packages/system/linstor:
- Add Dockerfile based on upstream linstor-csi
- Import patch from upstream PR #403 for RWX block volume validation
(prevents misuse of allow-two-primaries in KubeVirt live migration)
- Update Makefile to build both piraeus-server and linstor-csi images
- Configure LinstorCluster CR to use custom linstor-csi image in
CSI controller and node pods
The RWX validation patch ensures that RWX block volumes with
allow-two-primaries are only used by pods belonging to the same
KubeVirt VM during live migration.
Upstream PR: https://github.com/piraeusdatastore/linstor-csi/pull/403
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Removes multus memory limit due to short but unpredictable and large
memory consumption in some cases, such as starting up after a node
reboot (reported up to 3Gi).
The root cause will be fixed in future releases.
### Release note
```release-note
Multus memory limit removed.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated Multus system namespace configuration and DaemonSet naming for
improved environment organization.
* Adjusted container resource allocation: increased memory requests and
refined memory limits for optimized performance.
* Updated deployment template specifications to reflect infrastructure
changes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Update kube-ovn from v1.14.11 to v1.14.25.
Changes synced from upstream include:
- Updated chart templates
- New configuration options in values
### Release note
```release-note
[kube-ovn] Update to v1.14.25
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added tolerations support for VpcNatGateway resources
* Added automatic VLAN subinterface creation capability for provider
networks
* Enhanced installation documentation with OCI Registry and Talos Linux
deployment examples
* **Security Improvements**
* Applied runtime-default seccomp profiles across deployments
* Configured service account token mounting behavior for improved pod
security
* **Chores**
* Updated Kube-OVN to v1.14.25
* Updated Helm chart metadata and dependencies
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Problem
After satellite restart with patched linstor-server, some DRBD resources
ended up in **Unknown** state. Investigation revealed a race condition
between `drbdadm adjust` completion and `updateDiscGran()` lsblk check:
1. `drbdadm adjust` completes successfully (21:53:49.791) - brings
devices up
2. `updateDiscGran()` immediately tries to check discard granularity
(21:53:49.799)
3. `/dev/drbd*` device node doesn't exist yet - kernel hasn't created it
4. `lsblk` fails with exit code 32: "not a block device"
5. `StorageException` interrupts DeviceManager cycle (21:53:50.811)
6. DRBD device remains in incomplete state → **Unknown**
### Timeline from logs
```
21:53:49.791 - [DeviceManager] Resource 'pvc-aafbd92a' [DRBD] adjusted ✓
21:53:49.799 - [lsblk_parser] ERROR: /dev/drbd1169 not a block device
21:53:49.804-50.807 - [lsblk_parser] 10+ retry errors (every ~100ms)
21:53:50.811 - [DeviceManager] ERROR: Error executing lsblk
21:53:50.878 - [DeviceManager] End cycle 9 (WITH ERROR!)
```
## Solution
Update `skip-adjust-when-device-inaccessible.diff` patch to add physical
device path existence check before calling `lsblk`:
- Check `Files.exists(devicePath)` in `updateDiscGran()` before lsblk
- If device doesn't exist yet → skip check, set `discGran =
UNINITIALIZED_SIZE`
- Next DeviceManager cycle (few seconds later) → device node available →
lsblk succeeds
This complements the existing patch which checks **child layer** devices
(LUKS/Storage) for deletion scenarios, while this fix addresses the
**DRBD device itself** during adjust operations.
## Testing
Manual `drbdadm up` on affected devices confirmed they were in down
state and brought them back to UpToDate, proving the issue was
incomplete device initialization.
## Related
- Upstream linstor-server PR:
https://github.com/LINBIT/linstor-server/pull/471#issuecomment-3723392917
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced robustness for storage operations by adding device
accessibility validation. Operations now gracefully skip when device
paths are unavailable or invalid, preventing unnecessary failures and
improving system resilience during device access issues.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
By its nature, the kubernetes client provisioned in keycloak is public:
anyone can read the client secret. Therefore the secret is not necessary
and the client should be explicitly provisioned as a public client, not
as a confidential client.
### Release note
```release-note
[keycloak] Change the kubernetes client to be public, without a client
secret.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Add OIDC enablement flag and optional host derivation from tenant
release; allow explicit management kubeconfig endpoint override.
* **Refactor**
* Convert Keycloak client to public mode and remove reliance on a
Kubernetes client secret, simplifying authentication configuration.
* Remove secret-based credential plumbing from deployment templates,
streamlining kubeconfig generation.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Enable DRBD auto-diskful options to automatically convert diskless nodes
to diskful when they remain in Primary state for an extended period.
### How it works
When LINSTOR is integrated with Kubernetes, the platform may schedule
workloads on nodes that don't have local storage replicas. In such
cases, DRBD operates in "diskless" mode, accessing data over the network
from nodes that have actual disk replicas.
The `auto-diskful` feature addresses this by:
1. **Monitoring Primary state duration**: When a diskless node holds a
DRBD resource in Primary state (actively using the volume) for more than
the configured time (30 minutes), LINSTOR automatically creates a local
disk replica on that node.
2. **Automatic cleanup**: With `auto-diskful-allow-cleanup` enabled,
when the resource is no longer in Primary state on that node, LINSTOR
can automatically remove the disk replica that was created, freeing up
storage space.
### Configuration
- `DrbdOptions/auto-diskful: 30` — Convert diskless to diskful after 30
minutes in Primary state
- `DrbdOptions/auto-diskful-allow-cleanup: true` — Allow automatic
removal of auto-created replicas when no longer needed
### Benefits
- Improves I/O performance for long-running workloads by creating local
replicas
- Reduces network traffic for frequently accessed data
- Automatic cleanup prevents storage waste from temporary replicas
### Release note
```release-note
[linstor] Enable auto-diskful to automatically create local replicas on diskless nodes that hold volumes in Primary state for more than 30 minutes, improving I/O performance for long-running workloads.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced an "auto diskful" option to automatically manage diskful
resources with configurable interval and optional cleanup.
* **Chores**
* Added default configuration values to enable and time the auto-diskful
behavior and to control automatic cleanup.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Add linstor-scheduler-extender for optimal pod placement on nodes with
LINSTOR storage.
### How it works
The linstor-scheduler consists of two components:
#### 1. Scheduler Extender
A custom Kubernetes scheduler that works alongside the default
kube-scheduler. When a pod requests LINSTOR-backed storage, the
scheduler extender communicates with the LINSTOR controller to find
nodes that have local replicas of the requested volumes, prioritizing
placement on nodes with existing data to minimize network traffic.
#### 2. Admission Webhook
A MutatingWebhookConfiguration that automatically sets `schedulerName:
linstor-scheduler` on pods that use LINSTOR CSI driver volumes. This
ensures that only pods with LINSTOR PVCs are routed through the custom
scheduler, while all other pods continue using the default scheduler.
The webhook inspects pod creation requests and checks if any of the
pod's PVCs use a StorageClass with the LINSTOR CSI provisioner
(`linstor.csi.linbit.com`). If so, it mutates the pod spec to use the
linstor-scheduler.
### Components
- `linstor-scheduler` package in `packages/system/linstor-scheduler/`
- PackageSource definition in
`packages/core/platform/sources/linstor-scheduler.yaml`
- Integration into `paas-full` and `distro-full` bundles
### Upstream PRs
- https://github.com/piraeusdatastore/helm-charts/pull/67 — fix:
KubeSchedulerConfiguration v1 support for Kubernetes 1.25+
- https://github.com/piraeusdatastore/helm-charts/pull/68 — feat:
admission webhook support with cert-manager
- https://github.com/piraeusdatastore/helm-charts/pull/69 — fix chart
template issues
### Release note
```release-note
[linstor] Add linstor-scheduler for optimal pod placement on nodes with local LINSTOR replicas. Includes admission webhook that automatically routes pods using LINSTOR CSI volumes to the custom scheduler.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added LINSTOR scheduler component to the platform, packaged as a Helm
chart and installable package.
* Includes admission webhook, scheduler extender, autoscaling support,
pod disruption budget, RBAC and service account integration, and Helm
test hooks.
* **Documentation**
* Added chart README and default configuration values for easy
deployment and customization.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Addes traffic locality capabilities to Seaweedfs.
### Release note
```release-note
Added traffic locality capabilities to Seaweedfs
Add physical device path existence check to prevent race condition where
lsblk is called before kernel creates DRBD device node after drbdadm adjust.
This fixes issue where resources ended up in Unknown state after satellite
restart because:
- drbdadm adjust completes successfully (brings devices up)
- updateDiscGran() immediately tries lsblk
- /dev/drbd* doesn't exist yet (kernel hasn't created node)
- lsblk fails with "not a block device" exit code 32
- StorageException interrupts DeviceManager cycle
- Device remains in incomplete state
The patch now checks Files.exists(devicePath) before lsblk, allowing
the check to be retried in next cycle when device node is available.
Upstream: https://github.com/LINBIT/linstor-server/pull/471
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
By its nature, the kubernetes client provisioned in keycloak is public:
anyone can read the client secret. Therefore the secret is not
necessary and the client should be explicitly provisioned as a public
client, not as a confidential client.
### Release note
```release-note
[keycloak] Change the kubernetes client to be public, without a client
secret.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Add representation for backup Plans, BackupJobs and Backups in the dashboard and a readme
describing, how to add further resources to the dashboard.
### Release note
```release-note
[backups,dashboard] Add backup Plans, BackupJobs, and Backups to the dashboard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Backups section in the dashboard navigation with Plans,
BackupJobs, and Backups.
* Added detailed views for Plan, BackupJob, and Backup showing metadata,
references, status, timings, and artifacts.
* Enhanced list views with custom columns (application refs,
phase/status, created/taken timestamps).
* **Documentation**
* Added a Dashboard integration guide documenting how to add resources,
list/detail wiring, and verification steps.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Cozystack-api and lineage-webhook tolerate all taints, so they are able
to run on masters no matter what.
Needed for unschedulable control-plane setup, quorum nodes, etc.
### Release note
```release-note
Cozystack-api and lineage-webhook tolerate all taints.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated pod scheduling tolerations to improve deployment flexibility
and compatibility across diverse cluster configurations.
* Enhanced workload distribution by making pods more resilient to
cluster taints, enabling better resource utilization in various
environments.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add DRBD options to automatically convert diskless nodes to diskful
when they remain in Primary state for more than 30 minutes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add linstor-scheduler-extender for optimal pod placement on nodes
with LINSTOR storage.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
Fix bug in toggle-disk retry logic that left orphaned DRBD devices in
kernel.
## Problem
When toggle-disk retry was triggered (e.g., user retries after a failed
operation), the code called `removeLayerData()` to clean up and recreate
the layer stack. However, `removeLayerData()` only removes data from the
controller's database — it does NOT call `drbdadm down` on the
satellite.
This caused DRBD devices to remain in the kernel (visible in `drbdsetup`
but not managed by LINSTOR), occupying ports and blocking subsequent
operations.
## Solution
Changed retry logic to simply repeat the operation with existing layer
data intact. The satellite handles this idempotently without creating
orphaned resources.
## Upstream
- https://github.com/LINBIT/linstor-server/pull/475 (updated)
```release-note
[linstor] Fix orphaned DRBD devices during toggle-disk retry
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added cancel and retry capabilities for disk addition operations
* Added cancel and retry capabilities for disk removal operations
* Improved cleanup handling for diskless resources with orphaned storage
layers
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The previous retry logic in toggle-disk removed layer data from controller DB
and recreated it. However, removeLayerData() only deletes from the database
without calling drbdadm down on the satellite, leaving orphaned DRBD devices
in the kernel that occupy ports and block new operations.
This fix changes retry to simply repeat the operation with existing layer data,
allowing the satellite to handle it idempotently.
Upstream: https://github.com/LINBIT/linstor-server/pull/475
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Moved README describing how to add new resources to the Cozystack
dashboard to a separate commit to directly reference relevant code
changes from the document.
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Add representation for backup Plans in the dashboard and a readme
describing, how to add further resources to the dashboard.
### Release note
```release-note
[backups,dashboard] Add backup Plans to the dashboard.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Update kube-ovn from v1.14.11 to v1.14.25.
Sync chart templates with upstream changes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Add automatic `valuesFrom` injection with `cozystack-values` secret into
HelmReleases created by Package reconciler. This enables charts to
access cluster and namespace configuration via `.Values._cluster` and
`.Values._namespace`.
**Changes:**
- **Package Reconciler:** Inject `valuesFrom` referencing
`cozystack-values` secret, add `Owns(&HelmRelease{})` to reconcile on
changes
- **SecretReplicatorReconciler (new):** Replicate secret to namespaces
with active HelmReleases, cleanup orphaned secrets
- **PackageSource Reconciler:** Replace Watches with
`Owns(&ArtifactGenerator{})`
- **Platform:** Add `operator.cozystack.io/skip-cozystack-values`
annotation to platform PackageSource
### Release note
```release-note
[operator] Add valuesFrom injection to HelmReleases created by Package reconciler with automatic secret replication
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Configurable secret replication that copies designated values secrets
into namespaces selected by labels.
* Annotation to opt out of injecting cozystack-values into specific
PackageSource resources.
* New runtime flags to customize secret name, namespace, and namespace
selector.
* **Refactor**
* Simplified ownership/watcher setup to improve event handling and
reconciliation efficiency.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Build piraeus-server (linstor-server) from source with custom patches:
- **adjust-on-resfile-change.diff** — Use actual device path in res file
during toggle-disk; fix LUKS data offset
- Upstream: [#473](https://github.com/LINBIT/linstor-server/pull/473),
[#472](https://github.com/LINBIT/linstor-server/pull/472)
- **allow-toggle-disk-retry.diff** — Allow retry and cancellation of
failed toggle-disk operations
- Upstream: [#475](https://github.com/LINBIT/linstor-server/pull/475)
- **force-metadata-check-on-disk-add.diff** — Create metadata during
toggle-disk from diskless to diskful
- Upstream: [#474](https://github.com/LINBIT/linstor-server/pull/474)
- **skip-adjust-when-device-inaccessible.diff** — Skip DRBD adjust/res
file regeneration when child layer device is inaccessible
- Upstream: [#471](https://github.com/LINBIT/linstor-server/pull/471)
Also updates plunger-satellite script and values.yaml for the new build.
### Release note
```release-note
[linstor] Build linstor-server with custom patches for improved disk handling
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automatic DRBD stall detection and recovery, improving storage
resync resilience without manual intervention.
* Introduced configurable container image references via Helm values for
streamlined deployment.
* **Bug Fixes**
* Enhanced disk toggle operations with retry and cancellation support
for better error handling.
* Improved metadata creation during disk state transitions.
* Added device accessibility checks to prevent errors when underlying
storage devices are unavailable.
* Fixed LUKS encryption header sizing for consistent deployment across
nodes.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add patches for piraeus-server:
- adjust-on-resfile-change: handle resource file changes
- allow-toggle-disk-retry: enable disk operation retries
- force-metadata-check-on-disk-add: verify metadata on disk addition
Update plunger-satellite script and values.yaml for new build.
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Adds a `k=kubectl` alias and bash completion for kubectl to the
e2e-testing sandbox container to maintainers have an easier time
exec'ing into the CI container when something needs to be debugged.
### Release note
```release-note
[testing] Add k=kubectl alias and enable kubectl completion in the CI
container.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced the e2e sandbox image to enable shell bash-completion and
kubectl command completion.
* Added an alias (k) and completion wiring for kubectl to improve
interactive command use.
* These changes augment the test environment shell during image build to
provide a smoother developer/testing experience.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add automatic injection of cozystack-values secret reference into
HelmReleases created by Package reconciler. This enables charts to
access cluster and namespace configuration via .Values._cluster and
.Values._namespace.
Add annotation operator.cozystack.io/skip-cozystack-values to disable
injection for specific PackageSources (used for platform PackageSource
to avoid circular dependency).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Integrate fix
- https://github.com/kubevirt/cloud-provider-kubevirt/pull/379
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Fix endpoints for cilium-gateway
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved handling of incomplete endpoint data by introducing fallback
detection mechanisms.
* Enhanced service discovery to gather endpoints from all available
resources when standard detection fails.
* Updated logging to provide better visibility into fallback operations
and current resource status.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Replaces Helm lookup functions with FluxCD valuesFrom mechanism for
passing configuration to HelmReleases. This provides cleaner config
propagation and eliminates the need for force reconcile controllers.
### Changes:
**Platform/Tenant charts:**
- Add Secret `cozystack-values` creation in platform chart (for
tenant-root and system namespaces)
- Add Secret `cozystack-values` creation in tenant chart (for child
namespaces)
**cozystack-api:**
- Add `valuesFrom` references to HelmRelease when creating applications
- Filter keys starting with `_` when returning Application specs
- Validate that user values don't contain `_` prefixed keys
**cozystack-controller:**
- Add validation that HelmRelease contains correct valuesFrom
configuration
- Remove `CozystackConfigReconciler` (no longer needed)
- Remove `TenantHelmReconciler` (no longer needed)
**Helm charts (40+ files):**
- Add helper templates in cozy-lib for `_cluster`/`_namespace` access
- Replace ConfigMap lookups with `.Values._cluster.*`
- Replace Namespace annotation lookups with `.Values._namespace.*`
### Architecture:
```
Secret cozystack-values (in each namespace)
├── _cluster: YAML with data from ConfigMaps (cozystack, cozystack-branding, cozystack-scheduling)
└── _namespace: YAML with namespace service references (etcd, host, ingress, monitoring, seaweedfs)
HelmRelease
└── spec.valuesFrom:
├── Secret/cozystack-values → _namespace → .Values._namespace
└── Secret/cozystack-values → _cluster → .Values._cluster
```
### Release note
```release-note
[platform] Replace Helm lookup functions with FluxCD valuesFrom mechanism for configuration propagation
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Helm releases and namespaces now source centralized cluster/namespace
configuration via a new secret (cozystack-values), and many templates
read values from chart-provided _cluster/_namespace entries.
* **Bug Fixes**
* API now rejects application specs containing reserved keys prefixed
with "_" to prevent invalid configurations.
* **Refactor**
* Two background reconciler controllers were removed from startup,
simplifying controller initialization.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes an issue where backport releases incorrectly became marked as
"Latest" despite passing `make_latest: 'false'` to the GitHub API.
**Root cause:** The `getLatestRelease()` API returns the release with
the "Latest" flag, not the highest semver version. Combined with race
conditions during parallel release publishing and GitHub API potentially
ignoring `make_latest: 'false'`, backport releases were incorrectly
marked as latest.
**Solution:**
- Replace `getLatestRelease()` with semver-based max version detection
across all published releases
- After publishing a backport release, explicitly restore the latest
flag on the highest semver release
- Remove unused dead code from `tags.yaml` workflow
### Release note
```release-note
[ci] Fix latest release detection to use semver comparison instead of GitHub's "Latest" flag
```
## What this PR does
Reverts PR #1770 which incorrectly changed status/phase checks from
lowercase to uppercase.
The actual KubeVirt metrics use **lowercase**:
- `kubevirt_vm_info` uses `status="running"` (not `"Running"`)
- `kubevirt_vmi_info` uses `phase="running"` (not `"Running"`)
### Verification
Queried virt-controller metrics directly in the instories cluster:
```
kubevirt_vm_info{...,status="running",status_group="running",...} 1
kubevirt_vmi_info{...,phase="running",...} 1
```
Note: `kubectl get vm` shows `STATUS: Running` with capital R, but this
is display formatting — the actual metric labels use lowercase.
### Release note
```release-note
[kubevirt-operator] Fix VM alert rules to use correct lowercase status values
```
Replace Helm lookup functions with FluxCD valuesFrom mechanism for
reading cluster and namespace configuration.
Changes:
- Create Secret cozystack-values in each namespace with values.yaml key
containing _cluster and _namespace configuration as nested YAML
- Configure HelmReleases to read from this Secret via valuesFrom
(valuesKey defaults to values.yaml, so it can be omitted)
- Update cozy-lib helpers to access config via .Values._cluster
- Add default values for required _cluster keys to ensure all fields exist
- Update Go code (cozystack-api and helm reconciler) to use new format
This eliminates the need for Helm lookup functions while maintaining
the same configuration interface for charts.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Revert PR #1770 which incorrectly changed status check from lowercase
to uppercase. The actual metrics use lowercase:
- kubevirt_vm_info uses status="running" (not "Running")
- kubevirt_vmi_info uses phase="running" (not "Running")
Verified by querying virt-controller metrics in instories cluster.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds a `k=kubectl` alias and bash completion for kubectl to the
e2e-testing sandbox container to maintainers have an easier time
exec'ing into the CI container when something needs to be debugged.
### Release note
```release-note
[testing] Add k=kubectl alias and enable kubectl completion in the CI
container.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Implement a controller for BackupJobs referencing a Velero strategy. Creates a Backup.velero.io according to the template in the `Velero.strategy.backups.cozystack.io`.
### Release note
```release-note
[backups] Implement a backup strategy controller for Velero strategies.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Co-authored-by: Andrey Kolkov <androndo@gmail.com>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch narrows the scope of the Velero backup strategy controller to
simply template Velero Backups according to the application being backed
up and the template in the strategy. Creating storage locations is now
out of scope.
### Release note
```release-note
[backups] Implement templating for Velero backups and remove creation of
backup storage locations and volume snapshot locations.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch implements the Reconcile function for BackupJobs with a
Velero strategy ref.
### Release note
```release-note
[backups] Implement the Velero backup strategy controller.
```
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Replace unreliable getLatestRelease() API with semver-based max version
detection. After publishing a backport release, explicitly restore the
latest flag on the highest semver release to handle cases where GitHub
API ignores make_latest: 'false'.
Also remove dead code (unused steps) from tags.yaml workflow.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This PR splits the monolithic `cozystack-resource-definitions` package
into 25 individual resource definition packages (`*-rd`) for better
modularity and independent versioning.
**Changes:**
- Created 25 separate `*-rd` packages (bootbox-rd, bucket-rd,
clickhouse-rd, etcd-rd, ferretdb-rd, foundationdb-rd, http-cache-rd,
info-rd, ingress-rd, kafka-rd, kubernetes-rd, monitoring-rd, mysql-rd,
nats-rd, postgres-rd, rabbitmq-rd, redis-rd, seaweedfs-rd,
tcp-balancer-rd, tenant-rd, virtual-machine-rd, virtualprivatecloud-rd,
vm-disk-rd, vm-instance-rd, vpn-rd)
- Removed `packages/system/cozystack-resource-definitions`
- Updated platform bundles (paas-hosted, paas-full, distro-full) to
reference individual -rd packages
- Updated `hack/update-crd.sh` to use package-specific directories
Each `*-rd` package contains:
- `Chart.yaml` - package metadata
- `values.yaml` - default values
- `Makefile` - build instructions
- `cozyrds/<name>.yaml` - CRD definition
- `templates/cozyrd.yaml` - Helm template
**Benefits:**
- **Modularity**: Each resource definition is now a standalone package
- **Independent versioning**: Resources can be versioned independently
- **Maintainability**: Easier to update individual resources
- **Build efficiency**: Parallel building of resource packages
### Release note
```release-note
[platform] Split cozystack-resource-definitions into 25 separate *-rd packages for better modularity and independent versioning. Each resource definition is now a standalone package.
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **Refactor**
* Split the monolithic resource-definitions into many independent resource-definition packages (e.g., bootbox-rd, bucket-rd, clickhouse-rd, etc.) for modular deployment and finer-grained management
* Updated deployment bundles to reference the new per-resource releases with uniform cozy-system namespace and CRD dependency
* **Chores**
* Added packaging/Helm stubs (Chart.yaml, Makefile, values, templates) for each new resource-definition
* **Bug Fixes**
* Made CRD path resolution dynamic (NAME validated before assignment)
<sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Add lb tests for tenant k8s
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Increased readiness and port-forward timeouts to improve stability.
* Added full end-to-end provisioning and validation: automated namespace
and backend deployment, load balancer provisioning, health checks with
retries, reachability validation, and cleanup.
* Provisioning sequence now runs earlier and is duplicated within the
test flow, altering execution order and adding extra validation/cleanup
steps.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Service selector was using app: "${test_name}-backend" but the
Deployment pod template has app: backend. Fixed selector to match
the actual pod labels so endpoints are created correctly.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Allows maintainers to trigger test rerun by commenting /retest on a PR.
The workflow finds the latest run for the PR and reruns starting from
the "Prepare environment" job, skipping the build step.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
actions/checkout configures http.extraheader with GITHUB_TOKEN which
takes priority over URL credentials. This caused tag pushes to
authenticate as github-actions instead of cozystack-bot, preventing
the Versioned Tag workflow from being triggered.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Run curl directly from the testing container instead of creating
a separate pod with kubectl run. This avoids PodSecurity policy
violations and simplifies the test execution.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Added resource requests and limits for etcd-defrag container.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[system] Add resource requests and limits to etcd-defrag
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced resource configurations for the etcd defragmentation process
to ensure optimal performance and system stability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add documentation explaining how to enable Hubble for network
observability in Grafana. Include four pre-built Hubble dashboards
(overview, dns-namespace, l7-http-metrics, network-overview) and
register them in the monitoring hub's dashboard list.
Closes#749
Signed-off-by: majiayu000 <1835304752@qq.com>
## What this PR does
The Helm hook that creates a job deleting all applications in a tenant
before deleting the tenant itself now runs this job from the cozy-system
namespace. This prevents conflicts with resource quotas (not enough
resources to run cleanup job) without temporary increases of the quota
or similar vulnerability-introducing hacks.
### Release note
```release-note
[tenant] Run cleanup job in system namespace to avoid conflicts on
resource quotas.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Infrastructure Updates**
* Updated cleanup job namespace targeting to use a fixed system
configuration
* Adjusted cleanup job execution priority level
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes egress policy for nested Kubernetes clusters using `exposeMethod:
Proxied`.
The clusterwide egress policy blocks traffic from tenant pods to ingress
pods in parent namespaces. This breaks:
- cert-manager HTTP-01 self-check
- Any scenario where pods need to access services exposed through parent
ingress
Adds egress rule allowing traffic to ingress pods
(`cozystack.io/service: ingress`) in parent namespaces, following the
same pattern as existing vminsert and etcd rules.
### Release note
```release-note
[tenant] Fixed tenant egress policy to allow traffic to parent ingress pods, enabling cert-manager HTTP-01 challenges and external domain access for nested clusters with exposeMethod: Proxied
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced network policies for multi-tenant environments with improved
traffic routing based on namespace hierarchies, enabling more granular
egress control.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fix case sensitivity in status check for VMNotRunningFor10Minutes alert
rule.
The metric `kubevirt_vm_info` uses `status="Running"` (capital R), but
the alert rule was checking for `status!="running"` (lowercase), causing
false alerts for running VMs.
Fixes https://github.com/cozystack/cozystack/issues/1552
### Release note
```release-note
[kubevirt-operator] Fix VMNotRunningFor10Minutes alert false positives
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed alert rule condition for virtual machine status detection to
ensure proper matching of running VMs.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Add deployment files
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Deployable CozyStack operator with configurable enablement, image, and
platform-source settings
* Operator-managed package source and variant-based platform
installation options
* **Chores**
* Release/packaging targets added to automate image and package
publishing and update manifests
* Configuration schema extended to include operator-related fields and
versioning controls
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR renames cozypkg to cozyhr
https://github.com/cozystack/cozyhr
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[installer,dx] Rename cozypkg to cozyhr
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does

### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[platform] Add PackageSources for cozystack-operator
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for 76 new package sources, including operators,
applications, and system components for the CozyStack platform ecosystem
(e.g., backup controller, database operators, monitoring tools,
networking, storage, and virtualization packages).
* Introduced a template aggregation mechanism to manage package source
definitions.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozypkg] Add tool for managing Package and PackageSources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* New cozy CLI with entrypoint and root command.
* Interactive "add" command to install packages from args or files, with
dependency resolution, topological install order, variant selection, and
progress feedback.
* "del" command to remove packages safely, including dependent analysis,
confirmation prompts, and safe delete ordering.
* "list" command to show available or installed packages, with optional
per-component output and concise status lines.
* "dependencies" command to emit Graphviz DOT graphs of
package/component relationships.
* Common flags: kubeconfig, file inputs, --installed and --components
where applicable.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
depends on https://github.com/cozystack/cozystack/pull/1741
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Add Package and PackageSource reconcilers
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Launches CozyStack operator with CLI-configurable runtime
(metrics/health/leader election/HTTP2), optional pre-reconcile steps,
optional automatic Flux install, and platform-source URL support
(OCI/Git).
* Adds PackageSource and Package controllers for artifact generation,
variant resolution, dependency tracking, HelmRelease orchestration,
namespace reconciliation, orphan cleanup, and status propagation.
* Package status now exposes per-dependency readiness.
* **Chores**
* Embeds Flux manifests and adds a local Flux install workflow; updates
manifest generation path and Makefile targets.
* **Other**
* PackageSource CRD shortName simplified to "pks".
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Increases multus memory limit.
Based on multiple community reports stating that multus tend to consume
a lot of memory during startup after a node reboot.
### Release note
```release-note
Multus memory limit increased.
```
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Introduce Cozystack-operator core logic
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Operator entrypoint with flag-driven configuration (metrics, probes,
leader election, Flux pre-install, version, platform-source options).
* Optional pre-start Flux installation and automatic provisioning of
platform sources from OCI or Git/HTTP(S)/SSH.
* **Chores**
* Embedded Flux manifests packaged for simplified installation; build
tooling updated to produce new manifest outputs.
* **Behavior**
* Pre-start Flux and platform-source installation failures are treated
as fatal.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The Helm hook that creates a job deleting all applications in a tenant
before deleting the tenant itself now runs this job from the cozy-system
namespace. This prevents conflicts with resource quotas (not enough
resources to run cleanup job) without temporary increases of the quota
or similar vulnerability-introducing hacks.
### Release note
```release-note
[tenant] Run cleanup job in system namespace to avoid conflicts on
resource quotas.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch adds boilerplate code for a Velero strategy for the backups
API. Implementation can be added into the Reconcile function in a
subsequent PR.
###
```release-note
[backups] Add the boilerplate for a new Velero strategy for backups.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Velero as a supported backup strategy, including a
cluster-scoped Velero resource type and CRD for creating Velero-backed
strategies.
* Backup job controller updated to recognize and dispatch Velero-backed
jobs, with status conditions exposed for monitoring progress and health.
* Introduced basic reconciliation flow for Velero strategy to enable
future behavior and observability.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fix case sensitivity in status check: "running" → "Running".
The metric uses "Running" (capital R), so the lowercase check was
always triggering alerts even for running VMs.
Fixes: https://github.com/cozystack/cozystack/issues/1552
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
- Update Cilium from v1.17.8 to v1.18.5
- Add `SED_INPLACE` variable to `scripts/common-envs.mk` for macOS
compatibility
- Remove deprecated `enableRuntimeDeviceDetection` option (now default
behavior in 1.18)
Cilium 1.18 requires Linux kernel 5.10+ (compatible with Talos).
References:
- [Cilium 1.18 Upgrade
Guide](https://docs.cilium.io/en/stable/operations/upgrade/)
- [Cilium 1.18 Release
Blog](https://isovalent.com/blog/post/cilium-1-18/)
### Release note
```release-note
[cilium] Update Cilium to v1.18.5
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added configurable Kubernetes service discovery with ConfigMap-based
endpoint source
* Introduced exponential backoff settings for Kubernetes API client
connections
* Added metrics sampling interval configuration for internal agent
metrics
* Implemented identity management mode options for endpoint slices and
DNS proxy pre-allocation
* Enhanced Prometheus scrape timeout configuration across monitoring
components
* **Improvements**
* Upgraded core components to v1.18.5
* Strengthened security defaults by disabling privilege escalation
across pods
* Added startup and liveness probe configurations for improved health
monitoring
* Extended network policy correlation capabilities in Hubble
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-controller] Fix: move crds to definitions
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the Custom Resource Definition generation template to source
files from a new location, ensuring consistent configuration management.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Update Cilium from v1.17.8 to v1.18.5.
Breaking changes addressed:
- Remove deprecated enableRuntimeDeviceDetection option from
values-kubeovn.yaml (now default behavior in 1.18)
- Use SED_INPLACE variable for macOS compatibility in Makefile
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add SED_INPLACE variable to common-envs.mk that detects the OS and uses
the appropriate sed in-place syntax: `sed -i ''` on macOS (Darwin) and
`sed -i` on Linux.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Adds deterministic alphabetical sorting to all registry resource lists:
- TenantNamespace (sorted by name)
- TenantSecret (sorted by namespace/name)
- TenantModule (sorted by namespace/name)
- Application (sorted by namespace/name)
Introduces a new `pkg/registry/sorting` package with generic helper
functions
to avoid code duplication.
Also fixes pre-existing linter errors:
- Unused `helmReleaseGVR` variables
- Non-constant format strings in `klog.Errorf` calls
- Redundant embedded field selectors
### Release note
```release-note
[platform] Registry resource lists are now returned in alphabetical order
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a reusable alphabetical sorting utility and applied consistent
sorting across list endpoints.
* **Bug Fixes**
* Ensured ResourceVersion/UID are sourced correctly for accurate
list/table responses.
* Simplified and improved error logging and table ResourceVersion
handling.
* Made schema defaulting behavior more consistent.
* **Tests**
* Added unit tests validating alphabetical sorting for multiple resource
types.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Nested Kubernetes clusters with exposeMethod: Proxied cannot reach their
own external domains because the clusterwide egress policy blocks traffic
to ingress pods in parent namespaces.
This breaks cert-manager HTTP-01 self-check and any scenario where pods
need to access services exposed through parent ingress.
Add egress rule allowing traffic to ingress pods (cozystack.io/service:
ingress) in parent namespaces, following the same pattern as existing
vminsert and etcd rules.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Simplify ByName with strings.Compare
- Avoid string allocation in ByNamespacedName by comparing fields separately
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add sorting helper package with generic functions for sorting
resources by name (cluster-scoped) or namespace/name (namespace-scoped).
Apply sorting to:
- TenantNamespace (by name)
- TenantSecret (refactored to use helper)
- TenantModule (new)
- Application (new)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
This patch adds boilerplate code for a Velero strategy for the backups
API. Implementation can be added into the Reconcile function in a
subsequent PR.
###
```release-note
[backups] Add the boilerplate for a new Velero strategy for backups.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[agents] Add instructions for working with unresolved code review comments
```
This PR adds a new GitHub workflow that automatically creates patch
releases for maintenance branches.
## What it does
- Runs daily at 2:00 AM CET (1:00 UTC)
- Checks all `release-X.Y` branches
- For each branch, finds the latest published release with tag `vX.Y.Z`
(without suffixes like -rc, -alpha, -beta)
- If new commits exist after the release, creates a new tag `vX.Y.Z+1`
and force-pushes it
- This triggers the existing tag workflow to build and create a release
PR
## Why releases instead of tags
The workflow checks published releases (not just tags) because if a
release PR hasn't been merged yet, the workflow should run again the
next day and move the tag to newer commits if they exist.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Automated patch release workflow configured for release branches,
enabling automatic version tagging when new commits are detected.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds a GitHub Actions workflow that automatically updates
release notes from changelog files when changes are merged to main.
The workflow:
- Triggers on push to main branch
- Processes up to 30 releases from the first page
- For each release, checks if a corresponding changelog file exists in
`docs/changelogs/`
- Updates the release notes with the changelog content if it exists and
differs from the current content
- Skips releases that are already up to date to avoid unnecessary API
calls
This automates the process of keeping release notes synchronized with
changelog files.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Automated workflow added to synchronize GitHub releases with local
changelog files, updating release notes only when content differs.
* **Documentation**
* Changelogs and release notes reorganized and expanded with improved
sectioning and new tooling/documentation entries.
* **New Features**
* Added VM disk SVG icon entry under Features/Improvements.
* **Bug Fixes**
* Pinned CoreDNS image tag for reproducible deployments.
* Fixed a documentation typo that prevented a component from displaying
in the web UI.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Extract Talos-related functionality from `packages/core/installer` into
a separate `packages/core/talos` package.
This refactoring separates concerns:
- The `installer` package now focuses solely on the Cozystack installer
- The `talos` package handles all Talos Linux image building and assets
generation
Changes:
- Created new `packages/core/talos` package with Chart.yaml, Makefile,
and values.yaml
- Moved Talos profiles (initramfs, kernel, iso, installer, metal,
nocloud) to talos package
- Moved matchbox configuration and Dockerfile to talos package
- Moved `hack/gen-profiles.sh` and `hack/gen-versions.sh` scripts to
talos package
- Updated installer Makefile to remove all Talos-related targets
- Updated root Makefile to build talos package separately
- Updated matchbox Dockerfile paths to reference talos package
### Release note
```release-note
[core] Extract Talos package from installer into separate packages/core/talos package
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Reorganized build system to add a dedicated Talos package with its own
image and asset build workflow
* Switched asset generation to the new Talos packaging path and
simplified build dependency chain
* Added Helm chart manifest and streamlined image build/publishing steps
for Talos-related artifacts
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Add changelogs for v.0.39.0 and v.0.39.1
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Topology-aware routing for services
* Automatic pod rollouts on configuration changes
* Windows VM scheduling support
* SLACK_SEVERITY_FILTER for notification filtering
* VMAgent resource for metrics scraping
* **Improvements**
* Enhanced networking and storage capabilities
* Configuration management enhancements
* Updated dependencies
* **Bug Fixes**
* Schema nesting and resizing validation
* Namespace deletion regression
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
See the examples (not part of this PR)
PackageSource:
-
43cef5d50e/packages/core/platform/templates/sources/backup-controller.yaml
Package:
-
43cef5d50e/packages/core/platform/templates/packages/isp-full.yaml
Platform Package:
-
43cef5d50e/packages/core/installer/example/platform.yaml
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack-operator] Introduce API objects: packages and packagesources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced Package and PackageSource custom resources to manage
software packages and their configurations, with support for variants,
components, library management, and dependency handling.
* **Bug Fixes**
* Made the Chart field optional in CozystackResourceDefinitionRelease.
* **Chores**
* Updated code generation tooling and CRD artifact management.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This change is extracted from
- https://github.com/cozystack/cozystack/pull/1641
and reworked to work standalone
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[cozystack] Update go modules
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Go toolchain to 1.25.0 and upgraded core Kubernetes libraries,
OpenTelemetry, Prometheus, gRPC/protobuf and many indirect dependencies.
Bumped builder base images to golang:1.25-alpine across multiple
components.
* **Refactor**
* Removed legacy component versioning/emulation and simplified server
startup and configuration paths.
* **Tests**
* Removed tests related to the legacy versioning/emulation behavior.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When VMs are created without a public IP, no service is created for them
and they have no in-cluster DNS name. This PR fixes this and resolves
#1731.
### Release note
```release-note
[vm] Always expose VMs with at least a ClusterIP service.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Service templates now conditionally enable external-specific
annotations and settings when external mode is enabled.
* External LoadBalancer deployments support richer port configuration
(including whole-IP fallback), while internal services retain a single
default port (65535).
* External traffic policy and node port allocation are applied only in
external mode to preserve internal behavior.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This change is extracted from
- https://github.com/cozystack/cozystack/pull/1641
and reworked to work standalone
## What this PR does
This PR extracts changes from the
https://github.com/cozystack/cozystack/pull/1641. It adds application
metadata labels to HelmReleases and updates the filtering mechanism to
use labels instead of chart/sourceRef matching.
Changes:
- Add three application metadata labels
(`apps.cozystack.io/application.kind`,
`apps.cozystack.io/application.group`,
`apps.cozystack.io/application.name`) when creating/updating HelmRelease
via Cozystack-API
- Replace `shouldIncludeHelmRelease` filtering with label-based
filtering in Get, List, and Update methods
- Always add kind and group label requirements in List for precise
filtering
- Update CozystackResourceDefinitionController to watch only
HelmReleases with `cozystack.io/ui=true` label
- Update LineageControllerWebhook to extract metadata directly from
HelmRelease labels instead of using chart mapping configuration
- Add functionality to update HelmRelease chart from
CozystackResourceDefinition using label selectors
### Release note
```release-note
[registry] Add application labels and update filtering mechanism to use label-based filtering instead of chart/sourceRef matching
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added application metadata labels (kind, group, name) and exported
label keys for HelmRelease identification.
* New reconciler keeps HelmRelease charts in sync with application CRDs.
* **Refactor**
* Mapping, listing and selection moved to label-driven logic;
reconciliation responsibilities split so core reconciler focuses on
restart/debounce while a separate reconciler updates HelmReleases.
* **Chores**
* Migration script to backfill application labels on existing
HelmReleases.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Enable automatic restart of cilium and cilium-operator pods when
cilium-config ConfigMap is updated.
This change adds:
- `rollOutCiliumPods: true` - enables automatic rollout of cilium-agent
pods
- `operator.rollOutPods: true` - enables automatic rollout of
cilium-operator pods
When the ConfigMap is updated, pods will automatically restart due to
the `cilium.io/cilium-configmap-checksum` annotation that contains the
SHA256 hash of the configmap.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Cilium configuration settings to enable pod rollout behavior
for Cilium and operator pods by default.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR adds nodeAffinity configuration to virtual-machine and
vm-instance charts to support dedicated nodes for Windows VMs.
## Changes
- Added logic to check `cozystack-scheduling` ConfigMap in `cozy-system`
namespace
- If `dedicatedNodesForWindowsVMs` is enabled, adds appropriate
nodeAffinity:
- **Windows VMs**: Strong affinity (required) to nodes with label
`scheduling.cozystack.io/vm-windows=true`
- **Non-Windows VMs**: Soft affinity (preferred) to nodes without the
Windows label
## Implementation
- Windows detection based on `instanceProfile` value starting with
"windows"
- Strong affinity uses `requiredDuringSchedulingIgnoredDuringExecution`
- Soft affinity uses `preferredDuringSchedulingIgnoredDuringExecution`
with weight 100
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable node-affinity for VM scheduling: when the cluster
scheduling flag is enabled, Windows VMs are placed on dedicated Windows
nodes (required rule), while non-Windows VMs are preferred to avoid
those nodes (soft preference).
* Change is gated by the cluster scheduling configuration and only
affects placement rules; no other VM specs were altered.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
When VMs are created without a public IP, no service is created for them
and they have no in-cluster DNS name. This PR fixes this and resolves
#1731.
### Release note
```release-note
[vm] Always expose VMs with at least a ClusterIP service.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
- Add three application metadata labels to HelmRelease:
- apps.cozystack.io/application.kind
- apps.cozystack.io/application.group
- apps.cozystack.io/application.name
- Replace shouldIncludeHelmRelease filtering with label-based filtering
in Get, List, and Update methods
- Always add kind and group label requirements in List for precise filtering
- Update CozystackResourceDefinitionController to watch only HelmReleases
with cozystack.io/ui=true label
- Update LineageControllerWebhook to extract metadata directly from
HelmRelease labels instead of using mapping configuration
- Add functionality to update HelmRelease chart from CozystackResourceDefinition
using label selectors
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR adds Cloupard as new adopter to the list.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Add Cloupard to ADOPTERS.md
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated the list of project adopters with recent entries.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
This PR includes new seaweedfs with improved perfomance for S3 daemon
and fixes issue https://github.com/seaweedfs/seaweedfs/issues/7757
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Update SeaweedFS v4.02
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added all-in-one deployment mode with configurable replicas and update
strategy
* Expanded storage configuration supporting PersistentVolumeClaims with
customizable access modes and size
* Introduced configurable certificate duration and renewal periods
* Enhanced monitoring configuration with gateway host/port and
additional labels
* **Bug Fixes**
* Fixed probe endpoint scheme references across components
* **Chores**
* Updated to SeaweedFS 4.02
* Updated default ingress class configuration
* S3 disabled by default
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant
monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the
Alerta deployment to enable
filtering of alert severities for Slack notifications based on the
disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics
within tenant namespaces, improving
monitoring granularity and control.
```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added configurable severity filtering for Telegram alerts.
* Extended Slack severity filtering to accept lists of severities.
* **Bug Fixes / Behavior**
* Severity settings now accept arrays (multiple severities) instead of
single comma-separated strings.
* **Documentation**
* Updated configuration docs and examples to show list-style severity
settings.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the Alerta deployment to enable
filtering of alert severities for Slack notifications based on the disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics within tenant namespaces, improving
monitoring granularity and control.
```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Cozystack is a Kubernetes-based PaaS built on Helm umbrella charts and FluxCD.
Packages live in `packages/{core,system,apps,extra,library,tests}/`. The `library/` group holds reusable helper charts; the `tests/` group exists because library charts are not directly testable.
Each package wraps one or more upstream Helm charts.
Go code in `cmd/`, `internal/`, `pkg/` implements Kubernetes controllers and API server.
Static CRDs are generated by `hack/update-codegen.sh` from types under `api/v1alpha1/`, `api/backups/`, and `api/dashboard/`.
App Kinds (like `Postgres`, `Kafka`) live in `api/apps/v1alpha1/` but are registered dynamically at runtime from `ApplicationDefinition` resources — they are not static CRDs.
## Vendored Code — Critical Rules
**Never suggest editing files inside any `charts/` directory under `packages/`.**
Those are upstream Helm charts vendored via `make update` (which runs `helm pull`).
Any direct edit is overwritten on the next update and provides zero value.
If you find an issue that appears to live in vendored chart code:
- For configuration-level changes: suggest overrides in the package root `values.yaml`.
- For structural changes: suggest a patch file in `packages/<name>/patches/` applied by the Makefile.
- For source-code changes in images: suggest a patch in `packages/<name>/images/<name>/patches/`.
- For true upstream bugs: point to the upstream repository and suggest an upstream issue/PR.
- Do NOT suggest creating `charts/patches/` — patches never live inside `charts/`.
Similarly, never propose edits to:
- `vendor/` — Go dependencies. Changes go through `go get` and `go mod tidy`.
- `zz_generated.*.go` — regenerated by `make generate`.
- Package-specific: any `<package-name>` matching a directory under `packages/`
Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer.
Each commit must have a `Signed-off-by:` trailer (produced by `git commit --signoff`).
PR body must contain a release note block:
````text
```release-note
type(scope): human-readable changelog entry
```
````
Flag any PR whose commits lack the Conventional Commits format or signoff, or whose body has no release-note block.
## Helm Chart Conventions
Packages follow an umbrella chart pattern:
- `charts/` — vendored upstream, read-only
- `templates/` — Cozystack-specific extra manifests
- `values.yaml` — override values for the upstream chart
- `values.schema.json` — JSON Schema for dashboard UI and input validation
When reviewing `values.schema.json`:
- `make generate` regenerates this file and strips `title`, `description`, and `x-*` custom annotations. Do not suggest adding fields that will be stripped on the next regeneration.
- Focus on type correctness, `required` fields, `enum` values, and default values.
- Flag breaking changes: removing a field, changing its type, or narrowing its enum.
## Sensitive Components
**`packages/core/platform/`**: the platform chart that deploys everything. Changes here can require updates to the migration flow — the Helm hook in `packages/core/platform/templates/migration-hook.yaml` and the runner image with scripts in `packages/core/platform/images/migrations/`. Flag any change to this package that lacks a corresponding migration update or an explicit note that backward compatibility is preserved.
**`api/apps/v1alpha1/`**: app CRD Kinds are registered at runtime from `ApplicationDefinition` resources and the matching `values.schema.json`. Suggest changes to the relevant package schema rather than hand-editing generated types.
**RBAC, ServiceAccounts, and SecurityContext**: flag overly broad RBAC (`*` on resources or verbs without justification), missing `securityContext`, containers running as root without an explicit reason, and `hostPath`/`hostNetwork` usage without clear rationale.
## Go Code Standards
- Use controller-runtime patterns for reconcilers.
- Use structured logging via `logr` — flag `fmt.Print*` and `log.Print*` in controller code.
- Handle errors explicitly. Discarding meaningful errors with `_` is a bug.
- Propagate `context.Context` through call chains. Flag `context.Background()` created inside a reconciler or request handler.
- Prefer `ctrl.Result{RequeueAfter: ...}` over empty requeue for predictable reconciliation loops.
- Tests live beside the code (`*_test.go`). New behavior without tests is worth flagging.
--prompt "Generate the release changelog for tag v${VERSION}. Follow the instructions in @docs/agents/changelog.md exactly, including the 'Scope and boundaries' section at the top. Your deliverable is the single file docs/changelogs/v${VERSION}.md — write it and exit; this workflow handles branching, committing, pushing, and opening the PR." \
const version = '${{ steps.tag.outputs.version }}';
const changelogBranch = `changelog-v${version}`;
const baseBranch = 'main';
// Check if PR already exists
const prs = await github.rest.pulls.list({
owner:context.repo.owner,
repo:context.repo.repo,
head:`${context.repo.owner}:${changelogBranch}`,
base:baseBranch,
state:'open'
});
if (prs.data.length > 0) {
const pr = prs.data[0];
console.log(`PR#${pr.number} already exists for changelog branch ${changelogBranch}`);
// Update PR body with latest info
const body = `This PR adds the changelog for release \`v${version}\`.\n\n✅ Changelog has been automatically generated in \`docs/changelogs/v${version}.md\`.`;
await github.rest.pulls.update({
owner:context.repo.owner,
repo:context.repo.repo,
pull_number:pr.number,
body:body
});
console.log(`Updated existing PR#${pr.number}`);
}else {
// Create new PR
const pr = await github.rest.pulls.create({
owner:context.repo.owner,
repo:context.repo.repo,
head:changelogBranch,
base:baseBranch,
title: `docs(release):add changelog for v${version}`,
body:`This PR adds the changelog for release \`v${version}\`.\n\n✅ Changelog has been automatically generated in \`docs/changelogs/v${version}.md\`.`,
draft:false
});
// Add label if needed
await github.rest.issues.addLabels({
owner:context.repo.owner,
repo:context.repo.repo,
issue_number:pr.data.number,
labels:['kind/documentation','automated']
});
console.log(`Created PR#${pr.data.number} for changelog`);
@ -32,4 +32,4 @@ This list is sorted in chronological order, based on the submission date.
| [Urmanac](https://urmanac.com) | @kingdonb | 2024-12-04 | Urmanac is the future home of a hosting platform for the knowledge base of a community of personal server enthusiasts. We use Cozystack to provide support services for web sites hosted using both conventional deployments and on SpinKube, with WASM. |
| [Hidora](https://hikube.cloud) | @matthieu-robin | 2025-09-17 | Hidora is a Swiss cloud provider delivering managed services and infrastructure solutions through datacenters located in Switzerland, ensuring data sovereignty and reliability. Its sovereign cloud platform, Hikube, is designed to run workloads with high availability across multiple datacenters, providing enterprises with a secure and scalable foundation for their applications based on Cozystack. |
| [QOSI](https://qosi.kz) | @tabu-a | 2025-10-04 | QOSI is a non-profit organization driving open-source adoption and digital sovereignty across Kazakhstan and Central Asia. We use Cozystack as a platform for deploying sovereign, GPU-enabled clouds and educational environments under the National AI Program. Our goal is to accelerate the region’s transition toward open, self-hosted cloud-native technologies |
|
| [Cloupard](https://cloupard.kz/) | @serjiott | 2025-12-18 | Cloupard is a public cloud provider offering IaaS and PaaS services via datacenters in Kazakhstan and Uzbekistan. Uses CozyStack on bare metal to extend its managed PaaS offerings. |
| [releasing.md](./docs/agents/releasing.md) | Release process and workflow |
**CRITICAL**: When the user asks you to do something that matches the scope of a documented process, you MUST read the corresponding documentation file and follow the instructions exactly as written.
- **Commits, PRs, git operations** (e.g., "create a commit", "make a PR", "fix review comments", "rebase", "cherry-pick")
- Action: Use labels defined there. Conventions follow the Kubernetes scheme — `kind/*` (type), `area/*` (subsystem), `priority/*` (urgency), `triage/*` (review state), `lifecycle/*` (auto-close), `do-not-merge/*` (PR blockers), `security/*` (severity)
- For `area/*`: accuracy outweighs reuse. If no existing `area/*` truly fits the change, propose a new one via PR (extend `.github/labels.yml` and the scope mapping in `.github/workflows/pr-labeler.yaml`) — do not shoehorn the change into a wrong area. `area/uncategorized` is the auto-labeler fallback; treat it as a signal to pick a fit, create a new area, or correct the PR title
- PR titles: a Conventional Commits header (`type(scope): description`, types from [`contributing.md`](./docs/agents/contributing.md)) auto-applies `kind/*` and `area/*` via `.github/workflows/pr-labeler.yaml`. Append `!` (or add a `BREAKING CHANGE:` footer) to apply `kind/breaking-change`
**Important rules:**
- ✅ **ONLY read the file if the task matches the documented process scope** - do not read files for tasks that don't match their purpose
- ✅ **ALWAYS read the file FIRST** before starting the task (when applicable)
- ✅ **Follow instructions EXACTLY** as written in the documentation
- ✅ **Do NOT skip mandatory steps** (especially in changelog.md)
- ✅ **Do NOT assume** you know the process - always check the documentation when the task matches
- ❌ **Do NOT read files** for tasks that are outside their documented scope
- 📖 **Note**: [`overview.md`](./docs/agents/overview.md) can be useful as a reference to understand project structure and conventions, even when not explicitly required by the task
## Project Overview
@ -29,7 +59,7 @@ working with the **Cozystack** project.
### Conventions
- **Helm Charts**: Umbrella pattern, vendored upstream charts in `charts/`
[](https://www.bestpractices.dev/projects/10177)
# Cozystack
**Cozystack** is a free PaaS platform and framework for building clouds.
**Cozystack** is a free platform and framework for building clouds.
Cozystack is a [CNCF Sandbox Level Project](https://www.cncf.io/sandbox-projects/) that was originally built and sponsored by [Ænix](https://aenix.io/).
This policy applies to the [`cozystack/cozystack`](https://github.com/cozystack/cozystack) repository and to release artifacts produced from it, including Cozystack core components, operators, packaged manifests, container images, and installation assets published by the project.
Cozystack integrates and ships many upstream cloud native components. If you believe a vulnerability originates in an upstream project rather than in Cozystack-specific code, packaging, defaults, or integration logic, please report it to the upstream project as well. If you are unsure, report it to Cozystack first and we will help route or coordinate the issue.
## Supported Versions
As of March 17, 2026, the Cozystack project maintains multiple release lines. Security fixes are prioritized for the latest stable release line and, when needed, backported to other supported lines.
| Version line | Status | Notes |
| --- | --- | --- |
| `v1.1.x` | Supported | Current stable release line. |
| `v1.0.x` | Supported | Previous stable release line; receives security and important maintenance fixes. |
| `v0.41.x` | Limited support | Legacy pre-v1 line during the v0 to v1 transition; critical security and upgrade-blocking fixes may be backported at maintainer discretion. |
| `< v0.41` | Not supported | Please upgrade to a supported release line before requesting a security fix. |
| `alpha`, `beta`, `rc` releases | Not supported | Pre-release builds are for testing and evaluation only. |
Supported versions may change over time as new release lines are cut. The authoritative source for current releases is the GitHub Releases page:
<https://github.com/cozystack/cozystack/releases>
## Reporting a Vulnerability
Please do **not** report security vulnerabilities through public GitHub issues, discussions, pull requests, Telegram, Slack, or other public community channels.
At the moment, this repository does not publish a dedicated private security mailbox in-tree. If you need to report a vulnerability:
1. Contact one of the project maintainers listed in `CODEOWNERS` using an existing private channel you already have.
2. If you do not already have a private maintainer contact, use a public community channel only to request a private contact path, without disclosing any vulnerability details.
Please do not include exploit details, credentials, tokens, private keys, customer data, or other sensitive material in any public message.
When reporting a vulnerability, please include as much of the following as possible:
- affected Cozystack version, tag, or commit
- affected component or package, for example operator, API server, dashboard, installer, or a packaged system component
- deployment environment and provider, for example bare metal, Hetzner, Oracle Cloud, or other infrastructure
- prerequisites and exact reproduction steps
- impact, attack scenario, and expected blast radius
- whether authentication, tenant access, cluster-admin access, or network adjacency is required
- known mitigations or workarounds
- whether you believe the issue also affects an upstream dependency
## What to Expect
The maintainers will aim to:
- acknowledge receipt within 3 business days
- perform an initial triage and severity assessment within 7 business days
- keep the reporter informed as the fix and disclosure plan are developed
Resolution timelines depend on severity, complexity, release branch applicability, and whether coordination with upstream projects is required.
## Disclosure Process
The Cozystack project follows a coordinated disclosure model.
- We ask reporters to keep details private until a fix or mitigation is available and users have had a reasonable opportunity to upgrade.
- When appropriate, maintainers may use GitHub Security Advisories or equivalent coordinated disclosure tooling to manage remediation and public disclosure.
- If appropriate, the project may request or publish a GHSA and/or CVE as part of the disclosure process.
- Fixes will normally be released in the supported version lines affected by the issue, subject to severity and feasibility.
Public disclosure will typically happen through one or more of the following:
- GitHub Releases and release notes
- project changelogs and documentation updates
- GitHub Security Advisories, when used for coordinated disclosure
## Project Security Practices
Security is part of the normal Cozystack development and release process. Current project practices include:
- maintainer-owned review through pull requests and `CODEOWNERS`
- automated pull request checks, including pre-commit validation, unit tests, builds, and end-to-end testing
- release automation with patch releases, release branches, and backport workflows
- ongoing maintenance of packaged dependencies and platform integrations across supported release lines
Because Cozystack is an integration-heavy platform, some vulnerabilities may require coordination across multiple repositories or with upstream maintainers before a public fix can be released.
## Security Fixes and Announcements
Security fixes are published in normal release artifacts whenever possible. Users should monitor:
- the Cozystack website and documentation: <https://cozystack.io>
## Out of Scope
The following are generally out of scope for private security reporting unless there is a clear Cozystack-specific impact:
- vulnerabilities in unsupported or end-of-life Cozystack versions
- issues that require access already equivalent to cluster-admin, node root, or direct infrastructure administrator privileges, unless they bypass an expected Cozystack security boundary
- vulnerabilities that exist only in an upstream dependency and are not introduced or materially worsened by Cozystack packaging, configuration, or defaults
- requests for security best-practice advice without a concrete vulnerability
## Credits
We appreciate responsible disclosure and will credit reporters in public advisories or release notes unless anonymous disclosure is requested.
// Number of OpenBAO replicas. HA with Raft is automatically enabled when replicas > 1. Switching between standalone (file storage) and HA (Raft storage) modes requires data migration.
// +kubebuilder:default:=1
Replicasint`json:"replicas"`
// Explicit CPU and memory configuration for each OpenBAO replica. When omitted, the preset defined in `resourcesPreset` is applied.
// +kubebuilder:default:={}
ResourcesResources`json:"resources,omitempty"`
// Default sizing preset used when `resources` is omitted.
// Timestamp (RFC3339) for point-in-time recovery; empty means latest.
// +kubebuilder:default:=""
RecoveryTimestring`json:"recoveryTime,omitempty"`
// Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name.
// Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect.
// +kubebuilder:default:=true
ExternalAllowICMPbool`json:"externalAllowICMP"`
// Requested running state of the VirtualMachineInstance
// +kubebuilder:default:="Always"
RunStrategyRunStrategy`json:"runStrategy"`
// Virtual Machine instance type.
// +kubebuilder:default:="u1.medium"
InstanceTypestring`json:"instanceType"`
// Virtual Machine preferences profile.
// +kubebuilder:default:="ubuntu"
InstanceProfilestring`json:"instanceProfile"`
// List of disks to attach.
// +kubebuilder:default:={}
Disks[]Disk`json:"disks,omitempty"`
// Networks to attach the VM to.
// +kubebuilder:default:={}
Networks[]Network`json:"networks,omitempty"`
// Deprecated: use networks instead.
// +kubebuilder:default:={}
Subnets[]Network`json:"subnets,omitempty"`
// List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM).
// +kubebuilder:default:={}
Gpus[]GPU`json:"gpus,omitempty"`
// Model specifies the CPU model inside the VMI. List of available models https://github.com/libvirt/libvirt/tree/master/src/cpu_map
// +kubebuilder:default:=""
CpuModelstring`json:"cpuModel"`
// Resource configuration for the virtual machine.
// +kubebuilder:default:={}
ResourcesResources`json:"resources,omitempty"`
// List of SSH public keys for authentication.
// +kubebuilder:default:={}
SshKeys[]string`json:"sshKeys,omitempty"`
// Cloud-init user data.
// +kubebuilder:default:=""
CloudInitstring`json:"cloudInit"`
// Seed string to generate SMBIOS UUID for the VM.
* Set `ownerReferences` so the `BackupJob` is owned by the `Plan`.
**Note:** The `BackupJob` controller resolves the `BackupClass` to determine the appropriate strategy and parameters, based on the `ApplicationRef`. The strategy template is processed with a context containing the `Application` object and `Parameters` from the `BackupClass`.
The Plan controller does **not**:
* Execute backups itself.
@ -159,17 +159,64 @@ The Plan controller does **not**:
---
### 4.2 Storage
### 4.2 BackupClass
**API Shape**
**Group/Kind**
`backups.cozystack.io/v1alpha1, Kind=BackupClass`
TBD
**Purpose**
Define a class of backup configurations that encapsulate strategy and parameters per application type. `BackupClass` is a cluster-scoped resource that allows admins to configure backup strategies and parameters in a reusable way.
**Storage usage**
**Key fields (spec)**
* `Plan` and `BackupJob` reference `Storage` via `TypedLocalObjectReference`.
* Drivers read `Storage` to know how/where to store or read artifacts.
* Core treats `Storage` spec as opaque; it does not directly talk to S3 or buckets.
```go
type BackupClassSpec struct {
// Strategies is a list of backup strategies, each matching a specific application type.
**Note:** Parameters are not stored directly in `Backup`. Instead, they are resolved from `BackupClass` parameters when the backup was created. The storage location is managed by the driver (e.g., Velero's `BackupStorageLocation`) and referenced via parameters in the `BackupClass`.
**Key fields (status)**
```go
@ -290,7 +342,8 @@ type BackupStatus struct {
* Creates a `Backup` in the same namespace (typically owned by the `BackupJob`).
* Populates `spec` fields with:
* The application, storage, strategy references.
* The application reference.
* The strategy reference (resolved from `BackupClass` during `BackupJob` execution).
* `takenAt`.
* Optional `driverMetadata`.
* Sets `status` with:
@ -306,6 +359,8 @@ type BackupStatus struct {
* Anchor `RestoreJob` operations.
* Implement higher-level policies (retention) if needed.
**Note:** Parameters are resolved from `BackupClass` when the `BackupJob` is created. The driver uses these parameters to determine where to store backups. The storage location itself is managed by the driver (e.g., Velero's `BackupStorageLocation` CRD) and is not directly referenced in the `Backup` resource. When restoring, the driver resolves the storage location from the original `BackupClass` parameters or from the driver's own metadata.
---
### 4.5 RestoreJob
@ -353,13 +408,13 @@ type RestoreJobStatus struct {
* Determines effective:
* **Strategy**: `backup.spec.strategyRef`.
* **Storage**: `backup.spec.storageRef`.
* **Storage**: Resolved from driver metadata or `BackupClass` parameters (e.g., `backupStorageLocationName` stored in `driverMetadata` or resolved from the original `BackupClass`).
* **Target application**: `spec.targetApplicationRef` or `backup.spec.applicationRef`.
* If effective strategy’s GVK is one of its supported strategy types → driver is responsible.
3. Behaviour:
* On first reconcile, set `status.startedAt` and `phase = Running`.
@ -414,8 +469,10 @@ The Cozystack backups core API:
* Uses a single group, `backups.cozystack.io`, for all core CRDs.
* Cleanly separates:
* **When & where** (Plan + Storage) – core-owned.
* **When** (Plan schedule) – core-owned.
* **How & where** (BackupClass) – central configuration unit that encapsulates strategy and parameters (e.g., storage reference) per application type, resolved per BackupJob/Plan.
* **Execution** (BackupJob) – created by Plan when schedule fires, resolves BackupClass to get strategy and parameters, then delegates to driver.
* **What backup artifacts exist** (Backup) – driver-created but cluster-visible.
"Interval between telemetry data collection (e.g. 15m, 1h)")
flag.StringVar(&platformSourceURL,"platform-source-url","","Platform source URL (oci:// or https://). If specified, generates OCIRepository or GitRepository resource.")
flag.StringVar(&platformSourceName,"platform-source-name","cozystack-platform","Name for the generated platform source resource and PackageSource")
flag.StringVar(&platformSourceRef,"platform-source-ref","","Reference specification as key=value pairs (e.g., 'branch=main' or 'digest=sha256:...,tag=v1.0'). For OCI: digest, semver, semverFilter, tag. For Git: branch, tag, semver, name, commit.")
flag.StringVar(&cozyValuesSecretName,"cozy-values-secret-name","cozystack-values","The name of the secret containing cluster-wide configuration values.")
flag.StringVar(&cozyValuesSecretNamespace,"cozy-values-secret-namespace","cozy-system","The namespace of the secret containing cluster-wide configuration values.")
flag.StringVar(&cozyValuesNamespaceSelector,"cozy-values-namespace-selector","cozystack.io/system=true","The label selector for namespaces where the cluster-wide configuration values must be replicated.")