This PR prepares the release `v1.2.0`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Chores
- Updated container images across core infrastructure components
(operator, controllers, API services) with new versions and digests
- Migrated Ubuntu container disk image references from temporary to
permanent registry
- Updated image digests for cluster-autoscaler, kubevirt CSI driver, and
monitoring components
- Refreshed dashboard and system service container image versions
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes swapped `job` labels in the `cozy-linstor` VictoriaMetrics
`VMPodScrape` templates.
Before this change:
- `linstor-satellite` metrics were relabeled as `job=linstor-controller`
- `linstor-controller` metrics were relabeled as `job=linstor-satellite`
This caused `linstorControllerOffline` alerts to fire for satellite
endpoints such as `:9942` while reporting that the controller was
unreachable.
After this change:
- `linstor-satellite` metrics keep `job=linstor-satellite`
- `linstor-controller` metrics keep `job=linstor-controller`
This restores consistent alerting and dashboard semantics for LINSTOR
monitoring.
### Release note
```release-note
[linstor] Fix swapped VictoriaMetrics `VMPodScrape` job labels in `cozy-linstor` so `linstorControllerOffline` alerts target the controller instead of satellite endpoints.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected Prometheus relabeling so Linstor controller and satellite
scrapes now receive the correct job labels.
* Ensures metrics are attributed to the proper components, improving
accuracy of monitoring and alerting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
This fixes two issues in the LINSTOR alerts shipped by
`cozy-piraeus-operator`:
- `linstorSatelliteErrorRate` uses a non-existent `name` label in
annotations, which results in `Satellite ""` in notifications.
- `linstorSatelliteErrorRate` can produce false positives when the
`linstor-controller` scrape flaps and historical
`linstor_error_reports_count` counters reappear inside the alert window.
## Test plan
- Verified the rendered rule updates `{{ $labels.name }}` to `{{
$labels.hostname }}` for LINSTOR satellite alerts.
- Verified the `linstorSatelliteErrorRate` expression now requires
stable `up{job="linstor-controller"}` for the full 15 minute window.
- Applied the same logic in a live cluster and confirmed the
false-positive `linstorSatelliteErrorRate` alerts stopped firing once
controller scrape recovered.
## Release note
```release-note
Fix LINSTOR satellite alert annotations and reduce false positives when controller scraping flaps.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Controller alert now evaluates deployment availability and waits 3
minutes before firing to reduce transient noise.
* Satellite error alerts include an additional check requiring the
controller to have been continuously up recently to avoid spurious
triggers.
* Notifications and dashboards now include hostnames for clearer
identification.
* **New Features**
* Added a dedicated warning for controller metrics scrape failures with
a 10-minute delay.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Previously, removing a database from `values.databases` had no effect —
the database and its associated roles (`<db>_admin`, `<db>_readonly`)
persisted in PostgreSQL indefinitely. This made it impossible to
declaratively manage database lifecycle via Helm values.
Added two cleanup stages to the init script, mirroring the existing user
deletion logic:
- **Delete databases** that have the `database managed by helm` comment
but are no longer listed in `values.databases` — active connections are
terminated before dropping
- **Delete orphaned roles** (`<db>_admin`, `<db>_readonly`) with proper
membership revocation (`REVOKE ... FROM`) before `REASSIGN OWNED` /
`DROP OWNED` / `DROP ROLE`
This also fixes the reported issue where creating a database, removing
it, and creating it again would silently retain stale data from the
first instance.
### Release note
```release-note
[postgres] Databases removed from `values.databases` are now properly dropped along with their associated roles. Previously removed databases and roles would persist indefinitely.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced database initialization to automatically remove databases and
roles that were removed from your Helm configuration, avoiding orphaned
resources.
* Now force-terminates active sessions, reassigns owned objects, and
cleans up role memberships to ensure reliable removals.
* Improves consistency of database state during upgrades and
configuration changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes a crash on the Tenant details page when `resourceQuotas` are
configured. The dashboard renders a ResourceQuota table with a "Used"
column that references `$.status.used[_flatMapData_Key]`. When the
flatMap placeholder is not resolved during initial render, the raw
placeholder is passed to the JSONPath parser, causing:
```text
Parse error on line 1: $.status.used[_flatMapData_Key]
Expecting 'STAR', 'SCRIPT_EXPRESSION', 'INTEGER', ... got 'IDENTIFIER'
```
The original `flatmap-dynamic-key.diff` patch that handled this was
removed during the 1.4.0 dashboard upgrade, assuming upstream included
the fix. Upstream adopted the code reordering (flatMap expansion before
customFields resolution) but not the fallback protection for unresolved
placeholders.
Adds a patch to `openapi-k8s-toolkit` that checks for unresolved
`_flatMap*_Key` placeholders after regex substitution and returns `null`
instead of calling `jp.query()` with an invalid expression.
### Release note
```release-note
[dashboard] Fix crash on Tenant details page when resourceQuotas are configured
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Tables now gracefully handle unresolved placeholder fields by treating
them as empty/null instead of attempting lookup, preventing display
errors and preserving previously correct rendering of resolved fields.
* Reduces unexpected array-wrapping in single-value cells so values
display consistently.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds `discovery.k8s.io/endpointslices` read permissions (get, list,
watch) to `cozy:tenant:base` and `cozy:tenant:view:base` ClusterRoles.
Dashboard requests EndpointSlices to display the "Pod serving" section
on the Service details page. Without this permission, tenant users see a
403 error.
### Release note
```release-note
[system] Fix 403 error on Service details page by granting tenants read access to EndpointSlices
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced system permission configurations to improve platform
reliability and ensure proper access controls for core services.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Fixes the "Pod serving" (EndpointSlice) table on the service details
page which displayed "Raw:" prefixes and "Invalid Date" values. The root
cause was a missing `CustomColumnsOverride` for the
`factory-kube-service-details-endpointslice` customization ID referenced
by the `EnrichedTable` component.
Adds column definitions: Pod (`.targetRef.name`), Addresses
(`.addresses`), Ready (`.conditions.ready`), Node (`.nodeName`).
<img width="1113" height="260" alt="Pod serving table"
src="https://github.com/user-attachments/assets/5bf92080-feee-44b3-942f-a019ba388427"
/>
### Release note
```release-note
[dashboard] Fix EndpointSlice rendering in service details — Pod serving table now correctly displays pod name, addresses, readiness, and node.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added new columns to the service endpoints view in the dashboard,
displaying Pod name, IP addresses, Ready status, and assigned Node
information for better visibility into pod serving details.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds `get`/`list`/`watch` permissions for
`storage.k8s.io/storageclasses` to the `cozystack-dashboard-readonly`
ClusterRole.
The dashboard UI fetches StorageClasses to populate dropdowns (e.g. in
the Postgres form), but authenticated users lacked permissions on this
resource, causing "Error" to be displayed instead of the StorageClass
name.
### Release note
```release-note
[dashboard] Fix StorageClass dropdown showing "Error" by adding storageclasses read access to the dashboard readonly ClusterRole
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced dashboard read-only permissions to include access to storage
classes, improving visibility into cluster storage resources and
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Switch kilo from the cozystack/kilo fork to the upstream squat/kilo
repository (v0.7.0).
All functional patches from the fork have been upstreamed:
- `--internal-cidr` flag for IP auto-detection filtering (#403)
- Allowed-location-ips overlap with node IPs fix (#404)
- Auto-detect WireGuard MTU from underlay interface (#406)
- Preferred source for WireGuard routes (#408)
- Cilium IPIP overlay support (#409)
The fork only retained rebranding and CI-specific commits beyond
upstream, which are no longer needed.
Changes:
- Makefile: point to `squat/kilo` repo and `ghcr.io/squat/kilo` image
- values.yaml: updated via `make update` to `0.7.0` with digest pin
### Release note
```release-note
[kilo] Switch from cozystack/kilo fork to upstream squat/kilo v0.7.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Updates**
* Updated the kilo component to use a different source repository and
new version configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Reduces flakiness in E2E tests by addressing the most common failure
modes.
### Kubernetes tenant tests (`run-kubernetes.sh`)
- Increase node Ready timeout from 2m to 5m — CI runners are shared and
resource-constrained
- Fail fast when nodes are not Ready — saves ~7 minutes per failed
attempt by not running LB/NFS tests that will also fail
- Delete stale Kubernetes resources at test start — retries provision
from scratch instead of patching stuck state
- Wait for port-forward to be ready before using it (fixes race
condition)
- Reduce version check polling interval from 5s to 1s
### All app tests (postgres, redis, kafka, clickhouse, mariadb, mongodb,
qdrant, foundationdb, openbao, vminstance, bucket)
- Add pre-cleanup of stale resources from previous failed retries so
each attempt starts clean
### Test runner (`cozytest.sh`)
- Clean up temp directory on test failure (was only cleaned on success,
leaking `/tmp` dirs)
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Improved e2e robustness by deleting stale resources before creation
(ignore-if-missing, bounded time) and killing leftover port-forwards.
* Added an exit cleanup to consistently remove temporary artifacts and
teardown port-forwards on any exit.
* Added readiness polling for forwarded endpoints (curl with timeout);
made API/version retries more responsive.
* Extended node readiness wait (2m → 5m), dump debug info and fail fast
if unready.
* Made load‑balancer reachability checking explicit and more reliable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds 7 missing application resources to the `cozy:tenant:admin:base`
ClusterRole for `apps.cozystack.io`:
foundationdbs, harbors, mongodbs, openbaos, opensearches, qdrants, vpns.
Without these permissions, tenant admins could not create these
applications — the "Add" button was inactive in the dashboard.
### Release note
```release-note
[platform] Fix tenant admins unable to create FoundationDB, Harbor, MongoDB, OpenBAO, OpenSearch, Qdrant, and VPN applications by adding missing RBAC permissions
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Expanded tenant administrator permissions to manage additional
platform resources, including databases, search services, and network
infrastructure components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds missing `baseFactoriesMapping` entries for backup resources (plans,
backupjobs, backups) to the static Navigation resource.
Backup resources are not ApplicationDefinitions, so `ensureNavigation()`
never creates their navigation mappings. Without these mappings, the
OpenUI frontend cannot resolve the `{cluster}` context for backup pages,
producing broken sidebar links with an empty cluster segment (e.g.
`/openapi-ui//tenant-root/api-table/backups.cozystack.io/v1alpha1/plans`
instead of `/openapi-ui/default/tenant-root/...`).
### Release note
```release-note
[dashboard] Fix broken backup menu links by adding missing baseFactoriesMapping entries for backup resources
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Backup resources are now integrated into dashboard navigation,
providing quick access to backup plans, backup job monitoring, and
detailed backup information for streamlined operations management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cozy:tenant:admin:base ClusterRole was missing several application
resources from apps.cozystack.io, preventing tenant admins from creating
them (the "Add" button was inactive in the dashboard).
Add the following resources: foundationdbs, harbors, mongodbs, openbaos,
opensearches, qdrants, vpns.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The dashboard UI fetches StorageClasses via the Kubernetes API to populate
dropdowns (e.g. in the Postgres form), but the cozystack-dashboard-readonly
ClusterRole did not include storage.k8s.io/storageclasses. This caused
authenticated users to see "Error" instead of the StorageClass name.
Add get/list/watch permissions for storageclasses to the dashboard readonly
role, consistent with the existing backupclasses entry.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Update COSI controller/sidecar from commit `c2f6e65` to upstream
release `v0.2.2` and remove two patches that have been merged:
- `89-reconciliation.diff` — fix reconciliation logic for existing
and deleted objects (upstream PR #89)
- `90-bucket-name.diff` — prefix bucket names with `bucket-` using
UID only (upstream PR #90)
Both were contributed by kvaps and included in the v0.2.2 release
(2025-12-08).
Also removes `git` from Dockerfile dependencies as it was only
needed for applying patches.
Note: `make image` needs to be run to rebuild the container image
and update `values.yaml` with the new tag/digest.
### Release note
```release-note
[objectstorage-controller] Update to v0.2.2, drop upstreamed patches
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Updated object storage controller build configuration to streamline
upstream component retrieval and image build process.
* Modified resource deletion handling so deletion flows are centralized
and consistently enforced across create/update events.
* Changed bucket naming scheme—buckets will no longer include the bucket
class name in their identifier and follow the simplified naming pattern.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Extends the `monitoring` system chart with additional optional
configuration for the Alerta Slack plugin, a VMAgent environment label,
and adds a Grafana plugin to the custom image build.
### Alerta — Slack plugin (`alerta.alerts.slack`)
| Value | Env var | Description |
|---|---|---|
| `dashboardUrl` | `DASHBOARD_URL` | Links Slack alert messages back to
the Alerta console |
| `summaryFmt` | `SLACK_SUMMARY_FMT` | Jinja2 template string for
customizing Slack message format |
All new fields are optional and default to empty, preserving existing
behavior when unset.
### VMAgent — external labels (`vmagent.externalLabels`)
- Added optional `environment` label that gets attached to all scraped
metrics (e.g. `Staging`, `Production`)
- When left empty, no label is added and Alerta falls back to its
built-in default of `"Production"`
### Grafana — custom image
- Added
[`marcusolsson-dynamictext-panel`](https://grafana.com/grafana/plugins/marcusolsson-dynamictext-panel/)
plugin to the Grafana image build
Added marcusolsson-dynamictext-panel plugin to the Grafana image build
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[monitoring] Add optional Slack DASHBOARD_URL and SLACK_SUMMARY_FMT configuration for Alerta, optional vmagent environment external label, and marcusolsson-dynamictext-panel Grafana plugin
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an additional Grafana visualization panel to expand dashboard
options.
* Extended Slack alerting with optional dashboard URL and customizable
summary format.
* Added an optional environment label for external metric ingestion to
improve context and filtering.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The hunk header claimed 8/13 lines but only 6/11 were present,
causing "corrupt patch at line 18" during git apply in the
openapi-ui image build.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
The service details page showed "Raw:" and "Invalid Date" in the Pod
serving table because the EnrichedTable referenced customizationId
"factory-kube-service-details-endpointslice" which had no corresponding
CustomColumnsOverride. Add column definitions for Pod, Addresses, Ready,
and Node fields.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
## Developer Experience
### Go types published on pkg.go.dev
Per-app configuration types (Postgres, Kubernetes, Tenant, Redis, etc.)
are now published as a standalone Go module at
`github.com/cozystack/cozystack/api/apps/v1alpha1`. Each Cozystack
application has its own package with strongly-typed structs generated
from `values.yaml`. The module is tagged automatically on every release.
Import example:
```go
import "github.com/cozystack/cozystack/api/apps/v1alpha1/postgresql"
```
### OpenAPI schema released as a static artifact
The full OpenAPI 3.0 schema for the `apps.cozystack.io` API is now
generated at release time and attached to each GitHub release as
`openapi.json`. Previously this schema was only available from a live
running cluster. The schema covers all managed application kinds and
their configuration fields.
### Automated website docs updates on release
When a new Cozystack version is released, a pull request is
automatically opened in the
[cozystack/website](https://github.com/cozystack/website) repository
with updated application reference documentation and the new
`openapi.json`. This replaces the previous daily cron-based sync that
ran in the website repo.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[docs] Added per-app Go types as a standalone module at `github.com/cozystack/cozystack/api/apps/v1alpha1`
[docs] Added `openapi.json` artifact generation on every release
[docs] Automatic PRs on website repo on every release to keep docs up to date
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for many new apps and VM/network CRD schemas
(ClickHouse, FoundationDB, Harbor, Kafka, MariaDB, MongoDB, PostgreSQL,
Redis, RabbitMQ, NATS, Qdrant, and more).
* Produce a versioned OpenAPI v3 spec (openapi.json) as a build
artifact.
* **Documentation**
* Release workflow now updates website docs from each release and
creates/updates PRs for docs changes.
* **Chores**
* Improved release tagging, asset publishing, generation workflows and
Makefile generation integration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace separate pg_terminate_backend + DROP DATABASE calls with a
single DROP DATABASE ... WITH (FORCE) statement. This eliminates the
race window where new sessions could reconnect between termination
and drop. Available since PostgreSQL 13.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Root cause: EXIT trap called _k8s_test_cleanup after subshell exit,
but variables ${port} and ${test_name} were not available in that
context. set -u caught the unset variable and exited with error,
marking a successful test as failed. This caused every kubernetes
test to "fail" on attempt 1, then retry 2 more times with real
failures (NFS PVC timeout because the cluster was being deleted).
Fix: remove trap and cleanup function entirely. Pre-cleanup at
test start handles stale resources. Inline cleanup at test end
handles normal path. On failure, resources are cleaned up by
pre-cleanup on the next retry.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Root cause of NFS test failures: port-forward had `timeout 500s`
(8.3 minutes), but tests after it take up to 26 minutes (node
join 8m + node Ready 5m + LB test 2.5m + NFS 10m). Port-forward
was guaranteed to die mid-test.
Remove the timeout entirely. Cleanup is already handled by:
- EXIT trap (_k8s_test_cleanup does pkill port-forward)
- Explicit cleanup at every exit point
- Job-level timeout-minutes: 120 as backstop
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Increase NFS PVC Bound timeout from 2m to 5m — RWX via kubevirt
CSI provisions an NFS server pod which takes time
- Add pod describe and events dump when NFS test pod fails to
complete, making the root cause visible in CI logs
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Without explicit timeouts, stuck jobs use the GitHub Actions
default of 360 minutes (6 hours). This blocks the concurrency
group and prevents new runs from starting.
Set reasonable limits:
- Build: 30 minutes (normally ~5 min)
- E2E: 120 minutes (normally ~60 min)
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
On resource-constrained CI runners, HelmReleases may take close
to 15 minutes to reconcile, leaving very little time for CAPI
deployments to start and become available. Increase timeouts:
- CAPI deployment discovery: 60s -> 180s
- CAPI deployment availability: 1m -> 5m
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The explicit _k8s_test_cleanup calls cover known error paths, but
set -e can terminate the script at any kubectl/wait command. Add
an EXIT trap to catch these unexpected exits. The function runs
in a cozytest.sh subshell, so the trap is scoped and does not
affect parent traps. Removed 2>/dev/null from kubectl delete in
cleanup to preserve useful error output.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Use a _k8s_test_cleanup helper called at every exit point
instead of a process-scoped EXIT trap. This avoids any
potential trap scope issues and makes cleanup behavior
explicit and visible at each error path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
