CollectCRDNames now requires both apiVersion "apiextensions.k8s.io/v1"
and kind "CustomResourceDefinition", consistent with the validation in
crdinstall.Install.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Remove duplicate "Starting controller manager" log before install
phases, keep only the one before mgr.Start()
- Rename misleading test "document without kind returns error" to
"decoder rejects document without kind" to match actual behavior
- Document Helm uninstall CRD behavior in deployment template comment
- Use --health-probe-bind-address=0 consistently with metrics-bind
- Exclude all dotfiles in verify-crds diff, not just .gitattributes
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The workload reconciler was refactored to use the label
workloads.cozystack.io/monitor but the test still used the old
workloadmonitor.cozystack.io/name label, causing the reconciler to
delete the workload instead of keeping it.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Check both apiVersion and kind when validating embedded CRD manifests
to prevent applying objects with wrong API group
- Move ctrl.SetupSignalHandler() before install phases so CRD and Flux
installs respect SIGTERM instead of blocking for up to 2 minutes
- Replace custom contains/searchString helpers with strings.Contains
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Validate that all parsed objects are CustomResourceDefinition before
applying with force server-side apply. This prevents accidental
application of arbitrary resources if a non-CRD file is placed in
the manifests directory.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Move duplicated YAML parsing (ReadYAMLObjects, ParseManifestFile) and
CRD readiness check (WaitForCRDsEstablished, CollectCRDNames) into a
shared internal/manifestutil package. Both crdinstall and fluxinstall
now import from manifestutil instead of maintaining identical copies.
Replace fluxinstall's time.Sleep(2s) after CRD apply with proper
WaitForCRDsEstablished polling, matching the crdinstall behavior.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Use explicit apiextensions.k8s.io/v1 CRD GVK in waitForCRDsEstablished
instead of fragile objects[0].GroupVersionKind()
- Add TestInstall_crdNotEstablished for context timeout path
- Add --recursive to diff in verify-crds Makefile target
- Document why both crds/ and --install-crds exist in deployment template
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Wait for CRDs to have Established condition after server-side apply,
instead of returning immediately
- Add TestInstall with fake client and interceptor to simulate CRD
establishment
- Add TestInstall_noManifests and TestInstall_writeManifestsFails for
error paths
- Fix fluxinstall/manifests.embed.go: use filepath.Join for OS paths
and restrict permissions from 0666 to 0600 (same fix as crdinstall)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Use filepath.Join instead of path.Join for OS file paths
- Restrict extracted manifest permissions from 0666 to 0600
- Add unit tests for readYAMLObjects, parseManifests, and
WriteEmbeddedManifests including permission verification
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Embed Package and PackageSource CRDs in the operator binary using Go
embed, following the same pattern as --install-flux. The operator applies
CRDs at startup using server-side apply, ensuring they are updated on
every operator restart/upgrade.
This addresses the CRD lifecycle concern: Helm crds/ directory handles
initial install, while the operator manages updates on subsequent
deployments.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- Upgrade CRDs in CozyStack dashboard controller
- Add Ingress proxy timeouts for WebSocket to work without terminations
- Add CFOMapping custom resource
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Fixes a race condition where multiple Packages sharing the same
namespace could overwrite each other's PodSecurity Admission labels via
Server-Side Apply.
When two PackageSources deploy to the same namespace (e.g.
`cozystack.linstor` and `cozystack.linstor-scheduler` both use
`cozy-linstor`), each Package only knew about its own components. If a
Package without `privileged: true` reconciled the namespace, SSA would
remove the `pod-security.kubernetes.io/enforce=privileged` label set by
the other Package.
On Talos clusters (which default to `baseline` PodSecurity enforcement),
this caused privileged pods like LINSTOR satellites to be rejected.
The fix makes each Package check ALL PackageSources and their active
Packages when reconciling a namespace. A namespace is set to
`privileged` if ANY Package has a component with `privileged: true` in
it. This ensures a consistent, holistic decision regardless of
reconciliation order.
### Release note
```release-note
[platform] Fixed namespace PodSecurity label race condition when multiple Packages share a namespace
```
Instead of using per-Package SSA field owners (which is a workaround relying
on SSA mechanics), properly resolve whether a namespace should be privileged
by iterating all PackageSources and their active Packages. A namespace gets
the privileged PodSecurity label if ANY Package has a component with
privileged: true installed in it.
This fixes the race condition where Packages sharing a namespace (e.g.
linstor and linstor-scheduler in cozy-linstor) would overwrite each other's
labels depending on reconciliation order.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
When multiple Packages share a namespace (e.g. cozystack.linstor and
cozystack.linstor-scheduler both use cozy-linstor), the shared SSA field
owner "cozystack-package-controller" caused a race condition: the last
Package to reconcile would overwrite the namespace labels set by others.
This meant that if cozystack.linstor set pod-security.kubernetes.io/enforce=
privileged and then cozystack.linstor-scheduler reconciled without the
privileged flag, SSA would remove the label. On Talos clusters (which
default to "baseline" PodSecurity enforcement), this caused privileged
satellite pods to be rejected with PodSecurity violations.
Fix by using per-Package field owners (cozystack-package-{name}), so each
Package independently manages its own namespace labels without interfering
with other Packages sharing the same namespace.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The mysql chart actually deploys MariaDB via mariadb-operator, but was
incorrectly named "mysql". Rename all references to use the correct
"mariadb" name across the codebase.
Changes:
- Rename packages/apps/mysql -> packages/apps/mariadb
- Rename packages/system/mysql-rd -> packages/system/mariadb-rd
- Rename platform source and bundle references
- Update CRD kind from MySQL to MariaDB
- Update RBAC, e2e tests, backup controller tests
- Keep real MySQL CLI/config tool names unchanged (mysqldump, [mysqld], etc.)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[
- add BackupClass Velero and strategy for VM
- dashboard fixes for BackupClass and simplifed Plan/BackupJob API
]
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add external ips (LoadBalancer services with assigned IP address) count
to the Tenant details page
<img width="2560" height="1330" alt="external ip count on tenant details
page"
src="https://github.com/user-attachments/assets/f2256629-3815-4892-b80d-e150d0fd3908"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] add external ips count (count of services of type LoadBalancer with assigned ip) to the Tenant details page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* External IPs count now shown in the Tenant application Details tab — a
left-column item displays the number of external IPs for LoadBalancer
services.
* Application status now exposes an External IPs Count metric for Tenant
applications, so UI and APIs can surface the number of LoadBalancer
external IPs without changing other behaviors.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
- Add Tenant details page to dashboard
- Add Resource Quota usage to Tenant page in dashboard
<img width="1800" height="939" alt="resource quota in tenant"
src="https://github.com/user-attachments/assets/7f4f6ed6-e9b8-4258-a60b-15f203443a1f"
/>
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] Add Tenant details page and resource quota usage to tenant page
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Resource Quotas section to the Details tab to show allocation
limits and usage.
* Info and Tenant types now display quota data in an interactive table
with columns for resource, limits, and current usage.
* Enhanced quota table presentation with additional columns for
flattened resource keys and values for clearer reporting.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The cozystack-operator's manager cache is filtered to only include
specific secrets and namespaces with certain labels. This caused
telemetry collection to fail because resources like kube-system
namespace, nodes, services, and PVs were not in the cache.
Switch to using mgr.GetAPIReader() which bypasses the cache and
queries the API server directly. This is appropriate for telemetry
since it only runs every 15 minutes.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Introduce a new "External IPs" sidebar and associated dashboard factory
to display services of type LoadBalancer.
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Add External IPs tab to sidebar with all `Service` resources of type
`LoadBalancer` in tenant
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[dashboard] add External IPs tab with `Service` resources of type `LoadBalancer`
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added External IPs menu item in the Administration section for quick
access to load balancer services
* Introduced a new dashboard view displaying external IPs with tabbed
interface for enhanced visibility
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Introduce a new "External IPs" sidebar and associated dashboard factory to display services of type LoadBalancer.
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
This PR introduces a new flux-plunger controller that automatically
fixes HelmRelease resources experiencing "has no deployed releases"
errors by cleaning up corrupted Helm release secrets.
### Problem
When Helm releases fail with "has no deployed releases" error, FluxCD
cannot recover automatically. This typically happens when:
- Helm release secrets are corrupted
- Failed upgrades leave the release in an inconsistent state
- Manual intervention is required to delete the problematic secret
### Solution
The flux-plunger controller:
1. Watches HelmRelease resources for "has no deployed releases" errors
2. Suspends the HelmRelease (with `flux-client-side-apply` field
manager)
3. Identifies and deletes the latest (problematic) Helm release secret
4. Tracks processed versions via annotation to prevent recursive
deletion
5. Unsuspends the HelmRelease to allow FluxCD to retry
**Key features:**
- Only deletes one secret at a time (prevents cascading deletions)
- Uses version tracking: if `latest+1 == processed`, skips (already
handled)
- Handles suspended HelmReleases: only unsuspends if it was suspended by
flux-plunger itself
- Gracefully handles invalid annotations by treating them as unprocessed
### Components
- **Controller:** `internal/controller/fluxplunger/flux_plunger.go`
- **Main:** `cmd/flux-plunger/main.go`
- **Helm Chart:** `packages/system/flux-plunger/`
- **Platform Integration:** Added to `paas-full.yaml` bundle
### Release note
```release-note
[platform] Add flux-plunger controller to automatically fix HelmRelease resources with "has no deployed releases" error by cleaning up corrupted Helm secrets
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced a new controller that automatically resolves HelmRelease
resource failures with built-in health probes and graceful recovery
procedures.
* Enabled by default in system installations.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
This PR adds a new package `local-ccm` to the Cozystack platform.
The local-ccm package provides a Cloud Controller Manager for local
development and testing environments. It integrates the upstream Helm
chart from https://github.com/cozystack/local-ccm (version 0.2.0).
**Changes:**
- Created package structure in `packages/system/local-ccm/`
- Added Makefile with automated upstream chart synchronization from
GitHub
- Added PackageSource definition in
`packages/core/platform/sources/local-ccm.yaml`
- Configured dependency on networking components
- Set privileged mode as required for cloud controller operations
**Package details:**
- Namespace: `cozy-local-ccm`
- Release name: `local-ccm`
- Upstream: https://github.com/cozystack/local-ccm
### Release note
```release-note
[local-ccm] Add local-ccm package for cloud controller manager in local environments
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Local Cloud Controller Manager package providing automatic node
IP detection, internal and external IP support, initialization taint
removal, and continuous node reconciliation with configurable intervals.
* **Chores**
* Updated system component tolerations and scheduling policies to
improve compatibility with cloud infrastructure, including better
handling of node readiness, network initialization, and cloud provider
states.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
