ClusterIssuer solver referenced IngressClass "nginx" which does not
exist on cozystack clusters — real classes are named after tenant
namespaces (e.g. tenant-root). Cert issuance only worked because every
requesting Ingress overrode the ClusterIssuer via the legacy
acme.cert-manager.io/http01-ingress-class annotation.
Switch both sides to the modern cert-manager API (available since
cert-manager 1.12; cozystack ships 1.19.3):
- ClusterIssuer: http01.ingress.ingressClassName, value parameterized
from _cluster.expose-ingress (default "tenant-root")
- Ingress annotation: http01-ingress-ingressclassname
These must migrate together — mixing ingressClassName (ClusterIssuer)
with the old http01-ingress-class annotation triggers cert-manager's
"fields ingressClassName and class cannot be set at the same time"
validation and breaks issuance.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Without concurrencyPolicy and job limits, the defrag CronJob can
accumulate hundreds of running/failed pods during cluster upgrades
when etcd is temporarily unavailable. This was observed after
upgrading to v1.1.2 where defrag jobs piled up across tenants.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## Summary
- Adds `inlineScrapeConfig` support to the tenant `VMAgent` resource in
`packages/system/monitoring/templates/vm/vmagent.yaml`
- Adds commented usage example in both
`packages/system/monitoring/values.yaml` and
`packages/extra/monitoring/values.yaml`
## Problem
The tenant VMAgent resource does not support custom scrape
configurations. Using `additionalScrapeConfigs` (which references a
Kubernetes Secret) is not viable because tenant users have no access to
create or read Secrets — they can only manage resources under
`apps.cozystack.io`. Instead, `inlineScrapeConfig` accepts inline YAML
directly through Helm values, which is consistent with tenant
permissions.
Relates to #2194
## Usage
```yaml
vmagent:
inlineScrapeConfig: |
- job_name: "custom"
static_configs:
- targets: ["my-service:9090"]
```
## Test plan
- [ ] `helm template` monitoring with `inlineScrapeConfig` set — block
rendered
- [ ] `helm template` monitoring without it — no `inlineScrapeConfig` in
output
- [ ] Deploy and verify custom scrape targets are picked up by vmagent
```release-note
Add inlineScrapeConfig support to tenant vmagent
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Support for including an inline scrape configuration into VMAgent
monitoring when provided.
* **Documentation**
* Added a commented example showing how to supply an inline scrape job
and targets.
* Reordered remote-write URL entries in monitoring configuration for
clearer ordering.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Add external-dns as a standalone self-managed application in
`packages/extra/external-dns/`, allowing tenants to deploy and configure
their own DNS management directly from the dashboard.
## Motivation
Tenants need the ability to manage their own DNS domains with their own
provider. Following the [developers
guide](https://github.com/cozystack/website/pull/413), this is
implemented as an extra package (like `ingress` and `seaweedfs`) using
the HelmRelease-based pattern, rather than embedding it in the tenant
chart.
This enables multi-tenant scenarios where:
- Tenant A uses Cloudflare for `domain-a.com`
- Tenant B uses AWS Route53 for `domain-b.com`
- Each tenant deploys and manages external-dns independently from the
dashboard
## Changes
- **New package**: `packages/extra/external-dns/` — standalone
HelmRelease-based application
- **New PackageSource**:
`packages/core/platform/sources/external-dns-application.yaml` —
references `system/external-dns` and `extra/external-dns` components
- **Cleaned tenant chart**: removed the previously embedded
`externalDns` block from `packages/apps/tenant/`
## Features
- Support for 9 DNS providers: cloudflare, aws, azure, google,
digitalocean, linode, ovh, exoscale, godaddy
- Per-provider credential configuration with full JSON schema validation
- Domain filtering via `domainFilters`
- Configurable sync policy (`sync` or `upsert-only`)
- Namespaced operation (`namespaced: true`) for tenant isolation
- Unique `txtOwnerId` per namespace to prevent DNS record conflicts
- Resource sizing via presets or explicit CPU/memory
## Usage Example
Deploy from the dashboard, or via values:
```yaml
# Cloudflare
provider: cloudflare
domainFilters:
- example.com
cloudflare:
apiToken: "your-cloudflare-api-token"
```
```yaml
# AWS Route53
provider: aws
domainFilters:
- example.org
aws:
accessKeyId: "AKIAXXXXXXXX"
secretAccessKey: "your-secret-key"
region: "us-east-1"
```
## Test plan
- [ ] `helm template external-dns packages/extra/external-dns/ --set
provider=cloudflare --set cloudflare.apiToken=test` renders correctly
- [ ] `helm template external-dns packages/extra/external-dns/` fails
(provider required)
- [ ] `helm template wrong-name packages/extra/external-dns/ --set
provider=cloudflare` fails (release name check)
- [ ] Deploy external-dns from tenant dashboard
- [ ] Verify HelmRelease is created in tenant namespace with namespaced
RBAC
- [ ] Create an Ingress and verify DNS record is created
- [ ] Verify no conflict with global external-dns instance
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an External DNS package for automatic DNS record management.
* Support for 9 DNS providers: Cloudflare, AWS, Azure, Google,
DigitalOcean, Linode, OVH, Exoscale, GoDaddy.
* Helm-based deployment with namespaced/system variants and release
configuration options.
* Configurable synchronization policies, domain filtering, provider
credentials, extra args, and resource presets.
* **Documentation**
* New README and schema-driven values documentation for installation and
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace deprecated single-node VLogs CR with VLCluster (cluster mode)
for reliability and horizontal scalability.
Changes:
- Replace VLogs (v1beta1) with VLCluster (v1) using vlinsert/vlselect/vlstorage
- Update fluent-bit outputs to vlinsert-generic:9481
- Update Grafana datasource to vlselect:9471
- Update ExternalName service from vlogs-generic to vlinsert-generic
- Add VPA for all VLCluster components
- Update WorkloadMonitors for three-component architecture
- Add migration 35 to preserve old VLogs resources with keep annotation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
## What this PR does
Combines and unifies COSI enhancements across seaweedfs and bucket
charts:
**SeaweedFS (extra + system charts):**
- Rename storage pool BucketClass suffix from `-worm` to `-lock`
- Rename parameter `disk` to `diskType` for consistency with COSI driver
- Reduce default object lock retention from 36500 to 365 days
- Add `-lock` BucketClass (COMPLIANCE mode, 365 days) for client and
system topologies
- Add `-readonly` BucketAccessClass with explicit `accessPolicy` for all
topologies
- Add explicit `accessPolicy: readwrite` on default BucketAccessClass
- Update pool name validation to reject `-lock` suffix (was `-worm`)
**Bucket app:**
- Add `locking` parameter: provisions from `-lock` BucketClass
- Add `storagePool` parameter: selects pool-specific BucketClass
- Replace hardcoded BucketAccess with `users` map — each entry creates a
BucketAccess with optional `readonly` flag
- Update dashboard RBAC to dynamically list user credential secrets
- Update ApplicationDefinition schema with new properties
**Breaking change:** empty `users: {}` (default) produces zero
BucketAccess resources. Existing buckets that relied on the implicit
default BucketAccess will need to define users explicitly.
### Release note
```release-note
[apps] Add locking, storagePool, and users configuration to bucket app; rename COSI BucketClass suffix from -worm to -lock
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Bucket locking with a shorter retention option, storage-pool
selectable bucket classes, and per-user access (per-user BucketAccess
and readonly controls)
* S3 Manager login mode: user login/logout, per-session credentials, and
new login UI
* **Behavior Changes**
* Credential handling changed to per-user secrets/label selection;
previously generated secrets removed; Ingress basic auth annotations
removed
* **Documentation**
* Added parameters: locking, storagePool, users (including per-user
readonly)
* **Updates**
* Updated COSI driver and S3 manager images
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The seaweedfs-cosi-driver v0.3.0 expects the parameter key 'disk',
not 'diskType'. Restore the correct key to match the driver's
paramDisk constant.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add COSI resources for object locking and read-only access to both
client topology and system chart:
- BucketClass with -lock suffix (COMPLIANCE mode, 365 days retention)
- BucketAccessClass with -readonly suffix
- Explicit accessPolicy: readwrite on default BucketAccessClass
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Disabled private key rotation in every CA cert in cozystack system packages to prevent trustchain problems when CA cert reissued
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Disabled private-key rotation (set rotationPolicy: Never) for CA/root
certificates used by multiple system components (ingress-nginx, linstor,
linstor-scheduler, seaweedfs, victoria-metrics-operator,
kubeovn-webhook, lineage-controller-webhook, cozystack-api, etcd,
linstor API/internal, seaweedfs).
* Added patch application steps to relevant update workflows to ensure
the certificate template changes are applied during chart/update
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Redesign storage pools architecture:
- Move storagePools map from top-level into volume.pools (Simple topology)
and volume.zones[name].pools (MultiZone topology)
- Add nodeSelector, storageClass, dataCenter overrides for zones
- Add reserved suffix validation (-worm, -readonly) for pool names
- Block volume.pools usage in MultiZone (must use zone.pools instead)
- Use ternary/hasKey pattern for all optional replicas to handle 0 correctly
- Fix nodeSelector rendering for multiline values using indent
- Use disk: parameter (not diskType:) for COSI driver v0.3.0 BucketClass
- Bump seaweedfs-cosi-driver tag to v0.3.0
- Add minimum: 1 constraint for volume/zone/pool replicas in schema
- Regenerate README, CRD, and openAPISchema
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
- Remove dots from pool name regex (K8s resources don't allow dots)
- Add zone×pool name collision validation for MultiZone topology
- Use conditional storageClass rendering to omit empty values
- Fix README resourcesPreset default value
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
- Document MultiZone fallback chain for pool replicas and size
- Move `-volume` WorkloadMonitor reference inside Simple topology block in dashboard-resourcemap.yaml (it is only created for Simple)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Make storageClass optional in storagePools — pools inherit from
volume.storageClass when not explicitly set. Add full COSI resource set
per storage pool: BucketClass, BucketClass-worm (Retain + object lock),
BucketAccessClass readwrite, and BucketAccessClass readonly.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Add optional storagePools configuration that creates separate Volume
StatefulSets per disk type (SSD/HDD/NVMe), enabling tiered storage
within a single SeaweedFS tenant. Each pool gets its own BucketClass
and BucketAccessClass to prepare infrastructure for COSI driver
integration.
Supported in both Simple and MultiZone topologies:
- Simple: one StatefulSet per pool
- MultiZone: one StatefulSet per zone×pool combination
Also adds volume.diskType field for tagging default volume servers.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Dashboard files should only be in system/monitoring.
The dashboards are created by system/monitoring when deployed
via HelmRelease from extra/monitoring.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
WorkloadMonitor resources are only needed when extra/monitoring is
deployed. Move them from system/monitoring to extra/monitoring package.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add migration 26 that re-labels monitoring resources to be owned by
monitoring-system HelmRelease instead of monitoring. This allows
seamless transition from direct resource management in extra/monitoring
to delegated management via system/monitoring chart.
Migration steps:
- Find all monitoring HelmReleases in tenant namespaces
- Suspend HelmRelease to prevent reconciliation
- Delete helm secrets to orphan resources
- Relabel all resources to monitoring-system
- Delete suspended HelmRelease
Also updates:
- monitoring-application.yaml: add monitoring-system component
- helmrelease.yaml: reference monitoring-system ExternalArtifact
- values.yaml: bump targetVersion to 25
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Fixes#749
## Changes
- Add documentation explaining how to enable Hubble for network
observability in Grafana
- Add four Grafana dashboards for Hubble metrics visualization
(Overview, DNS, L7 HTTP, Network)
- Register Hubble dashboards in the monitoring package
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced Hubble DNS Overview dashboard for namespace-level DNS
metrics visibility.
* Introduced Hubble L7 HTTP Metrics dashboard for HTTP request and
performance monitoring.
* Introduced Hubble Network Overview dashboard for network flows and
policy insights.
* **Documentation**
* Added comprehensive Hubble observability guide covering setup,
configuration, and troubleshooting.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Increase startup probe failureThreshold to 300 (25 minutes) to allow
etcd members more time to sync with the cluster after restart or
recovery. This prevents pods from being killed during initial
synchronization when VPA assigns minimal resources.
Also increase liveness probe failureThreshold to 10 to reduce
unnecessary restarts during temporary network issues.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Move common-envs.mk and package.mk from scripts/ to hack/ directory.
Update all Makefile includes to use new paths. Remove unused
issue-flux-certificates.sh script.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add dedicated flux-tenants controller with label selector
--watch-label-selector=sharding.fluxcd.io/key=tenants to handle
tenant workloads separately from platform components.
Update all kubernetes app HelmReleases to:
- Use chartRef with ExternalArtifact instead of OCIRepository sourceRef
- Add sharding.fluxcd.io/key=tenants label
- Add cozystack.io/target-cluster-name label
Update fluxinstall to parse multiple YAML manifest files and
use flux service for storage-adv-addr.
Add cozystack-basics package for core tenant/namespace setup.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
By its nature, the kubernetes client provisioned in keycloak is public:
anyone can read the client secret. Therefore the secret is not
necessary and the client should be explicitly provisioned as a public
client, not as a confidential client.
### Release note
```release-note
[keycloak] Change the kubernetes client to be public, without a client
secret.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Replace Helm lookup functions with FluxCD valuesFrom mechanism for
reading cluster and namespace configuration.
Changes:
- Create Secret cozystack-values in each namespace with values.yaml key
containing _cluster and _namespace configuration as nested YAML
- Configure HelmReleases to read from this Secret via valuesFrom
(valuesKey defaults to values.yaml, so it can be omitted)
- Update cozy-lib helpers to access config via .Values._cluster
- Add default values for required _cluster keys to ensure all fields exist
- Update Go code (cozystack-api and helm reconciler) to use new format
This eliminates the need for Helm lookup functions while maintaining
the same configuration interface for charts.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
