Rename the CRD and all related types for better clarity:
- CozystackResourceDefinition -> ApplicationDefinition
- CozystackResourceDefinitionList -> ApplicationDefinitionList
- CozystackResourceDefinitionSpec -> ApplicationDefinitionSpec
- All related nested types updated accordingly
Updated components:
- API types and generated deepcopy code
- Controllers and reconcilers
- Dashboard, lineagecontrollerwebhook, crdmem packages
- CRD YAML definition and Helm chart
- All 25 cozyrds YAML manifests
- Migration scripts and documentation
Added migration 23 to remove old cozystack-resource-definition-crd HelmRelease.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
- Add pattern rule for building cozypkg binaries per platform
(assets-cozypkg-<os>-<arch>) with checksums generation
- Add Version variable to cozypkg CLI injected via ldflags
- Split manifests into separate cozystack-crds.yaml and
cozystack-operator.yaml files
- Update CI workflow to handle CRDs and operator as separate artifacts
- Update e2e tests and upload script for new manifest structure
- Suppress git describe stderr when no tags exist
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Move common-envs.mk and package.mk from scripts/ to hack/ directory.
Update all Makefile includes to use new paths. Remove unused
issue-flux-certificates.sh script.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Add dedicated flux-tenants controller with label selector
--watch-label-selector=sharding.fluxcd.io/key=tenants to handle
tenant workloads separately from platform components.
Update all kubernetes app HelmReleases to:
- Use chartRef with ExternalArtifact instead of OCIRepository sourceRef
- Add sharding.fluxcd.io/key=tenants label
- Add cozystack.io/target-cluster-name label
Update fluxinstall to parse multiple YAML manifest files and
use flux service for storage-adv-addr.
Add cozystack-basics package for core tenant/namespace setup.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Update all CozystackResourceDefinition files to use chartRef
with ExternalArtifact instead of OCIRepository sourceRef.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Implement a controller for BackupJobs referencing a Velero strategy. Creates a Backup.velero.io according to the template in the `Velero.strategy.backups.cozystack.io`.
### Release note
```release-note
[backups] Implement a backup strategy controller for Velero strategies.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Co-authored-by: Andrey Kolkov <androndo@gmail.com>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch narrows the scope of the Velero backup strategy controller to
simply template Velero Backups according to the application being backed
up and the template in the strategy. Creating storage locations is now
out of scope.
### Release note
```release-note
[backups] Implement templating for Velero backups and remove creation of
backup storage locations and volume snapshot locations.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch implements the Reconcile function for BackupJobs with a
Velero strategy ref.
### Release note
```release-note
[backups] Implement the Velero backup strategy controller.
```
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
## What this PR does
This PR splits the monolithic `cozystack-resource-definitions` package
into 25 individual resource definition packages (`*-rd`) for better
modularity and independent versioning.
**Changes:**
- Created 25 separate `*-rd` packages (bootbox-rd, bucket-rd,
clickhouse-rd, etcd-rd, ferretdb-rd, foundationdb-rd, http-cache-rd,
info-rd, ingress-rd, kafka-rd, kubernetes-rd, monitoring-rd, mysql-rd,
nats-rd, postgres-rd, rabbitmq-rd, redis-rd, seaweedfs-rd,
tcp-balancer-rd, tenant-rd, virtual-machine-rd, virtualprivatecloud-rd,
vm-disk-rd, vm-instance-rd, vpn-rd)
- Removed `packages/system/cozystack-resource-definitions`
- Updated platform bundles (paas-hosted, paas-full, distro-full) to
reference individual -rd packages
- Updated `hack/update-crd.sh` to use package-specific directories
Each `*-rd` package contains:
- `Chart.yaml` - package metadata
- `values.yaml` - default values
- `Makefile` - build instructions
- `cozyrds/<name>.yaml` - CRD definition
- `templates/cozyrd.yaml` - Helm template
**Benefits:**
- **Modularity**: Each resource definition is now a standalone package
- **Independent versioning**: Resources can be versioned independently
- **Maintainability**: Easier to update individual resources
- **Build efficiency**: Parallel building of resource packages
### Release note
```release-note
[platform] Split cozystack-resource-definitions into 25 separate *-rd packages for better modularity and independent versioning. Each resource definition is now a standalone package.
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **Refactor**
* Split the monolithic resource-definitions into many independent resource-definition packages (e.g., bootbox-rd, bucket-rd, clickhouse-rd, etc.) for modular deployment and finer-grained management
* Updated deployment bundles to reference the new per-resource releases with uniform cozy-system namespace and CRD dependency
* **Chores**
* Added packaging/Helm stubs (Chart.yaml, Makefile, values, templates) for each new resource-definition
* **Bug Fixes**
* Made CRD path resolution dynamic (NAME validated before assignment)
<sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[kubernetes] Add lb tests for tenant k8s
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Increased readiness and port-forward timeouts to improve stability.
* Added full end-to-end provisioning and validation: automated namespace
and backend deployment, load balancer provisioning, health checks with
retries, reachability validation, and cleanup.
* Provisioning sequence now runs earlier and is duplicated within the
test flow, altering execution order and adding extra validation/cleanup
steps.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Service selector was using app: "${test_name}-backend" but the
Deployment pod template has app: backend. Fixed selector to match
the actual pod labels so endpoints are created correctly.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Run curl directly from the testing container instead of creating
a separate pod with kubectl run. This avoids PodSecurity policy
violations and simplifies the test execution.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
This PR introduces a bare-bones Job backup strategy API type and stubs
out all the boilerplate for a new controller that will handle this
strategy, as well as any others in the `strategy.backups.cozystack.io`
API group.
### Release note
```release-note
[backups] Create stubs and minimal implmentations for controllers for
the strategy.backups.cozystack.io API group.
```
## What this PR does
This PR introduces a bare-bones Job backup strategy API type and stubs
out all the boilerplate for a new controller that will handle this
strategy, as well as any others in the `strategy.backups.cozystack.io`
API group.
### Release note
```release-note
[backups] Create stubs and minimal implmentations for controllers for
the strategy.backups.cozystack.io API group.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch creates a new backups.cozystack.io API group which includes
backup-related resources owned exclusively by Cozystack core. A
cronjob-like controller is implemented to create backup jobs that will
be handled by appropriate third-party or external controllers.
### Release note
```release-note
[backups] Implement the core backup API and an accompanying controller.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This commit refactors the apiserver REST handlers to use typed objects
(appsv1alpha1.Application) instead of unstructured.Unstructured, eliminating
the need for runtime conversions and simplifying the codebase.
Additionally, it fixes an issue where UnstructuredList objects were using
the first registered kind from typeToGVK instead of the kind from the
object's field when multiple kinds are registered with the same Go type.
This is a more comprehensive fix for the problem addressed in
https://github.com/cozystack/cozystack/pull/1630, which was reverted in
https://github.com/cozystack/cozystack/pull/1677.
The fix includes the upstream fix from kubernetes/kubernetes#135537,
which enables short-circuit path for UnstructuredList similar to regular
Unstructured objects, using GVK from the object field instead of
typeToGVK.
Changes:
- Refactored rest.go handlers to use typed Application objects
- Removed unstructured.Unstructured conversions
- Fixed UnstructuredList GVK handling
- Updated dependencies in go.mod/go.sum
- Added e2e test for OpenAPI validation
- Updated Dockerfile
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
The cozy-lib library package got complicated enough to warrant its own
unit tests. Since unit tests are a "good thing" (tm), a somewhat generic
framework for running all kinds of unit tests was introduced into the CI
pipeline and Makefile targets. For now all it runs is `make test`
against the `packages/{library,apps,system,extra}/*` directories,
wherever a `test` target is present in the Makefile, and for now this is
only for the `cozy-lib` Helm library chart.
### Release note
```release-note
[ci,dx] Introduce a scaffold for running unit tests locally and in CI
and add the first unit tests for the cozy-lib helper Helm chart.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
This patch fixes an issue with the Cozystack API server that causes it
to respond with the first registered dynamic list kind when listing
dynamic resources. E.g., when running `k get postgreses`, the raw JSON
response from the cozystack API would be
```json
{
"apiVersion":"apps.cozystack.io/v1alpha1",
"kind":"BootBoxList",
"items": [
{
"apiVersion":"apps.cozystack.io/v1alpha1",
"kind":"Postgres",
...
},
...
],
...
}
```
The root cause is the way the `Typer` interface is implemented for the
`runtime.Scheme` where the dynamic types are registered. Since the base
type for all dynamic types is a `&cozyv1alpha1.Application{}`, all types
are registered under the same key and the `Typer` defaults to the first
`GroupVersionKind` that was registered. Only when a correctly formed
`&unstructured.Unstructured{}` is returned by the API, is this resolving
logic circumvented and the `GroupVersionKind` is instead inferred from
the fields of the returned object. Even an `UnstructuredList` is not
acceptable as a return type, instead the `items` key should be directly
set on the underlying `map[string]interface{}`.
This patch implements the changes detailed above. Additionally, the
following features, fixes, and improvements are added:
* Makefile targets to build and run the Cozystack API locally, against a
Kubernetes server in the environment's KUBECONFIG. Debugging with
Delve is also supported.
* CI tests are added to verify the new changes.
* A bug in the registration of the corev1alpha1 types is fixed.
* Updated the `ConvertToTable` function to properly handle list kinds
which might be of the `&unstructured.Unstructured{}` concrete type
(not an `UnstructuredList`).
* The scheme used by the API server's Client and WatchClient is
separated from the scheme used to serve dynamic types.
* The client config for reading the runtime configuration now uses the
controller-runtime, which handles flags and environment variables
properly, unlike `clientcmd`.
### Release note
```release-note
[api] Fix incorrect list kind for list requests to the Cozystack API for
dynamic resources. Add Makefile targets for local testing. Minor schema
building improvements.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
The kubelet version of tenant k8s clusters is baked into the worker VM
image. Previously, selecting any version of tenant k8s had an impact
only on the controlplane, the workers were fixed at v1.33. This patch
modifies the KubeadmConfigTemplate to attempt to download the
user-selected versions of kubelet and kubeadm and replace the baked-in
versions with those. If failing, the bootstrap continues with the
baked-in versions.
### Release note
```release-note
[kubernetes] Make kubelet versions on tenant k8s clusters' worker nodes
user-configurable.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch increases the connection and request timeouts used in the
E2E tests when communicating with the Kubernetes API. The change improves
test stability under high load and slow cluster response conditions.
```release-note
[e2e] Increase connection and request timeouts for Kubernetes API calls in E2E tests to improve stability.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
[feature] add ferretdb tests
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- Tests
- Added an end-to-end test that provisions a FerretDB instance on
Kubernetes, waits for readiness and init completion, verifies
connectivity to read/write Postgres services (including endpoint
convergence), accounts for known RO delays, and performs full cleanup.
Uses timeouts and retries for stability.
- Chores
- No user-facing product changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This patch replaces bash-specific [[ ... ]] expressions in the
run_kubernetes_test function with POSIX-compliant case and test
constructs. It ensures that the Kubernetes version on each worker node
is verified correctly and that required components (CoreDNS, Cilium,
ingress-nginx, vsnap-crd) are ready before proceeding. Now the tests
work reliably even when executed with /bin/sh, such as in Bats.
```release-note
[tests] Make Kubernetes tests POSIX-compliant and more reliable:
verify worker node versions and ensure required releases (CoreDNS,
Cilium, ingress-nginx, vsnap-crd) are installed and ready.
```
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Due to a deficiency of cozypkg (--with-source reconciles the HelmChart,
but not the HelmRepository), we have to use workarounds to bulletproof
the latest migration, by applying directly from the assets server.
```release-note
[installer] Run 20th migration using helm charts directly from the
assets server instead of relying on cozypkg to reconcile its resources
properly.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This commit patches the Cozystack API server to tolerate an absence of
Cozystack Resource Definitions either registered as CRDs on the k8s API
or simply as an absence of CozyRDs persisted to etcd. This decouples the
upgrade of the CozyRD CRD from the upgrade of the Cozystack API.
```release-note
[api,platform] Decouple the Cozystack API from the Cozystack Resource
Definitions, allowing independent upgrades of either one and a more
reliable migration from 0.36 to 0.37.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
The etcd tenant module deploys by default with a large resource
limit/request and these values are not exposed at deploy time. This
patch lowers the default resources and adds a VPA to autoconfigure them
according to the real needs.
### Release note
```release-note
[etcd] Attach VPA to etcd and lower initial default resource requests.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Enabled automatic resource autoscaling for etcd with a Vertical Pod
Autoscaler (VPA).
- Chores
- Updated default etcd resource requests to CPU 1000m and memory 512Mi
(previously 4 and 1Gi), reflected across chart values and API schema.
- Changed the output location for generated CRDs.
- Documentation
- Revised README to document the new default CPU and memory values for
etcd.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
Improved tests for verifying installed kubernetes client clusters
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* New Features
* Added node readiness checks (expected node count, detailed node
display) and kubelet version validation with compatibility handling.
* Improvements
* Increased API port-forward timeout and extended rollout/machine
deployment waits for more reliable rollouts.
* Added per-component readiness waits for core cluster services.
* Chores
* Bumped default Kubernetes version to v1.33.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The etcd tenant module deploys by default with a large resource
limit/request and these values are not exposed at deploy time. This
patch lowers the default resources and adds a VPA to autoconfigure them
according to the real needs.
```release-note
[etcd] Attach VPA to etcd and lower initial default resource requests.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
This patch implements name-based selectors for
`CozystackResourceDefinitions.spec.secrets`. Application developers may
now specify secrets that should or should not be visible to end users by
specifying a `resourceNames` field with a string slice of acceptable
names. This will, for instance, let developers exclude a secret like
`postgres-dbname-superuser` that has a predictable name even if it does
not have predictable labels. Simple templates are supported, so
`postgres-{{ .name }}-superuser` is also a valid entry under
`resourceNames`.
```release-note
[lineage, controller] Let application developers determine resource
visibility for end users by name, as well as by labels.
```
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
[cozystack-controller] Introduce new dashboard-controller
[dashboard] Introduce new dashboard based on openapi-ui
Co-authored-by: kklinch0 <kklinch0@gmail.com>
Signed-off-by: kklinch0 <kklinch0@gmail.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
