vrr/cozystack
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
cozystack/packages/system
History
IvanHunters 52edffbb9a
fix(kamaji): increase memory limits and add startup probe (#2421) 
## Summary

- Increase kamaji controller memory limit from 500Mi to 512Mi
- Increase kamaji controller memory request from 100Mi to 256Mi  
- Add startup probe with 60-second timeout (12 attempts × 5s periods)
- Increase readiness/liveness probe initialDelaySeconds from 5s/15s to
30s

## Problem

The kamaji controller was experiencing frequent CrashLoopBackOff due to
OOMKilled errors. Analysis showed:

- Container was being killed with exit code 137 (OOMKilled) after ~20-25
seconds of runtime
- Memory limit of 500Mi was insufficient for controller initialization
- Readiness probe was failing because it started too early (5s
initialDelay), before the controller finished leader election (~17s)

## Solution

**Memory increase:**
- Limit: 500Mi → 512Mi (based on production testing)
- Request: 100Mi → 256Mi (ensures adequate reservation)

**Startup probe:**
- Added to give controller up to 60 seconds to initialize without being
killed by liveness probe
- 12 attempts × 5s period = 60s maximum startup time

**Probe delays:**
- ReadinessProbe: 5s → 30s initialDelay (controller needs ~17s to
acquire leader lease)
- LivenessProbe: 15s → 30s initialDelay (aligned with readiness)

## Testing

Verified in production cluster:
- Controller runs stable with 0 restarts
- No more OOMKilled events
- Successfully creates kubeconfig secrets for tenant clusters

## Related Issues

Fixes tenant cluster components stuck in ContainerCreating due to
missing kubeconfig secrets (caused by crashing kamaji controller).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Introduced automated health checks using HTTP-based probes to monitor
service status during startup, continuous operation, and readiness to
handle traffic.
* Adjusted container memory resource allocation for enhanced stability
and performance.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-24 23:11:53 +03:00
..
application-definition-crd refactor(api): rename CozystackResourceDefinition to ApplicationDefinition 2026-01-15 22:35:56 +01:00
backup-controller Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
backupstrategy-controller fix(backups): move velero-configmap Role to velero chart (#2459) 2026-04-23 11:57:54 +05:00
bootbox refactor(labels): remove cozystack.io/ui label 2026-01-19 13:59:29 +01:00
bootbox-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
bucket Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
bucket-rd fix(bucket): regenerate schema and docs from values.yaml annotations 2026-03-03 19:32:32 +03:00
capi-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
capi-providers-bootstrap refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
capi-providers-core refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
capi-providers-cpprovider fix(capi-provider): update Kamaji CAPI provider to v0.16.0 2026-02-19 22:12:33 +03:00
capi-providers-infraprovider refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
cert-manager [cert-manager] Added cleanup before crds upgrade 2026-02-26 18:32:35 +05:00
cert-manager-crds [cert-manager] Updated cert-manager to 1.19.3 with new crd packaging 2026-02-26 15:28:19 +03:00
cert-manager-issuers fix(platform): migrate ACME HTTP-01 to ingressClassName API 2026-04-21 17:00:31 +05:00
cilium fix(cilium): rebuild image multi-arch and pin tag to 1.19.3 2026-04-23 17:33:19 +03:00
cilium-networkpolicy refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
clickhouse-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
clickhouse-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
cluster-autoscaler feat(cluster-autoscaler): enable enforce-node-group-min-size by default 2026-02-14 11:42:52 +01:00
clustersecret-operator Add clustersecret-operator system package 2026-02-10 22:47:20 +05:00
coredns fix coredns serviceaccount to match kubernetes bootstrap rbac 2026-02-02 15:32:50 +01:00
cozy-proxy refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
cozystack-api Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
cozystack-basics [system] Fix tenant RBAC for endpointslices read access (#2257) 2026-03-27 17:48:46 +05:00
cozystack-controller Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
cozystack-scheduler feat(scheduler): storage-aware scheduling 2026-04-13 17:15:09 +03:00
dashboard Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
etcd-operator fix(etcd-operator): replace deprecated kube-rbac-proxy image 2026-03-10 08:30:43 +01:00
etcd-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
external-dns feat(extra): add external-dns as standalone extra package 2026-03-10 12:48:01 +01:00
external-dns-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
external-secrets-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
flux-plunger Fix pre-commit check 2026-01-20 03:24:32 +01:00
fluxcd refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
fluxcd-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
foundationdb-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
foundationdb-rd refactor(labels): remove cozystack.io/ui label 2026-01-19 13:59:29 +01:00
gateway-api-crds refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
goldpinger refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
gpu-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
grafana-operator Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
harbor Fix system postgresql images to 17.7-standard-trixie 2026-04-10 12:47:46 +05:00
harbor-rd fix(harbor): remove incorrect tenant module flags 2026-04-21 20:26:13 +02:00
hetzner-robotlb chore(hetzner-robotlb): update robotlb chart to appVersion 0.0.6 2026-04-23 16:57:23 +05:00
http-cache-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
info-rd refactor(labels): remove cozystack.io/ui label 2026-01-19 13:59:29 +01:00
ingress-nginx Replaced direct chart edits with patch files 2026-03-02 12:35:14 +05:00
ingress-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
kafka-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
kafka-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
kamaji fix(kamaji): increase memory limits and add startup probe (#2421) 2026-04-24 23:11:53 +03:00
keycloak fix(platform): migrate ACME HTTP-01 to ingressClassName API 2026-04-21 17:00:31 +05:00
keycloak-configure [keycloak-configure] Make values.yaml self-documenting 2026-04-02 19:29:54 +03:00
keycloak-operator [keycloak-operator] Update to v1.32.0 2026-03-11 15:15:09 +03:00
kilo [kilo] Switch from fork to upstream squat/kilo 2026-03-23 15:18:23 +03:00
kubeovn fix(kube-ovn): bump kube-ovn to v1.15.10 with port-group regression fix 2026-04-21 20:09:19 +02:00
kubeovn-plunger Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
kubeovn-webhook Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
kubernetes-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
kubevirt fix(kubevirt): disable serial console log to fix VM pod initialization 2026-02-21 20:46:31 +01:00
kubevirt-cdi feat(cdi): switch clone strategy to csi-clone 2026-03-10 12:42:01 +01:00
kubevirt-cdi-operator [cdi-operator] Update to v1.64.0 2026-02-20 20:16:33 +01:00
kubevirt-csi-node Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
kubevirt-instancetypes refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
kubevirt-operator [kubevirt-operator] Update to v1.6.4 2026-02-20 20:16:34 +01:00
lineage-controller-webhook Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
linstor Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
linstor-gui fix(platform): migrate ACME HTTP-01 to ingressClassName API 2026-04-21 17:00:31 +05:00
linstor-scheduler feat(scheduler): storage-aware scheduling 2026-04-13 17:15:09 +03:00
local-ccm Update local-ccm v0.3.0 2026-02-10 20:19:03 +01:00
mariadb-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
mariadb-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
metallb Prepare release v1.0.0-alpha.1 2026-01-15 23:32:41 +00:00
metrics-server refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
mongodb-operator Fix pre-commit check 2026-01-20 03:24:32 +01:00
mongodb-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
monitoring fix(platform): migrate ACME HTTP-01 to ingressClassName API 2026-04-21 17:00:31 +05:00
monitoring-agents feat(monitoring): migrate VictoriaLogs from VLogs to VLCluster 2026-03-10 12:04:12 +05:00
monitoring-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
multus Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
nats Add monitoring for NATs 2026-02-17 22:54:12 +01:00
nats-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
nfs-driver refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
objectstorage-controller Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
openbao feat(openbao): add system chart with vendored upstream 2026-02-17 20:03:01 +03:00
openbao-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
opencost Fix pre-commit check 2026-01-20 03:24:32 +01:00
opensearch-operator [opensearch] Address PR review feedback 2026-03-17 11:20:44 +01:00
opensearch-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
piraeus-operator [piraeus-operator] Split LINSTOR controller availability and metrics alerts 2026-03-25 16:11:14 +03:00
piraeus-operator-crds Move CRDs installation logic to piraeus-operator-crds chart 2026-02-11 22:54:56 +01:00
postgres-operator [postgres-operator] Update to v1.27.3 2026-03-17 01:59:28 +03:00
postgres-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
prometheus-operator-crds refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
qdrant [qdrant] Add Qdrant vector database application 2026-02-06 13:30:58 +03:00
qdrant-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
rabbitmq-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
rabbitmq-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
redis-operator refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
redis-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
reloader refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
seaweedfs Prepare release v1.3.0 2026-04-22 07:28:09 +00:00
seaweedfs-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
snapshot-controller refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
tcp-balancer-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
telepresence refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
tenant-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
velero fix(backups): move velero-configmap Role to velero chart 2026-04-23 10:41:41 +05:00
vertical-pod-autoscaler Post upgrade fixes for v1.0 packagesources 2026-01-28 20:08:35 +03:00
vertical-pod-autoscaler-crds refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
victoria-metrics-operator feat(monitoring): upgrade victoria-metrics-operator to v0.68.4 2026-04-20 15:48:53 +03:00
virtualprivatecloud-rd [docs] Update cozyvalues-gen 2026-03-25 15:59:22 +05:00
vm-default-images [vm-default-images] Default to replicated storageClass and document storage requirements 2026-04-14 19:59:20 +05:00
vm-disk-rd [vm-disk] Fix vm-disk-rd 2026-04-14 19:59:19 +05:00
vm-instance-rd chore(vm-instance): regenerate CRD and schema via make generate 2026-03-30 10:55:26 +05:00
vpn-rd [docs] Updated app go types 2026-03-25 15:57:25 +05:00
vsnap-crd refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00
Makefile refactor: move scripts to hack directory 2026-01-15 16:06:56 +01:00