Commit graph

3183 commits

Author SHA1 Message Date
Arsolitt
b64bfcc414
fix(dashboard): deduplicate pending GPU pods by (namespace, pod)
The Pending GPU pods counter on gpu-quotas joined raw
kube_pod_container_resource_requests (per-container series) against
kube_pod_status_phase (per-pod series). Multi-container pods were
counted once per requesting container instead of once per pod, so the
widget over-reported whenever a Pending pod had more than one GPU
container. Collapse the requests to pod level before the join.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 07:36:14 +03:00
Arsolitt
5db6ec3e1f
fix(dashboard): scope GPU panels to selected namespace
Pod-level panels on the efficiency dashboard and DCGM-level panels on
the performance dashboard ignored the $namespace template variable, so
changing it left the visualizations unchanged. Add the filter to each
query. Performance-side queries use the `$namespace|` empty-tolerant
form so host-level DCGM series without a namespace label remain
visible when a specific namespace is selected.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 07:27:33 +03:00
Arsolitt
4e37f64553
docs(gpu-operator): document tolerate-all on compat DaemonSet
Explain why tolerations: [{operator: Exists}] is safe on the driver
compat DaemonSet: the nodeSelector already confines scheduling to GPU
nodes, so the blanket toleration only kicks in when those nodes carry
the dedicated=gpu / nvidia.com/gpu taints that the GPU Operator's
default policy and many deployments apply.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 07:19:05 +03:00
Arsolitt
5b210ac7fd
docs(monitoring): mark gpu-fleet average utilization as legacy NVML
Clarify that the "Average utilization" panel on gpu-fleet reflects the
legacy NVML view (DCGM_FI_DEV_GPU_UTIL) rather than engine-active
profiling. For AI/LLM workloads the NVML number is optimistic; the
gpu-efficiency dashboard carries the profiling-based view.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 07:11:28 +03:00
Arsolitt
2518e09d67
refactor(monitoring): store pod:tensor_saturation as unitless ratio
Align pod:tensor_saturation:avg5m with namespace:tensor_active:avg and
DCGM's native 0..1 range by dropping the * 100 from the recording rule
and multiplying at display time in gpu-efficiency.json. Also scope
pod:util_per_watt:avg5m with avg by (Hostname, gpu, UUID, namespace,
pod) so the series mirrors pod:tensor_saturation's grouping and stays
usable in topk queries.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 07:03:42 +03:00
Arsolitt
ccfec2ef62
fix(monitoring): close DCGM coverage gap for gpu-fleet TDP panel
gpu-fleet.json references DCGM_FI_DEV_POWER_MGMT_LIMIT for its
"TDP vs draw" panel, but the custom DCGM Exporter CSV did not declare
it, so the panel silently rendered "No data" on clusters using that
config. Declare the counter, fix the dashboards table in the
gpu-operator examples README, and add a bats test that cross-checks
every DCGM_FI_* reference in tracked dashboards and recording rules
against the union of the upstream default set (snapshotted under
hack/) and the project's custom CSV.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:55:17 +03:00
Arsolitt
2fa4b3e31c
refactor(monitoring): tighten GPU dashboard queries
- gpu-efficiency: scope Tensor Saturation, Util-per-Watt and Power
  Throttle stats to the $namespace selector. Cluster-wide means were
  misleading when a user had narrowed the dashboard to specific
  tenants — the headline numbers lied relative to the panels below.
- gpu-fleet: show per-node power draw as % of combined TDP cap
  (DCGM_FI_DEV_POWER_MGMT_LIMIT) instead of raw watts. Thresholds
  (60 / 80 %) generalize across GPU SKUs without per-model tuning.
- gpu-quotas: read cluster:gpu_count:allocated from the recording
  rules instead of recomputing sum(kube_pod_container_resource_requests)
  inline. Keeps the dashboard aligned with the canonical definition
  in gpu-recording.rules.yaml so the two can't drift.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:48:33 +03:00
Arsolitt
7eb9fe8ade
refactor(monitoring): drop unused GPU recording rules
- namespace:gpu_count:sum — never consumed by any tracked dashboard;
  the billable view is already covered by namespace:gpu_count:allocated,
  and the admin view by cluster:gpu_count:allocated.
- namespace:energy_joules:sum — no panel integrates joules; kWh
  readings on the tenant dashboard compute their own integrations
  from namespace:power_watts:sum.
- pod:tensor_to_nvml_ratio:avg5m — interesting tenant signal in
  theory, but not wired into any panel and carrying it just burns
  cardinality on large fleets.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:41:45 +03:00
Arsolitt
6d9066f074
feat(monitoring): add GPU fleet and tenants dashboards
- gpu-fleet: cluster-wide admin view — inventory, capacity (total /
  allocated / free), per-node utilization and power, throttling,
  temperatures, XID errors.
- gpu-tenants: per-namespace view — live allocation, utilization,
  tensor saturation, power, and 24h GPU-hours / kWh integrations for
  billing inputs.

Register both under gpu/* in dashboards-infra.list so they ship as
GrafanaDashboard CRs and fall under the bats cross-check introduced
earlier on this branch.

Update examples/README to spell out which DCGM counters each of the
five gpu/* dashboards actually needs on top of the upstream default
CSV — gpu-performance needs profiling and throttling counters,
gpu-efficiency needs profiling, gpu-tenants needs only
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE for its tensor panel, and gpu-fleet
and gpu-quotas work on the default counter set alone.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:35:12 +03:00
Arsolitt
dbed4992b0
fix(monitoring): exclude system namespaces from namespace:gpu_count:allocated
Align namespace:gpu_count:allocated with every other namespace:* rule
by filtering out cozy-*/kube-*. All other per-namespace rules
(gpu_util, tensor_active, fb_used_bytes, power_watts) already exclude
system namespaces, so the label set produced by :allocated diverged
from them — any dashboard variable or join that reads across these
rules could end up with a different namespace list depending on which
rule supplied the :allocated column.

Trade-off: per-namespace GPU accounting for system workloads is no
longer available through this rule. If it's ever needed, add a
dedicated system:gpu_count:allocated rather than widening this one —
the "billable tenant view" invariant is what the filter is protecting.

Cluster-level cluster:gpu_count:allocated intentionally keeps system
pods so it stays aligned with cluster:gpu_count:total and
cluster:gpu_count:free remains meaningful. As a consequence,
sum(namespace:gpu_count:allocated) no longer equals
cluster:gpu_count:allocated; the delta is system-pod GPU usage, which
is fine for the cluster-admin view.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:28:49 +03:00
Arsolitt
49c1d7e7ab
test(monitoring): cross-check GPU dashboards against recording rules
Catch dangling references at PR time: every recording-rule name used
inside a tracked GPU dashboard must exist in
packages/system/monitoring-agents/alerts/gpu-recording.rules.yaml. The
first iteration of gpu-efficiency.json shipped panels keyed on
pod:tensor_saturation:avg5m without the rule defined; the test fails
on exactly that class of bug.

Scoped to dashboards listed under gpu/* in dashboards-infra.list, so
untracked drafts stay out of scope until they are registered. Reverse
direction (rule defined but unused) is intentionally NOT enforced —
some rules exist for ad-hoc PromQL or upcoming dashboards.

Auto-discovered by make bats-unit-tests via hack/cozytest.sh.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:22:31 +03:00
Arsolitt
38c8a37cb5
docs(gpu-operator): clarify minimum required DCGM metrics
The previous wording implied that the entire custom DCGM CSV was
required by the recording rules. In fact only the profiling counters
(DCGM_FI_PROF_*) need to be added on top of the upstream defaults —
everything else the rules consume is already in default-counters.csv.

Add a Verification status block flagging that the minimum-set claim is
derived from the DCGM Exporter version pinned in the currently shipped
gpu-operator package and must be re-checked when that package moves to
a newer release.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:15:58 +03:00
Arsolitt
0e20159bd9
fix(gpu-operator): scope compat DaemonSet to GPU nodes
Restrict the nvidia-driver-compat DaemonSet to nodes labelled
nvidia.com/gpu.present=true (NFD/GPU Operator label). Without the
nodeSelector it was scheduling onto every node — control-plane and
CPU-only workers included — burning a privileged pod slot per host
for no benefit.

Add resource requests and limits to the init and pause containers so
the DaemonSet stays within control-plane budget on small clusters.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:09:44 +03:00
Arsolitt
4e8731b588
refactor(monitoring): rework GPU recording rules
- Drop gpu.recording.30s group: per-GPU 30s aggregates had no consumers
  in tracked dashboards, only burned cardinality.
- Drop namespace:gpu_allocated_count:gauge: identical expression to
  namespace:gpu_count:sum under a different name.
- Reground :allocated on kube_pod_container_resource_requests so it
  reflects what tenants requested (Pending+Running) rather than what
  DCGM currently sees. namespace:gpu_count:sum stays DCGM-based and
  represents actually-running pods; the gap between the two is the
  signal admins want.
- Add namespace:gpu_count:allocated as the per-namespace counterpart.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:02:17 +03:00
Arsolitt
4f8cef47bf
fix(monitoring): restore trailing newline in GPU dashboards
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 05:54:21 +03:00
Arsolitt
11f7d3589b
chore: ignore CLAUDE.local.md
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 05:47:33 +03:00
Arsolitt
7e5f3a7f12
refactor(monitoring): clean up GPU dashboards
Strip Grafana export boilerplate (__inputs, __elements, __requires,
default annotations, embedded datasource inputs) and tighten panel
layouts across the three GPU dashboards. All three continue to use
the $ds_prometheus template variable.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-17 17:46:31 +03:00
Arsolitt
c1b9a06a36
feat(monitoring): expand GPU dashboards — efficiency and quotas
Revise gpu-performance and add two new dashboards, registered in
dashboards-infra.list:

- gpu-efficiency (GPU Efficiency Score) — utilization vs. capacity
  and workload efficiency signals.
- gpu-quotas (GPU Quotas & Allocation) — per-namespace requested vs.
  used GPUs for tenant capacity planning.

All three dashboards use the $ds_prometheus template variable, per
the project convention.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-17 17:08:18 +03:00
Arsolitt
5d6654c6f4
docs(gpu-operator): add native-pod Talos reference manifests
Add reference manifests (not templates) under
packages/system/gpu-operator/examples/ documenting one working
configuration for running CUDA workloads directly in pods on a Talos
cluster, with DCGM metrics that drive the gpu/gpu-performance
dashboard.

- values-native-talos.yaml: Cozystack Package values that disable the
  sandbox path, enable the device plugin, and wire DCGM to the custom
  metrics ConfigMap.
- dcgm-custom-metrics.yaml: ConfigMap extending the default DCGM CSV
  with profiling, ECC, throttling and energy counters used by the
  dashboard and recording rules.
- nvidia-driver-compat.yaml: DaemonSet that stages libnvidia-ml.so.1
  and nvidia-smi from the Talos glibc tree into a location the
  gpu-operator validator inspects. Workaround for
  NVIDIA/gpu-operator#1687.
- README.md: explains why these are shipped as references rather than
  first-class templates (sandbox vs native is a deployment choice),
  and how the pieces connect.

The out-of-the-box values-talos.yaml still targets the sandbox (VFIO
passthrough) scenario. Operators who want native pod GPU workloads can
start from these references.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-17 16:56:03 +03:00
Arsolitt
d1d19e9978
feat(monitoring): add GPU performance Grafana dashboard
Add the gpu/gpu-performance dashboard and register it in the infra
dashboard list. The dashboard provides:

- Cluster overview: total/allocated GPUs, average utilization,
  aggregate power draw.
- Utilization: GPU util (NVML), tensor pipe active (realistic load
  for LLM/AI workloads), graphics engine active, memory copy util.
- Memory: VRAM used/free per GPU.
- Power and temperature per GPU.
- Health: XID errors, power and thermal throttling.

The dashboard relies on DCGM_FI_* metrics plus the cluster:gpu_* and
namespace:gpu_* recording rules added to monitoring-agents.

The JSON follows the cozystack convention — Prometheus data source is
selected via the $ds_prometheus template variable.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-17 16:55:51 +03:00
Arsolitt
cb3e0adf64
feat(monitoring): add GPU recording rules
Add VMRule with recording rules for DCGM metrics at three levels:

- gpu.recording.30s: per-GPU aggregates over 30s windows
- gpu.recording.cluster.1m: cluster-wide totals for overview panels
- gpu.recording.namespace.1m: per-namespace aggregates for tenant
  reporting and GPU-hour calculations

The rules are safe to ship on clusters without DCGM — they evaluate to
empty series when no matching metrics are scraped.

Used by dashboards/gpu/gpu-performance.json.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-17 16:55:42 +03:00
Timofei Larkin
4053fad9ed
[apps] feat(bucket): add WorkloadMonitor and BucketClaim tracking to controller (#2391)
## What this PR does

Adds BucketClaim support to WorkloadMonitorReconciler, bringing S3
buckets in line with how Pods, PVCs, and Services are already tracked as
Workload resources.

- Add WorkloadMonitor CR to the bucket Helm chart with
`app.kubernetes.io/instance` label on BucketClaim for selector matching
- Add `reconcileBucketClaimForMonitor()` following the same pattern as
`reconcilePVCForMonitor()` — watches COSI `BucketClaim` objects and
creates corresponding `Workload` CRDs
- Query SeaweedFS bucket size metrics (logical + physical) from
VictoriaMetrics, with the monitoring endpoint resolved automatically
from the `namespace.cozystack.io/monitoring` namespace label
- COSI API types dependency (`container-object-storage-interface-api`)
for typed BucketClaim access

No configuration flags needed — the controller discovers the monitoring
stack for each tenant namespace automatically.

### Result

When a BucketClaim is matched by a WorkloadMonitor, the controller
creates a Workload with S3 storage metrics:

```yaml
apiVersion: cozystack.io/v1alpha1
kind: Workload
metadata:
  name: bucket-bucket-test-billing
  namespace: tenant-testing
  labels:
    workloads.cozystack.io/monitor: bucket-test-billing
  ownerReferences:
  - apiVersion: objectstorage.k8s.io/v1alpha1
    kind: BucketClaim
    name: bucket-test-billing
status:
  kind: bucket
  type: s3
  operational: true
  resources:
    s3-buckets: "1"
    s3-storage-bytes: "10485864"             # 10 MB logical
    s3-physical-storage-bytes: "20971728"    # 20 MB physical (replication factor 2)
```

- `s3-storage-bytes` — logical size (what the user stored), from
`SeaweedFS_s3_bucket_size_bytes`
- `s3-physical-storage-bytes` — physical size (with replicas), from
`SeaweedFS_s3_bucket_physical_size_bytes`
- When monitoring is not configured for the namespace, only `s3-buckets:
1` is tracked
- Sizes refresh every 60 seconds via `RequeueAfter`

Tested on a live dev cluster with SeaweedFS deployed in `tenant-root`
and a bucket created in `tenant-testing`.

### Release note

```release-note
[apps] Add WorkloadMonitor to bucket application. BucketClaims are now tracked as Workload resources with S3 storage size metrics resolved automatically from the tenant monitoring stack.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Monitor COSI bucket claims: creates/updates Workload records
reflecting bucket readiness and requeues periodically when buckets
exist.
* Prometheus-backed bucket storage metrics, with SeaweedFS-aware sizing
when available.
* Helm chart additions: WorkloadMonitor resource and instance label on
BucketClaim templates.

* **Tests**
* Unit tests covering bucket-monitor flows, metric querying, URL
resolution, and requeue behavior.

* **Chores**
  * Updated module dependencies to enable COSI integration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-17 12:37:57 +04:00
ZverGuy
3803e6e070 fix(controller): use dedicated HTTP client instead of http.DefaultClient
Replace http.DefaultClient with a package-level *http.Client with an
explicit 10-second timeout. Avoids sharing the process-wide default
transport with other libraries.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:48 +03:00
ZverGuy
c72fadec3e fix(controller): scope Prometheus query to only matched bucket names
Pass bucket names from BucketClaim.Status.BucketName into the PromQL
query as a bucket=~"name1|name2" filter. This prevents O(N²) load
where N WorkloadMonitors each fetch all buckets globally.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
8183b4669a perf(controller): skip Prometheus calls when no BucketClaims matched
Wrap resolvePrometheusURL and queryAllBucketMetrics in a
len(bucketClaimList.Items) > 0 guard. Avoids unnecessary namespace
GET and HTTP request to Prometheus on every reconcile of non-bucket
WorkloadMonitors (postgres, redis, kubernetes, etc.).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
475d24b029 perf(controller): batch all bucket metrics into single Prometheus query
Replace 2×N per-bucket HTTP requests with a single query that fetches
all SeaweedFS bucket size metrics at once:

  {__name__=~"SeaweedFS_s3_bucket_(size|physical_size)_bytes"}

Results are keyed by bucket name in memory, then looked up per
BucketClaim. This reduces HTTP round-trips from 2N+1 to 2 (one
namespace lookup + one Prometheus query) regardless of bucket count.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
ed9808fad6 perf(controller): resolve Prometheus URL once per reconcile, not per BucketClaim
Move resolvePrometheusURL call before the BucketClaim loop and pass
the URL as parameter. Avoids redundant namespace lookups and reduces
reconcile time from 2×N+1 to 2×N HTTP calls (where N = BucketClaims).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
260ce84e5f fix(controller): distinguish empty buckets from monitoring unavailable
Change queryPrometheusMetric to return (int64, bool) so callers can
emit s3-storage-bytes=0 for empty buckets while omitting the field
entirely when monitoring is not configured.

Also:
- Add RBAC marker for core/namespaces GET (used by resolvePrometheusURL)
- Log namespace read errors instead of silently returning empty URL
- Add TestQueryPrometheusMetricZeroValue test

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
a6f74f3198 refactor(controller): remove s3-buckets count resource from bucket workloads
Bucket existence is already tracked via LifetimeHours. The s3-buckets
count resource produced a meaningless "s3-buckets-Hours" fallback type
in billing. Only storage size metrics (s3-storage-bytes,
s3-physical-storage-bytes) are now set on bucket Workloads.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
40620b2911 fix(controller): fix nil map panic and scientific notation parsing
- Initialize workload.Labels map inside CreateOrUpdate mutate function
  to prevent nil map panic when existing Workload has no labels
- Use strconv.ParseFloat instead of resource.ParseQuantity for
  Prometheus metric values, which may use scientific notation
  (e.g. "1.048576e+06") that ParseQuantity does not support

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
7f7f4b218d refactor(controller): resolve Prometheus URL from namespace labels
Replace static --prometheus-url flag with dynamic resolution from
namespace.cozystack.io/monitoring label. Each tenant namespace knows
which tenant hosts its monitoring stack, so the controller constructs
the vmselect URL automatically. This correctly handles multi-tenant
setups where different tenants may use different monitoring instances.

- Remove PrometheusURL field from WorkloadMonitorReconciler struct
- Remove --prometheus-url flag and prometheusUrl chart value
- Add resolvePrometheusURL() that reads namespace label
- queryPrometheusMetric() now accepts prometheusBaseURL as parameter
- Add tests for resolvePrometheusURL with and without label

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
0e6b8f7ba1 fix(controller): address review findings for BucketClaim reconciler
- Set replicas/minReplicas to 0 in bucket WorkloadMonitor (buckets
  have no pods, minReplicas=1 would mark monitor as not operational)
- Remove dead IsNotFound check on List (List returns empty, not 404)
- Trim trailing slash from PrometheusURL before path append
- Add strings import for TrimRight

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
bfdfe989e0 fix(controller): add HTTP status check and limit response body size
- Check resp.StatusCode before parsing Prometheus response
- Limit response body read to 1 MB via io.LimitReader
- Use strings.HasPrefix in test instead of fragile slice indexing
- Add TestQueryPrometheusMetricServerError for 500 responses

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
04f340ab84 feat(controller): add physical storage size metric for S3 buckets
Query SeaweedFS_s3_bucket_physical_size_bytes alongside the logical
size metric. Physical size includes all replicas and reflects actual
disk usage, while logical size reflects what the user stored.

Refactor queryBucketSizeBytes into generic queryPrometheusMetric
to reuse for both metrics.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
ddbc5f6f83 feat(chart): add prometheus-url flag to cozystack-controller deployment
Pass --prometheus-url to the controller container when configured in
values. This enables querying SeaweedFS bucket size metrics from a
Prometheus-compatible API for S3 bucket billing.

RBAC already covers BucketClaim access via existing wildcard rule
(apiGroups: ['*'], resources: ['*'], verbs: ["get", "list", "watch"]).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
0aab19f500 feat(controller): add SeaweedFS bucket size metrics via Prometheus
Query SeaweedFS_s3_bucket_size_bytes from a Prometheus-compatible API
to populate s3-storage-bytes resource on bucket Workloads. The
Prometheus URL is configurable via --prometheus-url flag. When set,
bucket WorkloadMonitors are requeued every 60s to keep sizes current.

When Prometheus is not configured, buckets still get tracked with
s3-buckets=1 for existence-based billing.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
92286213ab feat(controller): add BucketClaim support to WorkloadMonitorReconciler
Add reconcileBucketClaimForMonitor() that watches COSI BucketClaim
objects and creates Workload CRDs with s3-buckets resource, following
the same pattern as PVC and Service reconcilers. This enables the
billing pipeline to discover and track S3 buckets per tenant.

Changes:
- Add COSI API types dependency (container-object-storage-interface-api)
- Register cosiv1alpha1 scheme in controller main
- Add BucketClaim watch in SetupWithManager
- Add BucketClaim list + reconcile in Reconcile loop
- Add RBAC annotation for objectstorage.k8s.io/bucketclaims
- Add unit tests for BucketClaim reconciliation

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuy
ee4e71b139 feat(bucket): add WorkloadMonitor for billing integration
Add WorkloadMonitor CR to bucket Helm chart so that the billing
pipeline can discover and track S3 buckets. Add instance label
to BucketClaim metadata for WorkloadMonitor selector matching.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:53:43 +03:00
myasnikovdaniil
985cb9c889
fix(linstor): restrict linstor-gui to cozystack-cluster-admin group (#2415)
## What this PR does

Tightens access control on the `linstor-gui` Ingress so it is reachable
only by members of the `cozystack-cluster-admin` Keycloak group (the
same group that grants cluster-admin RBAC on the host cluster).

Before this change, the oauth2-proxy gatekeeper in front of
`linstor-gui` enforced only that the user could authenticate against the
`cozy` realm (`--email-domain=*`, no group restriction). Any realm user
— including tenant-scoped accounts — could reach the UI and the
underlying LINSTOR controller REST API, which the gatekeeper proxies
with a static mTLS client cert and no per-user RBAC.

Changes:

- `packages/system/linstor-gui/templates/gatekeeper.yaml` — add
`--allowed-group=cozystack-cluster-admin` and include `groups` in
`--scope` so the claim is available for validation. Extend the
top-of-file rationale block to document the gate.
- `packages/system/linstor-gui/tests/ingress_auth_test.yaml` — assert
both new args on the rendered Deployment.
- `packages/system/linstor-gui/README.md` — document the group
restriction under "Option 1 — Keycloak-protected Ingress".

No Keycloak-side changes are required: the `cozystack-cluster-admin`
KeycloakRealmGroup and the `groups` client scope are already provisioned
by `keycloak-configure`.

Port-forward access (Option 2) is unchanged; it remains gated by kubectl
RBAC on `cozy-linstor`.

### Release note

```release-note
fix(linstor): restrict `linstor-gui` Ingress access to members of the `cozystack-cluster-admin` Keycloak group.
```
2026-04-17 12:36:11 +05:00
Myasnikov Daniil
9b54e46723
fix(linstor): restrict linstor-gui to cozystack-cluster-admin group
Before this change oauth2-proxy fronting linstor-gui only enforced that
the user could authenticate against the `cozy` Keycloak realm
(`--email-domain=*`, no group restriction). Any realm user could reach
the UI and, through it, the LINSTOR controller REST API — which the
gatekeeper proxies with a static mTLS client cert and no per-user RBAC.

Add `--allowed-group=cozystack-cluster-admin` and include `groups` in
the OIDC scope so the claim is present at validation time. The
`cozystack-cluster-admin` KeycloakRealmGroup and the `groups` client
scope are already provisioned by keycloak-configure, so no cluster-wide
changes are needed.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-04-17 11:32:44 +05:00
Andrei Kvapil
91dc23efda
docs(changelog): add v1.3.0-rc.1 changelog and update agent instructions (#2410)
## What this PR does

Adds the v1.3.0-rc.1 release changelog covering all changes since
v1.2.0, and updates the changelog agent instructions to include two new
side repositories.

**Changelog** (`docs/changelogs/v1.3.0-rc.1.md`):
- Feature Highlights: storage-aware scheduling, LINSTOR GUI, VM Default
Images, WorkloadsReady conditions, cross-namespace VM backup restore
- Covers main repo, website, talm, ansible-cozystack, and
external-apps-example changes
- Backported fixes marked with *(backported to v1.2.x)* annotation

**Agent instructions** (`docs/agents/changelog.md`):
- Adds `external-apps-example` and `ansible-cozystack` to the optional
repositories list

### Screenshots

N/A — documentation only.

### Release note

```release-note
docs(changelog): add changelog for v1.3.0-rc.1
```
2026-04-16 13:19:04 +02:00
Andrei Kvapil
1e1bb3eb37
docs(agents): add external-apps-example and ansible-cozystack to changelog instructions
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-04-16 13:18:18 +02:00
Andrei Kvapil
e76b1ccc69
docs(changelog): add changelog for v1.3.0-rc.1
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-04-16 13:18:14 +02:00
Andrei Kvapil
d9657bc4e9
Release v1.3.0-rc.1 (#2408)
This PR prepares the release `v1.3.0-rc.1`.
2026-04-16 12:30:49 +02:00
Timur Tukaev
53ba998777
Update README.md (#2409)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note

```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated the README's introductory description to refine the platform
positioning and improve clarity on its core capabilities.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-16 11:33:39 +02:00
Timur Tukaev
624f00f9c2
Update README.md
Signed-off-by: Timur Tukaev <90071493+tym83@users.noreply.github.com>
2026-04-16 14:30:51 +05:00
cozystack-ci[bot]
12bf6b0e26 Prepare release v1.3.0-rc.1
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-04-16 08:50:30 +00:00
Andrei Kvapil
1a210a2907
feat(application): add WorkloadsReady condition and Events tab (#2356)
## What this PR does

Adds two features to improve application observability in the dashboard,
plus a bug fix:

### 1. WorkloadsReady condition on Application status
- Queries WorkloadMonitor resources to determine if all application pods
are running
- Exposes `WorkloadsReady` as a separate condition alongside `Ready`
- `Ready` continues to reflect HelmRelease state only — no override — to
preserve backward compatibility with existing tooling (kubectl wait,
GitOps health checks) and avoid false-negative Ready=False during normal
startup windows
- Handles three states: operational, not operational, unknown (pending
reconciliation)
- Fails open on WorkloadMonitor query errors (prefers availability)
- Integrates with Application Watch to emit MODIFIED events on
WorkloadMonitor changes
- Registers cozystack.io/v1alpha1 types in API server scheme with
informer cache

### 2. Events tab in dashboard
- Shows Kubernetes Events scoped to the application's namespace
- Uses status.namespace for Tenant applications (consistent with
Resource Quotas tab)
- Includes both lastTimestamp and eventTime columns for Kubernetes
version compatibility

### 3. Bug fix: WorkloadMonitor Operational status persistence
- Operational field was written to stale `monitor` variable instead of
`fresh` inside RetryOnConflict, so it was never persisted to the cluster

Closes #2359
Closes #2360

### Release note

```release-note
[dashboard] Added Events tab to application detail pages showing namespace-scoped Kubernetes Events
[application] Added WorkloadsReady condition exposing aggregated WorkloadMonitor status
[workloadmonitor] Fixed bug where Operational status was never persisted to the cluster
```
2026-04-16 10:18:55 +02:00
myasnikovdaniil
482d813d01
Add vm-default-images package (#2258)
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
[vm-default-images] Added package that brings set of images that can be used clusterwide
[vm-disk] Updated source "image" for prettier dropdown selection
[vm-disk] Added new source for vm-disk called disk - to use as source vm-disk from same namespace.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Clone VM disks by specifying an existing vm-disk as the source.
* Global default image collection and chart to publish pre-provisioned
images.

* **UI**
* Forms provide selectable lists for default images and existing VM
disks.

* **Migration**
* Migration to rename existing image DataVolumes to the new
default-images naming and bumped migration version.

* **Documentation**
* VM disk docs and README updated to reflect image sourcing and cloning.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-16 08:05:23 +05:00
Aleksei Sviridkin
9cc5deeabe
fix(dashboard): fall back to .firstTimestamp in Event Time column
core/v1 Events populate .lastTimestamp and .firstTimestamp but leave
.eventTime null; events.k8s.io/v1 Events do the opposite. The previous
column bound to .eventTime alone and rendered 'Invalid Date' for every
Helm-generated event.

Extend createTimestampColumn with an optional second jsonPath that is
encoded as a nested reqsJsonPath fallback in the template, and use
.eventTime → .firstTimestamp for Event Time and .lastTimestamp →
.eventTime for Last Seen so both APIs render correctly.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-16 01:10:10 +03:00