Commit graph

3 commits

Author SHA1 Message Date
Arsolitt
16b2fd008b
docs(monitoring): comment bats regex rule-name convention
Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-19 11:10:43 +03:00
Arsolitt
ccfec2ef62
fix(monitoring): close DCGM coverage gap for gpu-fleet TDP panel
gpu-fleet.json references DCGM_FI_DEV_POWER_MGMT_LIMIT for its
"TDP vs draw" panel, but the custom DCGM Exporter CSV did not declare
it, so the panel silently rendered "No data" on clusters using that
config. Declare the counter, fix the dashboards table in the
gpu-operator examples README, and add a bats test that cross-checks
every DCGM_FI_* reference in tracked dashboards and recording rules
against the union of the upstream default set (snapshotted under
hack/) and the project's custom CSV.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:55:17 +03:00
Arsolitt
49c1d7e7ab
test(monitoring): cross-check GPU dashboards against recording rules
Catch dangling references at PR time: every recording-rule name used
inside a tracked GPU dashboard must exist in
packages/system/monitoring-agents/alerts/gpu-recording.rules.yaml. The
first iteration of gpu-efficiency.json shipped panels keyed on
pod:tensor_saturation:avg5m without the rule defined; the test fails
on exactly that class of bug.

Scoped to dashboards listed under gpu/* in dashboards-infra.list, so
untracked drafts stay out of scope until they are registered. Reverse
direction (rule defined but unused) is intentionally NOT enforced —
some rules exist for ad-hoc PromQL or upcoming dashboards.

Auto-discovered by make bats-unit-tests via hack/cozytest.sh.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
2026-04-18 06:22:31 +03:00