Compare commits
18 commits
main
...
release-1.
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4f309823d2 | ||
|
|
c0004e5c5a | ||
|
|
09edf16b85 | ||
|
|
5c68447763 | ||
|
|
e1f66927cb | ||
|
|
79c23d895a | ||
|
|
82fb7990d5 | ||
|
|
5eb52f00b0 | ||
|
|
fe577a26c6 | ||
|
|
f6da822cd0 | ||
|
|
3c2aeb133e | ||
|
|
c131acc8c6 | ||
|
|
cbc1606c41 | ||
|
|
f0f3b18add | ||
|
|
f384be5475 | ||
|
|
ed89553023 | ||
|
|
41bcb0becb | ||
|
|
5329dd180b |
194 changed files with 244 additions and 12965 deletions
2
.github/ISSUE_TEMPLATE/bug_report.md
vendored
2
.github/ISSUE_TEMPLATE/bug_report.md
vendored
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
name: Bug report
|
||||
about: Create a report to help us improve
|
||||
labels: 'kind/bug'
|
||||
labels: 'bug'
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
|
|
|||
7
.github/PULL_REQUEST_TEMPLATE.md
vendored
7
.github/PULL_REQUEST_TEMPLATE.md
vendored
|
|
@ -1,10 +1,9 @@
|
|||
<!-- Thank you for making a contribution! Here are some tips for you:
|
||||
- Use Conventional Commits for the PR title: `type(scope): description`
|
||||
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
|
||||
- Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples:
|
||||
- System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api
|
||||
- Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes
|
||||
- Development and maintenance: api, hack, tests, ci, docs, maintenance
|
||||
- Scopes for system components: dashboard, platform, cilium, kube-ovn, linstor, fluxcd, cluster-api
|
||||
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes
|
||||
- Scopes for development and maintenance: api, hack, tests, ci, docs, maintenance
|
||||
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer
|
||||
- If it's a work in progress, consider creating this PR as a draft.
|
||||
- Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft.
|
||||
|
|
|
|||
371
.github/labels.yml
vendored
371
.github/labels.yml
vendored
|
|
@ -1,371 +0,0 @@
|
|||
# Cozystack repository labels
|
||||
#
|
||||
# Label conventions follow the Kubernetes scheme:
|
||||
# https://github.com/kubernetes/test-infra/blob/master/label_sync/labels.md
|
||||
#
|
||||
# Synced into the repository by .github/workflows/labels.yaml
|
||||
# (EndBug/label-sync@v2). Edit this file via pull request — UI changes
|
||||
# will be overwritten on the next sync.
|
||||
#
|
||||
# Constraints (enforced by the validate job in labels.yaml):
|
||||
# - description ≤ 100 characters (GitHub REST API limit)
|
||||
# - color is a 6-character hex string (no leading #)
|
||||
# - label names are unique
|
||||
# - aliases do not collide with top-level names
|
||||
#
|
||||
# Categories:
|
||||
# kind/ issue or PR type
|
||||
# priority/ urgency
|
||||
# triage/ review state
|
||||
# lifecycle/ issue or PR lifecycle
|
||||
# area/ subsystem; extensible — add when 3+ open issues exist
|
||||
# do-not-merge/ PR merge blockers
|
||||
# security/ security-finding severity and status (Cozystack-specific)
|
||||
# size: PR size (auto-applied)
|
||||
#
|
||||
# `aliases:` lets EndBug/label-sync rename existing labels without losing
|
||||
# references on already-tagged issues and PRs.
|
||||
#
|
||||
# GitHub-default labels not migrated here (`wontfix`, `invalid`) currently
|
||||
# carry zero issues/PRs in this repo and will be removed in a follow-up
|
||||
# cleanup PR rather than aliased to a different namespace.
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# kind/ — issue or PR type
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: kind/bug
|
||||
color: 'd73a4a'
|
||||
description: Categorizes issue or PR as related to a bug
|
||||
aliases: ['bug']
|
||||
|
||||
- name: kind/feature
|
||||
color: 'a2eeef'
|
||||
description: Categorizes issue or PR as related to a new feature
|
||||
aliases: ['enhancement']
|
||||
|
||||
- name: kind/documentation
|
||||
color: '0075ca'
|
||||
description: Categorizes issue or PR as related to documentation
|
||||
aliases: ['documentation']
|
||||
|
||||
- name: kind/support
|
||||
color: 'd876e3'
|
||||
description: Categorizes issue as a support question
|
||||
aliases: ['question']
|
||||
|
||||
- name: kind/cleanup
|
||||
color: 'c7def8'
|
||||
description: Categorizes issue or PR as related to cleanup of code, process, or technical debt
|
||||
|
||||
- name: kind/regression
|
||||
color: 'e11d21'
|
||||
description: Categorizes issue or PR as related to a regression from a prior release
|
||||
|
||||
- name: kind/flake
|
||||
color: 'f7c6c7'
|
||||
description: Categorizes issue or PR as related to a flaky test
|
||||
|
||||
- name: kind/failing-test
|
||||
color: 'e11d21'
|
||||
description: Categorizes issue or PR as related to a consistently or frequently failing test
|
||||
|
||||
- name: kind/api-change
|
||||
color: 'c7def8'
|
||||
description: Categorizes issue or PR as related to adding, removing, or otherwise changing an API
|
||||
|
||||
- name: kind/breaking-change
|
||||
color: 'e11d21'
|
||||
description: Indicates the change introduces a breaking API or behaviour change
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# priority/ — urgency
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: priority/critical-urgent
|
||||
color: 'e11d21'
|
||||
description: Highest priority. Must be actively worked on as someone's top priority right now
|
||||
|
||||
- name: priority/important-soon
|
||||
color: 'eb6420'
|
||||
description: Must be staffed and worked on either currently, or very soon, ideally in time for the next release
|
||||
|
||||
- name: priority/important-longterm
|
||||
color: 'fbca04'
|
||||
description: Important over the long term, but may not be staffed and/or may need multiple releases to complete
|
||||
|
||||
- name: priority/backlog
|
||||
color: 'fef2c0'
|
||||
description: General backlog priority. Lower than priority/important-longterm
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# triage/ — review state
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: triage/needs-triage
|
||||
color: 'ededed'
|
||||
description: Indicates an issue needs triage by a maintainer
|
||||
|
||||
- name: triage/accepted
|
||||
color: '0e8a16'
|
||||
description: Indicates an issue is ready to be actively worked on
|
||||
|
||||
- name: triage/needs-information
|
||||
color: 'fbca04'
|
||||
description: Indicates an issue needs more information in order to work on it
|
||||
|
||||
- name: triage/not-reproducible
|
||||
color: 'fbca04'
|
||||
description: Indicates an issue can not be reproduced as described
|
||||
|
||||
- name: triage/duplicate
|
||||
color: 'cfd3d7'
|
||||
description: Indicates an issue is a duplicate of another issue
|
||||
aliases: ['duplicate']
|
||||
|
||||
- name: triage/unresolved
|
||||
color: 'cfd3d7'
|
||||
description: Indicates an issue that can not or will not be resolved
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# lifecycle/ — issue or PR lifecycle
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: lifecycle/active
|
||||
color: '1d76db'
|
||||
description: Indicates that an issue or PR is actively being worked on by a contributor
|
||||
|
||||
- name: lifecycle/frozen
|
||||
color: 'db5dd6'
|
||||
description: Indicates that an issue or PR should not be auto-closed due to staleness
|
||||
aliases: ['frozen']
|
||||
|
||||
- name: lifecycle/stale
|
||||
color: 'dadada'
|
||||
description: Denotes an issue or PR has remained open with no activity and has become stale
|
||||
aliases: ['stale']
|
||||
|
||||
- name: lifecycle/rotten
|
||||
color: '795548'
|
||||
description: Denotes an issue or PR that has aged beyond stale and will be auto-closed
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# area/ — subsystem (extensible)
|
||||
# Add a new area/* when there are 3+ open issues on the topic.
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: area/api
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to the cozystack-api aggregated API server
|
||||
|
||||
- name: area/ai
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to AI agent guides, AGENTS.md, docs/agents/
|
||||
|
||||
- name: area/build
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to image build infrastructure, multi-arch support
|
||||
|
||||
- name: area/ci
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to CI workflows, GitHub Actions, automation
|
||||
|
||||
- name: area/dashboard
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to the dashboard / UI
|
||||
|
||||
- name: area/extra
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to tenant-specific modules (packages/extra/)
|
||||
|
||||
- name: area/database
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to managed databases (postgres, mariadb, redis, etcd, kafka, clickhouse)
|
||||
|
||||
- name: area/kubernetes
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to the tenant Kubernetes app
|
||||
|
||||
- name: area/monitoring
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to the monitoring stack (vlogs, vmstack, grafana, workloadmonitor)
|
||||
|
||||
- name: area/networking
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to networking (ingress, gateway, vpn, metallb, cilium, kube-ovn)
|
||||
|
||||
- name: area/platform
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to platform infrastructure (bundle, flux, talos, installer)
|
||||
|
||||
- name: area/release
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to release tooling (changelog, backport, release pipeline)
|
||||
|
||||
- name: area/storage
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to storage (linstor, seaweedfs, bucket, velero, harbor)
|
||||
|
||||
- name: area/testing
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to testing (e2e, bats, unit tests)
|
||||
|
||||
- name: area/virtualization
|
||||
color: 'bfd4f2'
|
||||
description: Issues or PRs related to virtualization (kubevirt, cdi, vmi, vm-import)
|
||||
|
||||
- name: area/uncategorized
|
||||
color: 'fbca04'
|
||||
description: PR auto-labeler could not map title scope to a known area/*; please review
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# do-not-merge/ — PR merge blockers (Prow convention)
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: do-not-merge/work-in-progress
|
||||
color: 'e11d21'
|
||||
description: Indicates that a PR should not merge because it is a work in progress
|
||||
# Both legacy spellings collapse here. EndBug processes aliases sequentially;
|
||||
# the second rename hits a name collision and logs a warning — the legacy
|
||||
# label survives and gets cleaned up in the follow-up dedup PR.
|
||||
aliases: ['do-not-merge', 'do not merge']
|
||||
|
||||
- name: do-not-merge/hold
|
||||
color: 'e11d21'
|
||||
description: Indicates that a PR should not merge because someone has issued /hold
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# Cozystack-specific (preserved)
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: epic
|
||||
color: 'a335ee'
|
||||
description: A large development increment that brings definite value to Cozystack users
|
||||
|
||||
- name: community
|
||||
color: '97458a'
|
||||
description: Community contributions are welcome in this issue
|
||||
|
||||
- name: help wanted
|
||||
color: '008672'
|
||||
description: Extra attention is needed
|
||||
|
||||
- name: good first issue
|
||||
color: '7057ff'
|
||||
description: Good for newcomers
|
||||
|
||||
- name: quality-of-life
|
||||
color: 'aaaaaa'
|
||||
description: QoL improvements
|
||||
|
||||
- name: upstream-issue
|
||||
color: 'aaaaaa'
|
||||
description: Requires resolving an issue in an upstream project
|
||||
|
||||
- name: backport
|
||||
color: 'fbca04'
|
||||
description: Should change be backported on previous release
|
||||
|
||||
- name: backport-previous
|
||||
color: 'fbd876'
|
||||
description: Backport target — previous release line
|
||||
|
||||
- name: release
|
||||
color: 'aaaaaa'
|
||||
description: Releasing a new Cozystack version
|
||||
|
||||
- name: automated
|
||||
color: 'ededed'
|
||||
description: Created by automation
|
||||
|
||||
- name: debug
|
||||
color: '704479'
|
||||
description: Debugging in progress
|
||||
|
||||
- name: sponsored
|
||||
color: '00ff00'
|
||||
description: Sponsored work
|
||||
|
||||
- name: lgtm
|
||||
color: '238636'
|
||||
description: This PR has been approved by a maintainer
|
||||
|
||||
- name: ok-to-test
|
||||
color: '00ff00'
|
||||
description: Indicates a non-member PR is safe to run CI on
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# size: — PR size (auto-applied by sizing bot)
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: 'size:XS'
|
||||
color: '00ff00'
|
||||
description: This PR changes 0-9 lines, ignoring generated files
|
||||
|
||||
- name: 'size:S'
|
||||
color: '77b800'
|
||||
description: This PR changes 10-29 lines, ignoring generated files
|
||||
|
||||
- name: 'size:M'
|
||||
color: 'ebb800'
|
||||
description: This PR changes 30-99 lines, ignoring generated files
|
||||
|
||||
- name: 'size:L'
|
||||
color: 'eb9500'
|
||||
description: This PR changes 100-499 lines, ignoring generated files
|
||||
|
||||
- name: 'size:XL'
|
||||
color: 'ff823f'
|
||||
description: This PR changes 500-999 lines, ignoring generated files
|
||||
|
||||
- name: 'size:XXL'
|
||||
color: 'ffb8b8'
|
||||
description: This PR changes 1000+ lines, ignoring generated files
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# security/ — security-finding severity and status
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
- name: security
|
||||
color: 'aaaaaa'
|
||||
description: Security-related issues and features
|
||||
|
||||
- name: security/critical
|
||||
color: 'd73a4a'
|
||||
description: Critical security vulnerability
|
||||
|
||||
- name: security/high
|
||||
color: 'e99695'
|
||||
description: High severity security finding
|
||||
|
||||
- name: security/medium
|
||||
color: 'f9c513'
|
||||
description: Medium severity security finding
|
||||
|
||||
- name: security/low
|
||||
color: '0e8a16'
|
||||
description: Low severity security finding
|
||||
|
||||
- name: security/triage-needed
|
||||
color: 'fbca04'
|
||||
description: Needs security triage
|
||||
|
||||
- name: security/confirmed
|
||||
color: '1d76db'
|
||||
description: Confirmed vulnerability
|
||||
|
||||
- name: security/false-positive
|
||||
color: 'c5def5'
|
||||
description: Triaged as false positive
|
||||
|
||||
- name: security/accepted-risk
|
||||
color: 'bfd4f2'
|
||||
description: Risk accepted with justification
|
||||
|
||||
- name: security/in-progress
|
||||
color: '0075ca'
|
||||
description: Fix in progress
|
||||
|
||||
- name: security/fixed
|
||||
color: '0e8a16'
|
||||
description: Fix released
|
||||
61
.github/workflows/codegen-drift.yml
vendored
61
.github/workflows/codegen-drift.yml
vendored
|
|
@ -1,61 +0,0 @@
|
|||
name: Codegen Drift Check
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened]
|
||||
paths:
|
||||
- 'api/**'
|
||||
- 'pkg/apis/**'
|
||||
- 'pkg/generated/**'
|
||||
- 'internal/crdinstall/manifests/**'
|
||||
- 'packages/system/cozystack-controller/definitions/**'
|
||||
- 'packages/system/application-definition-crd/definition/**'
|
||||
- 'packages/system/backup-controller/definitions/**'
|
||||
- 'packages/system/backupstrategy-controller/definitions/**'
|
||||
- 'hack/update-codegen.sh'
|
||||
- 'hack/boilerplate.go.txt'
|
||||
- 'Makefile'
|
||||
- 'go.mod'
|
||||
- 'go.sum'
|
||||
- '.github/workflows/codegen-drift.yml'
|
||||
|
||||
concurrency:
|
||||
group: codegen-drift-${{ github.workflow }}-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
codegen-drift:
|
||||
name: Verify generated code is up to date
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
cache: true
|
||||
|
||||
- name: Pre-fetch k8s.io/code-generator module
|
||||
# hack/update-codegen.sh sources kube_codegen.sh from the Go module cache.
|
||||
# The module is not declared in go.mod, so fetch it explicitly at the
|
||||
# version pinned in the script.
|
||||
run: |
|
||||
version=$(grep -oP 'code-generator@\Kv[0-9.]+' hack/update-codegen.sh)
|
||||
tmpdir=$(mktemp -d)
|
||||
cd "$tmpdir"
|
||||
go mod init codegen-fetch
|
||||
go get "k8s.io/code-generator@${version}"
|
||||
|
||||
- name: Run make generate
|
||||
run: make generate
|
||||
|
||||
- name: Fail on drift
|
||||
run: |
|
||||
if [ -n "$(git status --porcelain)" ]; then
|
||||
echo "::error::'make generate' produced changes. Run 'make generate' locally and commit the result."
|
||||
git status --short
|
||||
git diff --color=always
|
||||
exit 1
|
||||
fi
|
||||
84
.github/workflows/labels.yaml
vendored
84
.github/workflows/labels.yaml
vendored
|
|
@ -1,84 +0,0 @@
|
|||
name: Labels
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- .github/labels.yml
|
||||
- .github/workflows/labels.yaml
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- .github/labels.yml
|
||||
- .github/workflows/labels.yaml
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: '17 4 * * 1' # Mondays at 04:17 UTC
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: labels-sync
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
validate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Validate labels.yml schema
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import re, sys, yaml
|
||||
|
||||
path = '.github/labels.yml'
|
||||
data = yaml.safe_load(open(path))
|
||||
errors = []
|
||||
|
||||
# 1. description ≤ 100 chars (GitHub REST API limit)
|
||||
for label in data:
|
||||
desc = label.get('description', '') or ''
|
||||
if len(desc) > 100:
|
||||
errors.append(f"{label['name']}: description {len(desc)} chars (max 100)")
|
||||
|
||||
# 2. color is 6-char hex without leading #
|
||||
for label in data:
|
||||
color = label.get('color', '') or ''
|
||||
if not re.match(r'^[0-9A-Fa-f]{6}$', color):
|
||||
errors.append(f"{label['name']}: bad color {color!r} (must be 6-char hex without #)")
|
||||
|
||||
# 3. unique top-level names
|
||||
names = [label['name'] for label in data]
|
||||
dups = sorted({n for n in names if names.count(n) > 1})
|
||||
for n in dups:
|
||||
errors.append(f"duplicate name: {n}")
|
||||
|
||||
# 4. aliases do not collide with any top-level name
|
||||
name_set = set(names)
|
||||
for label in data:
|
||||
for alias in (label.get('aliases') or []):
|
||||
if alias in name_set:
|
||||
errors.append(f"alias {alias!r} (under {label['name']!r}) collides with a top-level name")
|
||||
|
||||
if errors:
|
||||
for err in errors:
|
||||
print(f"::error::{err}")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"labels.yml schema OK ({len(data)} labels)")
|
||||
PY
|
||||
|
||||
sync:
|
||||
needs: validate
|
||||
if: github.event_name != 'pull_request'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: EndBug/label-sync@v2
|
||||
with:
|
||||
config-file: .github/labels.yml
|
||||
delete-other-labels: false
|
||||
206
.github/workflows/pr-labeler.yaml
vendored
206
.github/workflows/pr-labeler.yaml
vendored
|
|
@ -1,206 +0,0 @@
|
|||
name: PR Auto-Label
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [opened, edited, reopened, synchronize]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
label:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Apply labels from PR title
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
// Conventional Commits types accepted by Cozystack (per docs/agents/contributing.md):
|
||||
// feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert
|
||||
// Mapping below maps a subset to kind/* — types not listed do not produce a kind/*.
|
||||
const typeToKind = {
|
||||
feat: 'kind/feature',
|
||||
fix: 'kind/bug',
|
||||
docs: 'kind/documentation',
|
||||
chore: 'kind/cleanup',
|
||||
refactor: 'kind/cleanup',
|
||||
// style, perf, test, build, ci, revert — no kind mapping
|
||||
};
|
||||
|
||||
// scope -> area/* mapping. Keys are the scopes observed in cozystack issues
|
||||
// and PRs. Add new entries when a scope recurs (3+ times).
|
||||
const scopeToArea = {
|
||||
// area/api
|
||||
'api': 'area/api',
|
||||
'cozystack-api': 'area/api',
|
||||
|
||||
// area/ai
|
||||
'agents': 'area/ai',
|
||||
'ai': 'area/ai',
|
||||
|
||||
// area/build
|
||||
'build': 'area/build',
|
||||
|
||||
// area/ci
|
||||
'ci': 'area/ci',
|
||||
|
||||
// area/dashboard
|
||||
'dashboard': 'area/dashboard',
|
||||
|
||||
// area/database
|
||||
'postgres': 'area/database',
|
||||
'postgres-operator': 'area/database',
|
||||
'mariadb': 'area/database',
|
||||
'mariadb-operator': 'area/database',
|
||||
'redis': 'area/database',
|
||||
'etcd': 'area/database',
|
||||
'kafka': 'area/database',
|
||||
'clickhouse': 'area/database',
|
||||
|
||||
// area/extra
|
||||
'extra': 'area/extra',
|
||||
|
||||
// area/kubernetes
|
||||
'kubernetes': 'area/kubernetes',
|
||||
'apps/kubernetes': 'area/kubernetes',
|
||||
|
||||
// area/monitoring
|
||||
'monitoring': 'area/monitoring',
|
||||
'vlogs': 'area/monitoring',
|
||||
'vmstack': 'area/monitoring',
|
||||
'grafana': 'area/monitoring',
|
||||
'workloadmonitor': 'area/monitoring',
|
||||
|
||||
// area/networking
|
||||
'ingress': 'area/networking',
|
||||
'ingress-nginx': 'area/networking',
|
||||
'gateway': 'area/networking',
|
||||
'vpn': 'area/networking',
|
||||
'metallb': 'area/networking',
|
||||
'cilium': 'area/networking',
|
||||
'kube-ovn': 'area/networking',
|
||||
'tcp-balancer': 'area/networking',
|
||||
'securitygroups': 'area/networking',
|
||||
'cozy-proxy': 'area/networking',
|
||||
|
||||
// area/platform
|
||||
'platform': 'area/platform',
|
||||
'bundle': 'area/platform',
|
||||
'flux': 'area/platform',
|
||||
'fluxcd': 'area/platform',
|
||||
'cluster-api': 'area/platform',
|
||||
'talos': 'area/platform',
|
||||
'installer': 'area/platform',
|
||||
'cozyctl': 'area/platform',
|
||||
'cozystack-engine': 'area/platform',
|
||||
'cozy-lib': 'area/platform',
|
||||
|
||||
// area/release
|
||||
'backport': 'area/release',
|
||||
'release': 'area/release',
|
||||
|
||||
// area/storage
|
||||
'seaweedfs': 'area/storage',
|
||||
'seaweedfs-cosi-driver': 'area/storage',
|
||||
'bucket': 'area/storage',
|
||||
'linstor': 'area/storage',
|
||||
'velero': 'area/storage',
|
||||
'harbor': 'area/storage',
|
||||
'backups': 'area/storage',
|
||||
|
||||
// area/testing
|
||||
'tests': 'area/testing',
|
||||
'e2e': 'area/testing',
|
||||
|
||||
// area/virtualization
|
||||
'kubevirt': 'area/virtualization',
|
||||
'cdi': 'area/virtualization',
|
||||
'vmi': 'area/virtualization',
|
||||
'vm-import': 'area/virtualization',
|
||||
'virtual-machine': 'area/virtualization',
|
||||
'hami': 'area/virtualization',
|
||||
'gpu-operator': 'area/virtualization',
|
||||
};
|
||||
|
||||
const pr = context.payload.pull_request;
|
||||
const title = pr.title || '';
|
||||
const body = pr.body || '';
|
||||
const existing = new Set((pr.labels || []).map(l => l.name));
|
||||
const toAdd = new Set();
|
||||
|
||||
// 1. Strip "[Backport release-1.x]" prefix if present.
|
||||
const backportMatch = title.match(/^\[Backport ([^\]]+)\]\s+(.+)$/);
|
||||
const cleanTitle = backportMatch ? backportMatch[2] : title;
|
||||
if (backportMatch) {
|
||||
toAdd.add('area/release');
|
||||
toAdd.add('backport');
|
||||
}
|
||||
|
||||
// 2. Try Conventional Commits form: type(scope)?(!)?: description
|
||||
const conv = cleanTitle.match(/^([a-z]+)(?:\(([^)]+)\))?(!)?:\s*.+$/);
|
||||
// 3. Fall back to bracket form: [scope] description
|
||||
const bracket = !conv && cleanTitle.match(/^\[([^\]]+)\]\s+.+$/);
|
||||
|
||||
let type = null, scopeStr = null, breaking = false;
|
||||
if (conv) {
|
||||
type = conv[1];
|
||||
scopeStr = conv[2] || null;
|
||||
breaking = !!conv[3];
|
||||
} else if (bracket) {
|
||||
scopeStr = bracket[1];
|
||||
}
|
||||
|
||||
// 4. Detect BREAKING CHANGE: or BREAKING-CHANGE: footer in body.
|
||||
// Conventional Commits 1.0 spec item 16 treats them as synonymous.
|
||||
if (/^BREAKING[ -]CHANGE:/m.test(body)) {
|
||||
breaking = true;
|
||||
}
|
||||
|
||||
// 5. Apply kind/* from type.
|
||||
if (type) {
|
||||
if (typeToKind[type]) {
|
||||
toAdd.add(typeToKind[type]);
|
||||
} else {
|
||||
core.warning(`type "${type}" has no kind/* mapping — typo or new type? See .github/workflows/pr-labeler.yaml typeToKind`);
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Apply area/* from scope. Composite scopes split on comma.
|
||||
const scopes = (scopeStr || '')
|
||||
.split(/,\s*/)
|
||||
.map(s => s.trim())
|
||||
.filter(Boolean);
|
||||
for (const s of scopes) {
|
||||
if (scopeToArea[s]) {
|
||||
toAdd.add(scopeToArea[s]);
|
||||
} else {
|
||||
core.warning(`scope "${s}" has no area/* mapping — consider extending scopeToArea in .github/workflows/pr-labeler.yaml if it recurs`);
|
||||
}
|
||||
}
|
||||
|
||||
// 7. kind/breaking-change.
|
||||
if (breaking) {
|
||||
toAdd.add('kind/breaking-change');
|
||||
}
|
||||
|
||||
// 8. Fallback: no area/* applied -> area/uncategorized.
|
||||
const hasArea = [...toAdd].some(l => l.startsWith('area/'));
|
||||
if (!hasArea) {
|
||||
toAdd.add('area/uncategorized');
|
||||
}
|
||||
|
||||
// 9. Additive only — never remove existing labels.
|
||||
const newLabels = [...toAdd].filter(l => !existing.has(l));
|
||||
if (newLabels.length === 0) {
|
||||
core.info('No new labels to apply');
|
||||
return;
|
||||
}
|
||||
|
||||
await github.rest.issues.addLabels({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: pr.number,
|
||||
labels: newLabels,
|
||||
});
|
||||
core.info(`Applied labels: ${newLabels.join(', ')}`);
|
||||
94
.github/workflows/tags.yaml
vendored
94
.github/workflows/tags.yaml
vendored
|
|
@ -223,7 +223,7 @@ jobs:
|
|||
repo: context.repo.repo,
|
||||
head,
|
||||
base,
|
||||
title: `chore(release): cut v${version}`,
|
||||
title: `Release v${version}`,
|
||||
body: `This PR prepares the release \`v${version}\`.`,
|
||||
draft: false
|
||||
});
|
||||
|
|
@ -255,21 +255,6 @@ jobs:
|
|||
private-key: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
|
||||
owner: cozystack
|
||||
|
||||
# Read-only token for the AI step. Minting a separate scoped token
|
||||
# means the Generate changelog using AI step cannot push branches,
|
||||
# open PRs, or mutate any repository even with --allow-all-tools,
|
||||
# regardless of whether the agent follows the prompt's instructions.
|
||||
- name: Generate read-only GitHub App token
|
||||
id: app-token-read
|
||||
uses: actions/create-github-app-token@v1
|
||||
with:
|
||||
app-id: ${{ secrets.COZYSTACK_CI_APP_ID }}
|
||||
private-key: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
|
||||
owner: cozystack
|
||||
permission-contents: read
|
||||
permission-pull-requests: read
|
||||
permission-metadata: read
|
||||
|
||||
- name: Parse tag
|
||||
id: tag
|
||||
uses: actions/github-script@v7
|
||||
|
|
@ -318,59 +303,50 @@ jobs:
|
|||
|
||||
- name: Generate changelog using AI
|
||||
if: steps.check_changelog.outputs.exists == 'false'
|
||||
timeout-minutes: 30
|
||||
env:
|
||||
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ steps.app-token-read.outputs.token }}
|
||||
VERSION: ${{ steps.tag.outputs.version }}
|
||||
GH_TOKEN: ${{ steps.app-token.outputs.token }}
|
||||
run: |
|
||||
copilot \
|
||||
--prompt "Generate the release changelog for tag v${VERSION}. Follow the instructions in @docs/agents/changelog.md exactly, including the 'Scope and boundaries' section at the top. Your deliverable is the single file docs/changelogs/v${VERSION}.md — write it and exit; this workflow handles branching, committing, pushing, and opening the PR." \
|
||||
copilot --prompt "prepare changelog file for tagged release v${{ steps.tag.outputs.version }}, use @docs/agents/changelog.md for it. Create the changelog file at docs/changelogs/v${{ steps.tag.outputs.version }}.md" \
|
||||
--allow-all-tools --allow-all-paths < /dev/null
|
||||
|
||||
- name: Create changelog branch and commit
|
||||
if: steps.check_changelog.outputs.exists == 'false'
|
||||
env:
|
||||
APP_TOKEN: ${{ steps.app-token.outputs.token }}
|
||||
VERSION: ${{ steps.tag.outputs.version }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
CHANGELOG_FILE="docs/changelogs/v${VERSION}.md"
|
||||
CHANGELOG_BRANCH="changelog-v${VERSION}"
|
||||
|
||||
if [ ! -f "$CHANGELOG_FILE" ]; then
|
||||
echo "::error::Changelog file $CHANGELOG_FILE was not produced by the Generate changelog using AI step"
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -s "$CHANGELOG_FILE" ]; then
|
||||
echo "::error::Changelog file $CHANGELOG_FILE is empty"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Snapshot the file across the branch switch — the checkout below
|
||||
# resets tracked files to match origin/main.
|
||||
TEMP_FILE="$(mktemp)"
|
||||
trap 'rm -f "$TEMP_FILE"' EXIT
|
||||
cp "$CHANGELOG_FILE" "$TEMP_FILE"
|
||||
|
||||
git config user.name "cozystack-ci[bot]"
|
||||
git config user.email "274107086+cozystack-ci[bot]@users.noreply.github.com"
|
||||
git remote set-url origin "https://x-access-token:${APP_TOKEN}@github.com/${GITHUB_REPOSITORY}"
|
||||
|
||||
git fetch origin main
|
||||
git branch -D "$CHANGELOG_BRANCH" 2>/dev/null || true
|
||||
git checkout -b "$CHANGELOG_BRANCH" origin/main
|
||||
|
||||
mkdir -p "$(dirname "$CHANGELOG_FILE")"
|
||||
cp "$TEMP_FILE" "$CHANGELOG_FILE"
|
||||
|
||||
# The `check_changelog` step gated this job on the file being absent
|
||||
# from origin/main, so `git add` + `git commit` must produce a diff.
|
||||
# If they don't, something is wrong (e.g. empty file) — fail loud.
|
||||
git add "$CHANGELOG_FILE"
|
||||
git commit -m "docs: add changelog for v${VERSION}" -s
|
||||
git push -f origin "$CHANGELOG_BRANCH"
|
||||
git remote set-url origin https://x-access-token:${APP_TOKEN}@github.com/${GITHUB_REPOSITORY}
|
||||
|
||||
CHANGELOG_FILE="docs/changelogs/v${{ steps.tag.outputs.version }}.md"
|
||||
CHANGELOG_BRANCH="changelog-v${{ steps.tag.outputs.version }}"
|
||||
|
||||
if [ -f "$CHANGELOG_FILE" ]; then
|
||||
# Fetch latest main branch
|
||||
git fetch origin main
|
||||
|
||||
# Delete local branch if it exists
|
||||
git branch -D "$CHANGELOG_BRANCH" 2>/dev/null || true
|
||||
|
||||
# Create and checkout new branch from main
|
||||
git checkout -b "$CHANGELOG_BRANCH" origin/main
|
||||
|
||||
# Add and commit changelog
|
||||
git add "$CHANGELOG_FILE"
|
||||
if git diff --staged --quiet; then
|
||||
echo "⚠️ No changes to commit (file may already be committed)"
|
||||
else
|
||||
git commit -m "docs: add changelog for v${{ steps.tag.outputs.version }}" -s
|
||||
echo "✅ Changelog committed to branch $CHANGELOG_BRANCH"
|
||||
fi
|
||||
|
||||
# Push the branch (force push to update if it exists)
|
||||
git push -f origin "$CHANGELOG_BRANCH"
|
||||
else
|
||||
echo "⚠️ Changelog file was not generated"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Create PR for changelog
|
||||
if: steps.check_changelog.outputs.exists == 'false'
|
||||
|
|
@ -411,7 +387,7 @@ jobs:
|
|||
repo: context.repo.repo,
|
||||
head: changelogBranch,
|
||||
base: baseBranch,
|
||||
title: `docs(release): add changelog for v${version}`,
|
||||
title: `docs: add changelog for v${version}`,
|
||||
body: `This PR adds the changelog for release \`v${version}\`.\n\n✅ Changelog has been automatically generated in \`docs/changelogs/v${version}.md\`.`,
|
||||
draft: false
|
||||
});
|
||||
|
|
@ -421,7 +397,7 @@ jobs:
|
|||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: pr.data.number,
|
||||
labels: ['kind/documentation', 'automated']
|
||||
labels: ['documentation', 'automated']
|
||||
});
|
||||
|
||||
console.log(`Created PR #${pr.data.number} for changelog`);
|
||||
|
|
|
|||
|
|
@ -1,17 +1,6 @@
|
|||
repos:
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: run-make-generate-root
|
||||
name: Run 'make generate' at repo root
|
||||
entry: |
|
||||
flock -x .git/pre-commit.lock sh -c '
|
||||
echo "Running make generate at repo root"
|
||||
make generate || exit $?
|
||||
git diff --color=always | cat
|
||||
'
|
||||
language: system
|
||||
files: ^(api/|pkg/apis/|pkg/generated/|internal/crdinstall/manifests/|packages/system/cozystack-controller/definitions/|packages/system/application-definition-crd/definition/|packages/system/backup-controller/definitions/|packages/system/backupstrategy-controller/definitions/|hack/update-codegen\.sh$|hack/boilerplate\.go\.txt$|Makefile$)
|
||||
pass_filenames: false
|
||||
- id: run-make-generate
|
||||
name: Run 'make generate' in all app directories
|
||||
entry: |
|
||||
|
|
|
|||
|
|
@ -27,12 +27,6 @@ working with the **Cozystack** project.
|
|||
- Read: [`contributing.md`](./docs/agents/contributing.md)
|
||||
- Action: Read the file to understand git workflow, commit format, PR process
|
||||
|
||||
- **Issue and PR labeling, triage** (e.g., "label this issue", "what label should I use", "triage this", "categorize")
|
||||
- Read: [`.github/labels.yml`](./.github/labels.yml)
|
||||
- Action: Use labels defined there. Conventions follow the Kubernetes scheme — `kind/*` (type), `area/*` (subsystem), `priority/*` (urgency), `triage/*` (review state), `lifecycle/*` (auto-close), `do-not-merge/*` (PR blockers), `security/*` (severity)
|
||||
- For `area/*`: accuracy outweighs reuse. If no existing `area/*` truly fits the change, propose a new one via PR (extend `.github/labels.yml` and the scope mapping in `.github/workflows/pr-labeler.yaml`) — do not shoehorn the change into a wrong area. `area/uncategorized` is the auto-labeler fallback; treat it as a signal to pick a fit, create a new area, or correct the PR title
|
||||
- PR titles: a Conventional Commits header (`type(scope): description`, types from [`contributing.md`](./docs/agents/contributing.md)) auto-applies `kind/*` and `area/*` via `.github/workflows/pr-labeler.yaml`. Append `!` (or add a `BREAKING CHANGE:` footer) to apply `kind/breaking-change`
|
||||
|
||||
**Important rules:**
|
||||
- ✅ **ONLY read the file if the task matches the documented process scope** - do not read files for tasks that don't match their purpose
|
||||
- ✅ **ALWAYS read the file FIRST** before starting the task (when applicable)
|
||||
|
|
|
|||
10
Makefile
10
Makefile
|
|
@ -83,19 +83,11 @@ test:
|
|||
make -C packages/core/testing apply
|
||||
make -C packages/core/testing test
|
||||
|
||||
unit-tests: helm-unit-tests bats-unit-tests go-unit-tests
|
||||
unit-tests: helm-unit-tests bats-unit-tests
|
||||
|
||||
helm-unit-tests:
|
||||
hack/helm-unit-tests.sh
|
||||
|
||||
# Scoped go test over the cozystack-api surface that this repo owns. Kept
|
||||
# narrow intentionally - running `go test ./...` pulls in generated code
|
||||
# round-trip suites whose behavior depends on tool versions outside this
|
||||
# repo's control (kubebuilder, openapi-gen, etc.) and is better exercised
|
||||
# from their generator workflows.
|
||||
go-unit-tests:
|
||||
go test ./pkg/registry/... ./pkg/config/... ./pkg/cmd/server/...
|
||||
|
||||
# Discover every hack/*.bats file that is NOT an e2e test and run it
|
||||
# through cozytest.sh. Drop a new *.bats file in hack/ and it is picked
|
||||
# up automatically on the next `make unit-tests` run.
|
||||
|
|
|
|||
|
|
@ -36,9 +36,6 @@ type ConfigSpec struct {
|
|||
// Kubernetes control-plane configuration.
|
||||
// +kubebuilder:default:={}
|
||||
ControlPlane ControlPlane `json:"controlPlane"`
|
||||
// Optional image overrides for air-gapped or rate-limited registries.
|
||||
// +kubebuilder:default:={}
|
||||
Images Images `json:"images"`
|
||||
}
|
||||
|
||||
type APIServer struct {
|
||||
|
|
@ -69,9 +66,6 @@ type Addons struct {
|
|||
// NVIDIA GPU Operator.
|
||||
// +kubebuilder:default:={}
|
||||
GpuOperator GPUOperatorAddon `json:"gpuOperator"`
|
||||
// HAMi GPU virtualization middleware.
|
||||
// +kubebuilder:default:={}
|
||||
Hami HAMiAddon `json:"hami"`
|
||||
// Ingress-NGINX controller.
|
||||
// +kubebuilder:default:={}
|
||||
IngressNginx IngressNginxAddon `json:"ingressNginx"`
|
||||
|
|
@ -163,21 +157,6 @@ type GatewayAPIAddon struct {
|
|||
Enabled bool `json:"enabled"`
|
||||
}
|
||||
|
||||
type HAMiAddon struct {
|
||||
// Enable HAMi (requires GPU Operator).
|
||||
// +kubebuilder:default:=false
|
||||
Enabled bool `json:"enabled"`
|
||||
// Custom Helm values overrides.
|
||||
// +kubebuilder:default:={}
|
||||
ValuesOverride k8sRuntime.RawExtension `json:"valuesOverride"`
|
||||
}
|
||||
|
||||
type Images struct {
|
||||
// Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag.
|
||||
// +kubebuilder:default:=""
|
||||
WaitForKubeconfig string `json:"waitForKubeconfig,omitempty"`
|
||||
}
|
||||
|
||||
type IngressNginxAddon struct {
|
||||
// Enable the controller (requires nodes labeled `ingress-nginx`).
|
||||
// +kubebuilder:default:=false
|
||||
|
|
|
|||
|
|
@ -49,7 +49,6 @@ func (in *Addons) DeepCopyInto(out *Addons) {
|
|||
in.Fluxcd.DeepCopyInto(&out.Fluxcd)
|
||||
out.GatewayAPI = in.GatewayAPI
|
||||
in.GpuOperator.DeepCopyInto(&out.GpuOperator)
|
||||
in.Hami.DeepCopyInto(&out.Hami)
|
||||
in.IngressNginx.DeepCopyInto(&out.IngressNginx)
|
||||
in.MonitoringAgents.DeepCopyInto(&out.MonitoringAgents)
|
||||
in.Velero.DeepCopyInto(&out.Velero)
|
||||
|
|
@ -136,7 +135,6 @@ func (in *ConfigSpec) DeepCopyInto(out *ConfigSpec) {
|
|||
}
|
||||
in.Addons.DeepCopyInto(&out.Addons)
|
||||
in.ControlPlane.DeepCopyInto(&out.ControlPlane)
|
||||
out.Images = in.Images
|
||||
}
|
||||
|
||||
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigSpec.
|
||||
|
|
@ -262,37 +260,6 @@ func (in *GatewayAPIAddon) DeepCopy() *GatewayAPIAddon {
|
|||
return out
|
||||
}
|
||||
|
||||
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
|
||||
func (in *HAMiAddon) DeepCopyInto(out *HAMiAddon) {
|
||||
*out = *in
|
||||
in.ValuesOverride.DeepCopyInto(&out.ValuesOverride)
|
||||
}
|
||||
|
||||
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HAMiAddon.
|
||||
func (in *HAMiAddon) DeepCopy() *HAMiAddon {
|
||||
if in == nil {
|
||||
return nil
|
||||
}
|
||||
out := new(HAMiAddon)
|
||||
in.DeepCopyInto(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
|
||||
func (in *Images) DeepCopyInto(out *Images) {
|
||||
*out = *in
|
||||
}
|
||||
|
||||
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Images.
|
||||
func (in *Images) DeepCopy() *Images {
|
||||
if in == nil {
|
||||
return nil
|
||||
}
|
||||
out := new(Images)
|
||||
in.DeepCopyInto(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
|
||||
func (in *IngressNginxAddon) DeepCopyInto(out *IngressNginxAddon) {
|
||||
*out = *in
|
||||
|
|
|
|||
|
|
@ -92,9 +92,6 @@ type Bootstrap struct {
|
|||
// Timestamp (RFC3339) for point-in-time recovery; empty means latest.
|
||||
// +kubebuilder:default:=""
|
||||
RecoveryTime string `json:"recoveryTime,omitempty"`
|
||||
// Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name.
|
||||
// +kubebuilder:default:=""
|
||||
ServerName string `json:"serverName,omitempty"`
|
||||
}
|
||||
|
||||
type Database struct {
|
||||
|
|
|
|||
|
|
@ -26,9 +26,6 @@ type ConfigSpec struct {
|
|||
// Ports to forward from outside the cluster.
|
||||
// +kubebuilder:default:={22}
|
||||
ExternalPorts []int `json:"externalPorts,omitempty"`
|
||||
// Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect.
|
||||
// +kubebuilder:default:=true
|
||||
ExternalAllowICMP bool `json:"externalAllowICMP"`
|
||||
// Requested running state of the VirtualMachineInstance
|
||||
// +kubebuilder:default:="Always"
|
||||
RunStrategy RunStrategy `json:"runStrategy"`
|
||||
|
|
|
|||
|
|
@ -620,11 +620,6 @@ func (in *RestoreJobSpec) DeepCopyInto(out *RestoreJobSpec) {
|
|||
*out = new(v1.TypedLocalObjectReference)
|
||||
(*in).DeepCopyInto(*out)
|
||||
}
|
||||
if in.Options != nil {
|
||||
in, out := &in.Options, &out.Options
|
||||
*out = new(runtime.RawExtension)
|
||||
(*in).DeepCopyInto(*out)
|
||||
}
|
||||
}
|
||||
|
||||
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreJobSpec.
|
||||
|
|
|
|||
|
|
@ -132,16 +132,6 @@ type ComponentInstall struct {
|
|||
// DependsOn is a list of component names that must be installed before this component
|
||||
// +optional
|
||||
DependsOn []string `json:"dependsOn,omitempty"`
|
||||
|
||||
// UpgradeCRDs controls how CRDs from the chart's crds/ directory are
|
||||
// handled on HelmRelease upgrades. Maps to HelmRelease.Spec.Upgrade.CRDs.
|
||||
// Empty string (default) preserves the helm-controller default (Skip).
|
||||
// Use "CreateReplace" for operators that evolve their CRD set between
|
||||
// versions. Warning: CreateReplace overwrites CRDs and may cause data
|
||||
// loss if upstream drops fields from a CRD with live objects.
|
||||
// +optional
|
||||
// +kubebuilder:validation:Enum=Skip;Create;CreateReplace
|
||||
UpgradeCRDs string `json:"upgradeCRDs,omitempty"`
|
||||
}
|
||||
|
||||
// Component defines a single Helm release component within a package source
|
||||
|
|
|
|||
|
|
@ -1,819 +0,0 @@
|
|||
{
|
||||
"uid": "gpu-efficiency",
|
||||
"title": "GPU Efficiency Score",
|
||||
"description": "Tensor saturation, util/watt and throttling — reveals inefficient GPU workloads",
|
||||
"tags": [
|
||||
"gpu",
|
||||
"efficiency",
|
||||
"finops"
|
||||
],
|
||||
"timezone": "browser",
|
||||
"editable": true,
|
||||
"graphTooltip": 1,
|
||||
"time": {
|
||||
"from": "now-1h",
|
||||
"to": "now"
|
||||
},
|
||||
"fiscalYearStartMonth": 0,
|
||||
"schemaVersion": 42,
|
||||
"panels": [
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Overall efficiency metrics",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
"id": 1,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 2,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "avg(gpu:tensor_saturation:avg5m) * 100",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Avg Tensor Saturation",
|
||||
"description": "Mean tensor core saturation across all GPUs. \u003c10% means GPUs are used inefficiently (workloads could move to CPU or optimize their code). Cluster-wide — DCGM metrics cannot be attributed to workload namespaces.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 5,
|
||||
"w": 8,
|
||||
"x": 0,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "area",
|
||||
"colorMode": "background",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "red"
|
||||
},
|
||||
{
|
||||
"value": 10,
|
||||
"color": "orange"
|
||||
},
|
||||
{
|
||||
"value": 30,
|
||||
"color": "yellow"
|
||||
},
|
||||
{
|
||||
"value": 60,
|
||||
"color": "green"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 3,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "avg(gpu:util_per_watt:avg5m)",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Avg Utilization per Watt",
|
||||
"description": "NVML utilization % per watt across all GPUs. Higher value = more efficient workload. Cluster-wide — DCGM metrics cannot be attributed to workload namespaces.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 5,
|
||||
"w": 8,
|
||||
"x": 8,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "area",
|
||||
"colorMode": "background",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "none",
|
||||
"decimals": 3,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "red"
|
||||
},
|
||||
{
|
||||
"value": 0.5,
|
||||
"color": "yellow"
|
||||
},
|
||||
{
|
||||
"value": 1,
|
||||
"color": "green"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 4,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "avg(gpu:power_throttle_fraction:rate5m)",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Avg Power Throttling",
|
||||
"description": "Fraction of time GPUs hit the TDP cap and lose performance. \u003e5% means tenants underutilize billed FLOPS. Cluster-wide — rule aggregates by (Hostname, gpu, UUID) and drops namespace label.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 5,
|
||||
"w": 8,
|
||||
"x": 16,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "area",
|
||||
"colorMode": "background",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percentunit",
|
||||
"decimals": 2,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 0.05,
|
||||
"color": "yellow"
|
||||
},
|
||||
{
|
||||
"value": 0.2,
|
||||
"color": "red"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "NVML vs Tensor (mismatch detector)",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 6
|
||||
},
|
||||
"id": 10,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 11,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_DEV_GPU_UTIL",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "NVML GPU Utilization",
|
||||
"description": "Classic utilization metric. Shows activity of any engine (SM, copy, encoder). Cluster-wide — DCGM namespace is the exporter's own namespace, not the workload namespace.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 7
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 12,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_PROF_PIPE_TENSOR_ACTIVE * 100",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Tensor Pipe Active",
|
||||
"description": "Real tensor core load (HMMA). For AI/LLM inference it should be ≥20%, otherwise the workload is unoptimized. Cluster-wide — DCGM namespace is the exporter's own namespace, not the workload namespace.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 7
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Per-GPU ranking",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 15
|
||||
},
|
||||
"id": 20,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "table",
|
||||
"id": 21,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "topk(20, gpu:tensor_saturation:avg5m * 100)",
|
||||
"instant": true,
|
||||
"range": false,
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Tensor Saturation per GPU (5m avg)",
|
||||
"description": "Which GPUs are exercising tensor cores and which are not. Sorted descending. Grouped by Hostname/gpu/UUID — DCGM metrics cannot be attributed to workload namespaces.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 16
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"transformations": [
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {
|
||||
"DCGM_FI_DRIVER_VERSION": true,
|
||||
"Time": true,
|
||||
"__name__": true,
|
||||
"cluster": true,
|
||||
"container": true,
|
||||
"device": true,
|
||||
"endpoint": true,
|
||||
"gpu_driver_version": true,
|
||||
"instance": true,
|
||||
"job": true,
|
||||
"modelName": true,
|
||||
"namespace": true,
|
||||
"pci_bus_id": true,
|
||||
"pod": true,
|
||||
"prometheus": true,
|
||||
"service": true,
|
||||
"tenant": true,
|
||||
"tier": true,
|
||||
"uid": true,
|
||||
"unit": true
|
||||
},
|
||||
"indexByName": {
|
||||
"Hostname": 0,
|
||||
"UUID": 2,
|
||||
"Value": 3,
|
||||
"gpu": 1
|
||||
},
|
||||
"renameByName": {
|
||||
"Hostname": "Node",
|
||||
"UUID": "UUID",
|
||||
"Value": "Saturation",
|
||||
"gpu": "GPU"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"options": {
|
||||
"frameIndex": 0,
|
||||
"showHeader": true,
|
||||
"showTypeIcons": false,
|
||||
"sortBy": [
|
||||
{
|
||||
"displayName": "Saturation",
|
||||
"desc": true
|
||||
}
|
||||
],
|
||||
"footer": {
|
||||
"show": false,
|
||||
"reducer": []
|
||||
},
|
||||
"cellHeight": "sm"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {
|
||||
"id": "byName",
|
||||
"options": "Saturation"
|
||||
},
|
||||
"properties": [
|
||||
{
|
||||
"id": "unit",
|
||||
"value": "percent"
|
||||
},
|
||||
{
|
||||
"id": "min",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "max",
|
||||
"value": 100
|
||||
},
|
||||
{
|
||||
"id": "custom.cellOptions",
|
||||
"value": {
|
||||
"mode": "gradient",
|
||||
"type": "gauge"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "thresholds",
|
||||
"value": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "red"
|
||||
},
|
||||
{
|
||||
"color": "orange",
|
||||
"value": 10
|
||||
},
|
||||
{
|
||||
"color": "yellow",
|
||||
"value": 30
|
||||
},
|
||||
{
|
||||
"color": "green",
|
||||
"value": 60
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "table",
|
||||
"id": 22,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "topk(20, gpu:util_per_watt:avg5m)",
|
||||
"instant": true,
|
||||
"range": false,
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Utilization / Watt per GPU (5m avg)",
|
||||
"description": "How efficiently each GPU spends watts. Low value = poor optimization. Grouped by Hostname/gpu/UUID — DCGM metrics cannot be attributed to workload namespaces.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 16
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"transformations": [
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {
|
||||
"DCGM_FI_DRIVER_VERSION": true,
|
||||
"Time": true,
|
||||
"__name__": true,
|
||||
"cluster": true,
|
||||
"container": true,
|
||||
"device": true,
|
||||
"endpoint": true,
|
||||
"gpu_driver_version": true,
|
||||
"instance": true,
|
||||
"job": true,
|
||||
"modelName": true,
|
||||
"namespace": true,
|
||||
"pci_bus_id": true,
|
||||
"pod": true,
|
||||
"prometheus": true,
|
||||
"service": true,
|
||||
"tenant": true,
|
||||
"tier": true,
|
||||
"uid": true,
|
||||
"unit": true
|
||||
},
|
||||
"indexByName": {
|
||||
"Hostname": 0,
|
||||
"UUID": 2,
|
||||
"Value": 3,
|
||||
"gpu": 1
|
||||
},
|
||||
"renameByName": {
|
||||
"Hostname": "Node",
|
||||
"UUID": "UUID",
|
||||
"Value": "Util/Watt",
|
||||
"gpu": "GPU"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"options": {
|
||||
"frameIndex": 0,
|
||||
"showHeader": true,
|
||||
"showTypeIcons": false,
|
||||
"sortBy": [
|
||||
{
|
||||
"displayName": "Util/Watt",
|
||||
"desc": true
|
||||
}
|
||||
],
|
||||
"footer": {
|
||||
"show": false,
|
||||
"reducer": []
|
||||
},
|
||||
"cellHeight": "sm"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {
|
||||
"id": "byName",
|
||||
"options": "Util/Watt"
|
||||
},
|
||||
"properties": [
|
||||
{
|
||||
"id": "unit",
|
||||
"value": "none"
|
||||
},
|
||||
{
|
||||
"id": "decimals",
|
||||
"value": 3
|
||||
},
|
||||
{
|
||||
"id": "min",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "max",
|
||||
"value": 1.5
|
||||
},
|
||||
{
|
||||
"id": "custom.cellOptions",
|
||||
"value": {
|
||||
"mode": "gradient",
|
||||
"type": "gauge"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "thresholds",
|
||||
"value": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "red"
|
||||
},
|
||||
{
|
||||
"color": "yellow",
|
||||
"value": 0.5
|
||||
},
|
||||
{
|
||||
"color": "green",
|
||||
"value": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Throttling",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 24
|
||||
},
|
||||
"id": 30,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 31,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "gpu:power_throttle_fraction:rate5m",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Power throttle fraction per GPU",
|
||||
"description": "Fraction of time the GPU was power-throttled. 1.0 = always throttled. Cluster-wide — rule aggregates by (Hostname, gpu, UUID) and drops namespace label.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 25
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percentunit",
|
||||
"min": 0,
|
||||
"max": 1,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 0.05,
|
||||
"color": "yellow"
|
||||
},
|
||||
{
|
||||
"value": 0.2,
|
||||
"color": "red"
|
||||
}
|
||||
]
|
||||
},
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 25,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 32,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "gpu:thermal_throttle_fraction:rate5m",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Thermal throttle fraction per GPU",
|
||||
"description": "Fraction of time the GPU was thermal-throttled. Cluster-wide — rule aggregates by (Hostname, gpu, UUID) and drops namespace label.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 25
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percentunit",
|
||||
"min": 0,
|
||||
"max": 1,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 0.05,
|
||||
"color": "yellow"
|
||||
},
|
||||
{
|
||||
"value": 0.2,
|
||||
"color": "red"
|
||||
}
|
||||
]
|
||||
},
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 25,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
}
|
||||
],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"type": "datasource",
|
||||
"name": "ds_prometheus",
|
||||
"label": "Prometheus",
|
||||
"skipUrlSync": false,
|
||||
"query": "prometheus",
|
||||
"current": {
|
||||
"selected": false,
|
||||
"text": "default",
|
||||
"value": "default"
|
||||
},
|
||||
"multi": false,
|
||||
"allowCustomValue": true,
|
||||
"includeAll": false,
|
||||
"regex": "",
|
||||
"auto": false,
|
||||
"auto_min": "10s",
|
||||
"auto_count": 30
|
||||
}
|
||||
]
|
||||
},
|
||||
"annotations": {}
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,957 +0,0 @@
|
|||
{
|
||||
"uid": "gpu-performance",
|
||||
"title": "GPU Performance",
|
||||
"tags": [
|
||||
"gpu",
|
||||
"dcgm"
|
||||
],
|
||||
"timezone": "browser",
|
||||
"editable": true,
|
||||
"graphTooltip": 1,
|
||||
"time": {
|
||||
"from": "now-1h",
|
||||
"to": "now"
|
||||
},
|
||||
"fiscalYearStartMonth": 0,
|
||||
"schemaVersion": 42,
|
||||
"panels": [
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Overview",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
"id": 1,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 2,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "cluster:gpu_count:total",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Total GPUs",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 0,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "none",
|
||||
"colorMode": "value",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "blue"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 3,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "cluster:gpu_count:allocated",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Allocated",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 6,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "none",
|
||||
"colorMode": "value",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 1,
|
||||
"color": "yellow"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 4,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "cluster:gpu_util:avg",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Average utilization",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 12,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "area",
|
||||
"colorMode": "value",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "blue"
|
||||
},
|
||||
{
|
||||
"value": 30,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 80,
|
||||
"color": "orange"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 5,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "cluster:gpu_power_watts:sum",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Power draw",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 18,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "area",
|
||||
"colorMode": "value",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "watt",
|
||||
"decimals": 0,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Utilization",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 5
|
||||
},
|
||||
"id": 10,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 11,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_DEV_GPU_UTIL{Hostname=~\"$Hostname\"}",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "GPU Utilization (NVML)",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 6
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never",
|
||||
"spanNulls": false
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 12,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{Hostname=~\"$Hostname\"} * 100",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Tensor Pipe Active (realistic load for LLM/AI)",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 6
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never",
|
||||
"spanNulls": false
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 13,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_PROF_GR_ENGINE_ACTIVE{Hostname=~\"$Hostname\"} * 100",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Graphics Engine Active",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 14
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never",
|
||||
"spanNulls": false
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 14,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_DEV_MEM_COPY_UTIL{Hostname=~\"$Hostname\"}",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Memory Copy Utilization",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 14
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never",
|
||||
"spanNulls": false
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Memory",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 22
|
||||
},
|
||||
"id": 20,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 21,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_DEV_FB_USED{Hostname=~\"$Hostname\"} * 1048576",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}} {{pod}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "VRAM Used",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 23
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "bytes",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 25,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 22,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_DEV_FB_FREE{Hostname=~\"$Hostname\"} * 1048576",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "VRAM Free",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 23
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"min"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "bytes",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Power \u0026 Temperature",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 31
|
||||
},
|
||||
"id": 30,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 31,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_DEV_POWER_USAGE{Hostname=~\"$Hostname\"}",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Power Usage per GPU",
|
||||
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 32
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "watt",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 32,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "DCGM_FI_DEV_GPU_TEMP{Hostname=~\"$Hostname\"}",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "GPU Temperature",
|
||||
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 32
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": [
|
||||
"mean",
|
||||
"max"
|
||||
]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "celsius",
|
||||
"min": 20,
|
||||
"max": 100,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 75,
|
||||
"color": "yellow"
|
||||
},
|
||||
{
|
||||
"value": 85,
|
||||
"color": "red"
|
||||
}
|
||||
]
|
||||
},
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Health",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 40
|
||||
},
|
||||
"id": 40,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 41,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "max by (Hostname, gpu) (DCGM_FI_DEV_XID_ERRORS{Hostname=~\"$Hostname\"})",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "XID errors (latest)",
|
||||
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 5,
|
||||
"w": 8,
|
||||
"x": 0,
|
||||
"y": 41
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "none",
|
||||
"colorMode": "background",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value_and_name",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 1,
|
||||
"color": "red"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 42,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "rate(DCGM_FI_DEV_POWER_VIOLATION{Hostname=~\"$Hostname\"}[5m])",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Power Violation (µs/s throttled due to power)",
|
||||
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 5,
|
||||
"w": 8,
|
||||
"x": 8,
|
||||
"y": 41
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": []
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "µs",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 43,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "rate(DCGM_FI_DEV_THERMAL_VIOLATION{Hostname=~\"$Hostname\"}[5m])",
|
||||
"legendFormat": "{{Hostname}}/{{gpu}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Thermal Violation (µs/s throttled due to thermals)",
|
||||
"description": "Hardware-level metric — shown cluster-wide regardless of namespace filter.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 5,
|
||||
"w": 8,
|
||||
"x": 16,
|
||||
"y": 41
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": []
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "µs",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "linear",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
}
|
||||
],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"type": "datasource",
|
||||
"name": "ds_prometheus",
|
||||
"label": "Prometheus",
|
||||
"skipUrlSync": false,
|
||||
"query": "prometheus",
|
||||
"current": {
|
||||
"selected": false,
|
||||
"text": "default",
|
||||
"value": "default"
|
||||
},
|
||||
"multi": false,
|
||||
"allowCustomValue": true,
|
||||
"includeAll": false,
|
||||
"regex": "",
|
||||
"auto": false,
|
||||
"auto_min": "10s",
|
||||
"auto_count": 30
|
||||
},
|
||||
{
|
||||
"type": "query",
|
||||
"name": "Hostname",
|
||||
"label": "Host",
|
||||
"skipUrlSync": false,
|
||||
"query": "label_values(DCGM_FI_DEV_GPU_UTIL, Hostname)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"current": {
|
||||
"selected": true,
|
||||
"text": "All",
|
||||
"value": "$__all"
|
||||
},
|
||||
"multi": true,
|
||||
"allowCustomValue": true,
|
||||
"refresh": 2,
|
||||
"sort": 1,
|
||||
"includeAll": true,
|
||||
"auto": false,
|
||||
"auto_min": "10s",
|
||||
"auto_count": 30
|
||||
}
|
||||
]
|
||||
},
|
||||
"annotations": {}
|
||||
}
|
||||
|
|
@ -1,637 +0,0 @@
|
|||
{
|
||||
"uid": "gpu-quotas",
|
||||
"title": "GPU Quotas \u0026 Allocation",
|
||||
"tags": [
|
||||
"gpu",
|
||||
"quotas"
|
||||
],
|
||||
"timezone": "browser",
|
||||
"editable": true,
|
||||
"graphTooltip": 1,
|
||||
"time": {
|
||||
"from": "now-6h",
|
||||
"to": "now"
|
||||
},
|
||||
"fiscalYearStartMonth": 0,
|
||||
"schemaVersion": 42,
|
||||
"panels": [
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Allocation overview",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
"id": 1,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 2,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(kube_node_status_allocatable{resource=\"nvidia_com_gpu\"})",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "GPU allocatable",
|
||||
"description": "Total GPU capacity the cluster can schedule to pods.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 0,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "none",
|
||||
"colorMode": "value",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "blue"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 3,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "cluster:gpu_count:allocated",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "GPU requested",
|
||||
"description": "Sum of GPU requests across all pods cluster-wide, including system namespaces.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 6,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "none",
|
||||
"colorMode": "value",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 1,
|
||||
"color": "yellow"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "gauge",
|
||||
"id": 4,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "cluster:gpu_count:allocated / sum(kube_node_status_allocatable{resource=\"nvidia_com_gpu\"}) * 100",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Allocation ratio",
|
||||
"description": "Percentage of allocatable GPUs currently requested by pods.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 12,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"showThresholdLabels": false,
|
||||
"showThresholdMarkers": true,
|
||||
"sizing": "auto",
|
||||
"minVizWidth": 75,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"minVizHeight": 75,
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percent",
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "blue"
|
||||
},
|
||||
{
|
||||
"value": 40,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 80,
|
||||
"color": "yellow"
|
||||
},
|
||||
{
|
||||
"value": 95,
|
||||
"color": "red"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"id": 5,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "count((sum by (namespace, pod) (kube_pod_container_resource_requests{resource=\"nvidia_com_gpu\"}) \u003e 0) * on(namespace, pod) group_left() (kube_pod_status_phase{phase=\"Pending\"} == 1)) or vector(0)",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Pending pods (GPU)",
|
||||
"description": "Pods requesting GPUs that are stuck in Pending state — indicates capacity shortage.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
"w": 6,
|
||||
"x": 18,
|
||||
"y": 1
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"graphMode": "none",
|
||||
"colorMode": "background",
|
||||
"justifyMode": "auto",
|
||||
"textMode": "value",
|
||||
"wideLayout": true,
|
||||
"showPercentChange": false,
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"percentChangeColorMode": "standard",
|
||||
"orientation": ""
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
},
|
||||
{
|
||||
"value": 1,
|
||||
"color": "red"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Per namespace",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 5
|
||||
},
|
||||
"id": 10,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "bargauge",
|
||||
"id": 11,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum by (namespace) (namespace:gpu_count:allocated{namespace=~\"$namespace\"})",
|
||||
"legendFormat": "{{namespace}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "GPU requested per namespace",
|
||||
"description": "GPU allocation breakdown by namespace — spot top consumers at a glance.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 6
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"displayMode": "gradient",
|
||||
"valueMode": "color",
|
||||
"namePlacement": "auto",
|
||||
"showUnfilled": true,
|
||||
"sizing": "auto",
|
||||
"minVizWidth": 8,
|
||||
"minVizHeight": 16,
|
||||
"legend": {
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": []
|
||||
},
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
]
|
||||
},
|
||||
"maxVizHeight": 300,
|
||||
"orientation": "horizontal"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"min": 0,
|
||||
"max": 8,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"value": null,
|
||||
"color": "green"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"id": 12,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(kube_node_status_allocatable{resource=\"nvidia_com_gpu\"})",
|
||||
"legendFormat": "Allocatable (total)",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "sum(namespace:gpu_count:allocated{namespace=~\"$namespace\"})",
|
||||
"legendFormat": "Requested",
|
||||
"refId": "B"
|
||||
}
|
||||
],
|
||||
"title": "GPU allocated over time",
|
||||
"description": "Requested vs allocatable GPUs over time — shows allocation pressure trends.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 6
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": false,
|
||||
"calcs": []
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": ""
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineInterpolation": "stepAfter",
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {
|
||||
"id": "byName",
|
||||
"options": "Allocatable (total)"
|
||||
},
|
||||
"properties": [
|
||||
{
|
||||
"id": "color",
|
||||
"value": {
|
||||
"fixedColor": "blue",
|
||||
"mode": "fixed"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"collapsed": false,
|
||||
"title": "Pods with GPU",
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 14
|
||||
},
|
||||
"id": 20,
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"type": "table",
|
||||
"id": 21,
|
||||
"targets": [
|
||||
{
|
||||
"expr": "kube_pod_container_resource_requests{resource=\"nvidia_com_gpu\", namespace=~\"$namespace\"} * on(namespace, pod) group_left(phase) (kube_pod_status_phase{phase=~\"Running|Pending\"} == 1)",
|
||||
"instant": true,
|
||||
"range": false,
|
||||
"format": "table",
|
||||
"refId": "requested"
|
||||
},
|
||||
{
|
||||
"expr": "kube_pod_container_resource_limits{resource=\"nvidia_com_gpu\", namespace=~\"$namespace\"} * on(namespace, pod) group_left() (kube_pod_status_phase{phase=~\"Running|Pending\"} == 1)",
|
||||
"instant": true,
|
||||
"range": false,
|
||||
"format": "table",
|
||||
"refId": "limits"
|
||||
}
|
||||
],
|
||||
"title": "Pods requesting GPU",
|
||||
"description": "Per-pod GPU requests and limits with scheduling status — Running or Pending.",
|
||||
"transparent": false,
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 10,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 15
|
||||
},
|
||||
"repeatDirection": "h",
|
||||
"transformations": [
|
||||
{
|
||||
"id": "joinByField",
|
||||
"options": {
|
||||
"byField": "pod",
|
||||
"mode": "outer"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {
|
||||
"Time": true,
|
||||
"Time 1": true,
|
||||
"Time 2": true,
|
||||
"__name__": true,
|
||||
"__name__ 1": true,
|
||||
"__name__ 2": true,
|
||||
"cluster": true,
|
||||
"cluster 1": true,
|
||||
"cluster 2": true,
|
||||
"container 2": true,
|
||||
"endpoint": true,
|
||||
"endpoint 1": true,
|
||||
"endpoint 2": true,
|
||||
"instance": true,
|
||||
"instance 1": true,
|
||||
"instance 2": true,
|
||||
"job": true,
|
||||
"job 1": true,
|
||||
"job 2": true,
|
||||
"namespace 2": true,
|
||||
"node 2": true,
|
||||
"prometheus": true,
|
||||
"prometheus 1": true,
|
||||
"prometheus 2": true,
|
||||
"resource": true,
|
||||
"resource 1": true,
|
||||
"resource 2": true,
|
||||
"service": true,
|
||||
"service 1": true,
|
||||
"service 2": true,
|
||||
"tenant": true,
|
||||
"tenant 1": true,
|
||||
"tenant 2": true,
|
||||
"tier": true,
|
||||
"tier 1": true,
|
||||
"tier 2": true,
|
||||
"uid": true,
|
||||
"uid 1": true,
|
||||
"uid 2": true,
|
||||
"unit": true,
|
||||
"unit 1": true,
|
||||
"unit 2": true
|
||||
},
|
||||
"indexByName": {
|
||||
"Value #limits": 5,
|
||||
"Value #requested": 4,
|
||||
"container 1": 2,
|
||||
"namespace 1": 0,
|
||||
"node 1": 3,
|
||||
"phase": 6,
|
||||
"pod": 1
|
||||
},
|
||||
"renameByName": {
|
||||
"Value #limits": "Limit",
|
||||
"Value #requested": "Req",
|
||||
"container 1": "Container",
|
||||
"namespace 1": "Namespace",
|
||||
"node 1": "Node",
|
||||
"phase": "Status"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"options": {
|
||||
"frameIndex": 0,
|
||||
"showHeader": true,
|
||||
"showTypeIcons": false,
|
||||
"footer": {
|
||||
"show": false,
|
||||
"reducer": []
|
||||
},
|
||||
"cellHeight": "sm"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {
|
||||
"id": "byName",
|
||||
"options": "Status"
|
||||
},
|
||||
"properties": [
|
||||
{
|
||||
"id": "custom.cellOptions",
|
||||
"value": {
|
||||
"mode": "basic",
|
||||
"type": "color-background"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "mappings",
|
||||
"value": [
|
||||
{
|
||||
"options": {
|
||||
"Failed": {
|
||||
"color": "red",
|
||||
"index": 2
|
||||
},
|
||||
"Pending": {
|
||||
"color": "orange",
|
||||
"index": 1
|
||||
},
|
||||
"Running": {
|
||||
"color": "green",
|
||||
"index": 0
|
||||
}
|
||||
},
|
||||
"type": "value"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"type": "datasource",
|
||||
"name": "ds_prometheus",
|
||||
"label": "Prometheus",
|
||||
"skipUrlSync": false,
|
||||
"query": "prometheus",
|
||||
"current": {
|
||||
"selected": false,
|
||||
"text": "default",
|
||||
"value": "default"
|
||||
},
|
||||
"multi": false,
|
||||
"allowCustomValue": true,
|
||||
"includeAll": false,
|
||||
"regex": "",
|
||||
"auto": false,
|
||||
"auto_min": "10s",
|
||||
"auto_count": 30
|
||||
},
|
||||
{
|
||||
"type": "query",
|
||||
"name": "namespace",
|
||||
"label": "Namespace",
|
||||
"skipUrlSync": false,
|
||||
"query": "label_values(kube_pod_container_resource_requests{resource=\"nvidia_com_gpu\"}, namespace)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "$ds_prometheus"
|
||||
},
|
||||
"current": {
|
||||
"selected": true,
|
||||
"text": "All",
|
||||
"value": "$__all"
|
||||
},
|
||||
"multi": true,
|
||||
"allowCustomValue": true,
|
||||
"refresh": 2,
|
||||
"sort": 1,
|
||||
"includeAll": true,
|
||||
"auto": false,
|
||||
"auto_min": "10s",
|
||||
"auto_count": 30
|
||||
}
|
||||
]
|
||||
},
|
||||
"annotations": {}
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -6,20 +6,6 @@ This file contains detailed instructions for AI-powered IDE on how to generate c
|
|||
|
||||
Follow these instructions when the user explicitly asks to generate a changelog.
|
||||
|
||||
## Scope and boundaries
|
||||
|
||||
**Your single deliverable is the file `docs/changelogs/v<version>.md`.** Write the complete, verified changelog to that path. That is the entire task. Exit as soon as the file is written and verified against the checklist in Step 9.
|
||||
|
||||
Unless the caller explicitly instructs otherwise:
|
||||
|
||||
- **In the cozystack working tree**, do not run `git commit`, `git push`, `git checkout` (to switch branches), `git branch`, `git tag`, `git reset`, `git merge`, or `git rebase`. Do not write to local branches, tags, or HEAD. `git fetch` is expected and fine (see the read-only analysis list below).
|
||||
- **Do not** push to any remote, open pull requests, or issue GitHub API write calls (POST / PATCH / DELETE) for any repository.
|
||||
- In the cozystack working tree, the **only** file you create or modify is `docs/changelogs/v<version>.md`. Cloning auxiliary repositories under `_repos/` for cross-repo analysis (see Step 6) is fine; local git operations inside those disposable clones (`git checkout`, `git pull`, etc.) are allowed — just never push from them or open PRs against them.
|
||||
|
||||
The caller — a GitHub Actions workflow in CI, or a developer running you interactively — owns branching, committing, pushing, and PR creation. They will perform those actions after you exit. Do not pre-empt them even if the working tree looks ready.
|
||||
|
||||
Read-only analysis is expected and encouraged: `git log`, `git show`, `git fetch`, `git diff`, `gh pr view`, `gh api` GET requests, and reading any file in the repository.
|
||||
|
||||
## Required Tools
|
||||
|
||||
Before generating changelogs, ensure you have access to `gh` (GitHub CLI) tool, which is used to fetch commit and PR author information. The GitHub CLI is used to correctly identify PR authors from commits and pull requests.
|
||||
|
|
@ -622,8 +608,6 @@ Create a new changelog file in the format matching previous versions:
|
|||
**Save the changelog:**
|
||||
Save the changelog to file `docs/changelogs/v<version>.md` according to the version for which the changelog is being generated.
|
||||
|
||||
**Then exit.** Do not commit, push, create a branch, or open a pull request — the caller handles all git and GitHub operations after you return. See the "Scope and boundaries" section at the top of this document.
|
||||
|
||||
### Important notes
|
||||
|
||||
- **After fetch with --force** local tags are up-to-date, use them for work
|
||||
|
|
|
|||
|
|
@ -8,30 +8,6 @@ Project-side conventions for commits, branches, and pull requests in Cozystack.
|
|||
- [ ] Commit is signed off with `--signoff`
|
||||
- [ ] Branch is rebased on `upstream/main` (no extra commits)
|
||||
- [ ] PR body includes description and release note
|
||||
- [ ] Ran `make generate` in every package whose `values.yaml`, `values.schema.json`, `Chart.yaml`, or `README.md` was touched, and committed the regenerated files
|
||||
|
||||
## Regenerate Artifacts Before Committing
|
||||
|
||||
Several files in each package are produced by `make generate` from `values.yaml` + `values.schema.json` and must stay in sync with the hand-edited sources:
|
||||
|
||||
- `packages/(apps|extra)/<name>/README.md` — regenerated by `cozyvalues-gen` (parameter table, formatting).
|
||||
- `packages/(apps|extra)/<name>/values.schema.json` — `cozyvalues-gen` rewrites ordering and derived fields.
|
||||
- `packages/system/<name>-rd/cozyrds/<name>.yaml` — produced by `hack/update-crd.sh`, which `make generate` invokes.
|
||||
|
||||
**Before committing edits to any of those sources**, run `make generate` inside the package and stage the full diff:
|
||||
|
||||
```bash
|
||||
make -C packages/<apps-or-extra>/<name> generate
|
||||
git add packages/<apps-or-extra>/<name>/ packages/system/<name>-rd/
|
||||
```
|
||||
|
||||
The repo's pre-commit CI job runs `make generate` in every package and then `git diff --exit-code`. Any unstaged generator output fails the job with exit code 123 and blocks the PR. Also rerun `make generate` after a `git commit --amend` if the amended change touched any of the sources above.
|
||||
|
||||
To locate packages a WIP branch likely needs to be regenerated:
|
||||
|
||||
```bash
|
||||
git diff --name-only | xargs -n1 dirname | sort -u | grep ^packages/
|
||||
```
|
||||
|
||||
## Commit Format
|
||||
|
||||
|
|
@ -43,11 +19,10 @@ git commit --signoff -m "type(scope): brief description"
|
|||
|
||||
**Types:** `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`
|
||||
|
||||
**Scopes** (examples — not an exhaustive list; pick the most specific scope that describes the change, and introduce a new one if a genuinely new area needs its own):
|
||||
|
||||
- System, e.g.: `dashboard`, `platform`, `operator`, `cilium`, `kube-ovn`, `linstor`, `fluxcd`, `cluster-api`
|
||||
- Apps, e.g.: `postgres`, `mariadb`, `redis`, `kafka`, `clickhouse`, `virtual-machine`, `kubernetes`
|
||||
- Other, e.g.: `api`, `hack`, `tests`, `ci`, `docs`, `agents`, `maintenance`
|
||||
**Scopes:**
|
||||
- System: `dashboard`, `platform`, `cilium`, `kube-ovn`, `linstor`, `fluxcd`, `cluster-api`
|
||||
- Apps: `postgres`, `mariadb`, `redis`, `kafka`, `clickhouse`, `virtual-machine`, `kubernetes`
|
||||
- Other: `api`, `hack`, `tests`, `ci`, `docs`, `maintenance`
|
||||
|
||||
Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer.
|
||||
|
||||
|
|
@ -58,49 +33,6 @@ git commit --signoff -m "fix(postgres): update operator to version 1.2.3"
|
|||
git commit --signoff -m "docs(contributing): add installation guide"
|
||||
```
|
||||
|
||||
## PR Title Auto-Labeling
|
||||
|
||||
`.github/workflows/pr-labeler.yaml` parses the PR title on `opened`, `edited`, `reopened`, and `synchronize` events and applies labels additively (never removes). The title is expected to follow Conventional Commits — same format as commit messages above.
|
||||
|
||||
**Type → `kind/*`:**
|
||||
|
||||
| type | label |
|
||||
| --------- | ------------------ |
|
||||
| feat | kind/feature |
|
||||
| fix | kind/bug |
|
||||
| docs | kind/documentation |
|
||||
| chore | kind/cleanup |
|
||||
| refactor | kind/cleanup |
|
||||
| style, perf, test, build, ci, revert | (no kind label) |
|
||||
|
||||
**Scope → `area/*`** (full mapping in `.github/workflows/pr-labeler.yaml`):
|
||||
|
||||
| scope (examples) | label |
|
||||
| --- | --- |
|
||||
| agents, ai | area/ai |
|
||||
| api, cozystack-api | area/api |
|
||||
| build | area/build |
|
||||
| ci | area/ci |
|
||||
| dashboard | area/dashboard |
|
||||
| postgres, mariadb, redis, etcd, kafka, clickhouse, postgres-operator, mariadb-operator | area/database |
|
||||
| extra | area/extra |
|
||||
| kubernetes | area/kubernetes |
|
||||
| monitoring, vlogs, vmstack, grafana, workloadmonitor | area/monitoring |
|
||||
| ingress, gateway, vpn, metallb, cilium, kube-ovn, cozy-proxy, ... | area/networking |
|
||||
| platform, bundle, flux, fluxcd, cluster-api, talos, installer, cozyctl, cozystack-engine, cozy-lib | area/platform |
|
||||
| backport, release | area/release |
|
||||
| seaweedfs, bucket, linstor, velero, harbor, backups | area/storage |
|
||||
| tests, e2e | area/testing |
|
||||
| kubevirt, cdi, vmi, vm-import, virtual-machine, hami, gpu-operator | area/virtualization |
|
||||
|
||||
**Special handling:**
|
||||
|
||||
- `[Backport release-1.x]` prefix is stripped before parsing; `area/release` and `backport` labels are added.
|
||||
- Composite scope (`feat(platform, system, apps): ...`) — each comma-separated part is mapped independently.
|
||||
- `!` after type or `BREAKING CHANGE:` footer in the body → `kind/breaking-change`.
|
||||
- Unmapped scope or non-conventional title → `area/uncategorized` (signals the PR needs manual area selection).
|
||||
- Bracket-style fallback (`[scope] description`) maps `scope` → `area/*` but cannot infer `kind/*`.
|
||||
|
||||
### AI Agent Attribution
|
||||
|
||||
When an AI agent authors or materially assists with a commit, add an `Assisted-By:` trailer naming the model:
|
||||
|
|
|
|||
|
|
@ -83,14 +83,6 @@ packages/<category>/<package-name>/
|
|||
- Reference PR numbers when available
|
||||
- Keep commits atomic and focused
|
||||
|
||||
### PackageSource CRD upgrade policy
|
||||
|
||||
Each component in a `PackageSource` may set `install.upgradeCRDs` to control how CRDs from the chart's `crds/` directory are handled on `HelmRelease` upgrades. Allowed values: `Skip` (default — helm-controller does not touch CRDs on upgrade), `Create` (create new CRDs only), `CreateReplace` (create new and overwrite existing).
|
||||
|
||||
Set `upgradeCRDs: CreateReplace` for operators whose upstream regularly adds new CRDs between versions (etcd-operator, cnpg, kubevirt, kamaji). Without it, new CRDs from a chart bump do not land on existing clusters — only fresh installs get them.
|
||||
|
||||
Do **not** set `CreateReplace` blindly: it overwrites every CRD in `crds/` and can cause silent data loss if upstream drops a field from a CRD that has live objects. Only enable it for operators whose schema evolution is additive-only. When in doubt, leave it unset and apply new CRDs manually.
|
||||
|
||||
### Documentation
|
||||
|
||||
Documentation is organized as follows:
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ The dashboard now ships a complete **RestoreJob experience**: list view, details
|
|||
|
||||
* **[keycloak-configure] Add email verification and SMTP configuration**: Adds configurable Keycloak settings for user self-registration, email verification, and SMTP server configuration, enabling automated user onboarding flows ([**@BROngineer**](https://github.com/BROngineer) in #2318).
|
||||
|
||||
* **[postgres] Pin system PostgreSQL to 17.7-standard-trixie**: Pins the PostgreSQL image for system databases (Grafana, Alerta, Harbor, Keycloak, SeaweedFS) to `17.7-standard-trixie` across chart templates and `values.yaml`, and ships migration 37 to patch existing CNPG Cluster `imageName` fields to the same variant (handling unset, any PG 17 tag, and bare-version tags). This prevents CNPG from defaulting to PostgreSQL 18 and locks system databases to the trixie variant consistent with the monitoring stack requirements *(related backports shipped in v1.2.1 via #2309 and v1.2.2 via #2364)* ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2369).
|
||||
* **[postgres] Hardcode PostgreSQL 17 for monitoring databases**: Pins PostgreSQL 17.7 images for system databases (Grafana, Alerta, Harbor, Keycloak, SeaweedFS) and adds migration 37 to backfill `spec.version=v17` for existing PostgreSQL resources, preventing CNPG from defaulting to PostgreSQL 18 *(backported to v1.2.1)* ([**@IvanHunters**](https://github.com/IvanHunters) in #2304).
|
||||
|
||||
* **[platform] Prevent installed packages deletion**: Adds the `helm.sh/resource-policy: keep` annotation to platform packages so disabling a package no longer triggers automatic Helm deletion, restoring the documented behavior where operators must explicitly delete a package *(backported to v1.2.1)* ([**@kvaps**](https://github.com/kvaps) in #2273).
|
||||
|
||||
|
|
@ -76,6 +76,8 @@ The dashboard now ships a complete **RestoreJob experience**: list view, details
|
|||
|
||||
* **[monitoring] Fix infra dashboards missing in default variant**: Includes the `cozy-monitoring` namespace in the dashboard rendering condition, fixing infrastructure Grafana dashboards not rendering in the default platform variant (only the `tenant-root` namespace was previously checked) *(backported to v1.2.2)* ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #2365).
|
||||
|
||||
* **[postgres] Fix system PostgreSQL images to 17.7-standard-trixie**: Normalizes system PostgreSQL image tags to the `17.7-standard-trixie` variant with migration logic for existing CNPG clusters, ensuring system databases use the correct image variant consistent with the monitoring stack requirements introduced in v1.2.1 *(backported to v1.2.2)* ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2364).
|
||||
|
||||
* **[build] Filter git describe to match only v* tags**: Adds `--match 'v*'` to all `git describe` calls in `hack/common-envs.mk`, preventing the `api/apps/v1alpha1/vX.Y.Z` subtag from being picked up instead of the release tag and producing invalid Docker image tags *(backported to v1.2.2)* ([**@kvaps**](https://github.com/kvaps) in #2386).
|
||||
|
||||
* **[platform] Fix resource allocation ratios not propagated to packages**: Restores propagation of `cpuAllocationRatio`, `memoryAllocationRatio`, and `ephemeralStorageAllocationRatio` from `platform/values.yaml` to the `cozystack-values` Secret that managed applications and KubeVirt read, fixing a regression introduced in the bundle restructure that silently ignored operator-configured ratios *(backported to v1.2.1)* ([**@sircthulhu**](https://github.com/sircthulhu) in #2296).
|
||||
|
|
@ -98,6 +100,8 @@ The dashboard now ships a complete **RestoreJob experience**: list view, details
|
|||
|
||||
* **docs: add OpenSSF Best Practices badge to README**: Adds the OpenSSF Best Practices passing badge to the project README ([**@lexfrei**](https://github.com/lexfrei) in #2320).
|
||||
|
||||
* **[linstor-gui] Restrict to cozystack-cluster-admin group**: Tightens access control on the `linstor-gui` Ingress so the UI and its underlying LINSTOR controller REST API are reachable only by members of the `cozystack-cluster-admin` Keycloak group. Previously, the oauth2-proxy gatekeeper enforced only realm membership (`--email-domain=*`), allowing any tenant-scoped account to reach the gatekeeper's static mTLS client cert *(backported to release-1.3 via #2419)* ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2415, #2419).
|
||||
|
||||
## Dependencies & Version Updates
|
||||
|
||||
* **[kube-ovn] Bump kube-ovn to v1.15.10 with port-group regression fix**: Updates `packages/system/kubeovn` to upstream v1.15.10 (from v1.15.3) and carries a patch for `pkg/controller/pod.go` that preserves a VM LSP's port-group memberships when Kubernetes GCs a completed virt-launcher pod while another virt-launcher pod of the same VM is still running. Without the patch, the destination pod of a successful live migration lost its security groups, network policies, and node-scoped routing until `kube-ovn-controller` was restarted ([**@kvaps**](https://github.com/kvaps) in #2443).
|
||||
|
|
|
|||
|
|
@ -1,37 +0,0 @@
|
|||
<!--
|
||||
https://github.com/cozystack/cozystack/releases/tag/v1.3.1
|
||||
-->
|
||||
|
||||
# v1.3.1 (2026-04-28)
|
||||
|
||||
Patch release covering a TenantNamespace IDOR fix in the API, a destructive `post-upgrade` hook removed from the etcd chart, kamaji controller stability, a `linstor-csi` bump that fixes live migration on Protocol-A/B DRBD resources, the missing `linstor-gui` build wiring, and a velero RBAC fix that unblocked installs on bundles without Velero.
|
||||
|
||||
## Security
|
||||
|
||||
* **fix(api): prevent IDOR in TenantNamespace Get and Watch handlers**: Two IDOR (Insecure Direct Object Reference) vulnerabilities allowed authenticated users to read TenantNamespace metadata they had no RoleBinding for. The `Get` and `Watch` handlers now go through a new `hasAccessToNamespace()` helper that lists RoleBindings scoped only to the target namespace (orders of magnitude faster than the previous all-cluster scan), returns `NotFound` instead of leaking existence on unauthorized access, and applies the same check on the `Watch` filter path. Includes regression tests for the unauthorized paths. ([**@IvanHunters**](https://github.com/IvanHunters) in #2471, backport #2524)
|
||||
|
||||
## Features
|
||||
|
||||
* **feat(linstor): bump linstor-csi to v1.10.6 with Protocol-C dual-attach fix**: Live migration of KubeVirt VMs on Protocol-A/B (async) DRBD volumes no longer fails with `Protocol C required`. `linstor-csi` v1.10.6 now installs a `Protocol=C` override on the resource-definition during dual-attach and reverts it on detach, so `replicated-async` StorageClasses and other Protocol-A/B resource groups support live migration without manual `drbdadm adjust` intervention. ([**@kvaps**](https://github.com/kvaps) in #2496, backport #2505)
|
||||
|
||||
## Fixes
|
||||
|
||||
* **fix(backups): move velero-configmap Role to velero chart**: The `backupstrategy-controller` (a default package) declared a Role/RoleBinding scoped to the `cozy-velero` namespace for managing `ResourceModifier` ConfigMaps. On bundles where Velero was not enabled, that namespace did not exist and the HelmRelease failed with `namespaces "cozy-velero" not found`, blocking installation. The Role/RoleBinding now lives in the velero chart, so it is created only when velero is actually deployed. ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2459, backport #2467)
|
||||
|
||||
* **fix(etcd): remove destructive post-upgrade cert-regeneration hook**: The etcd chart ran a `post-upgrade` Helm hook on every upgrade that deleted etcd TLS Secrets (`etcd-ca-tls`, `etcd-peer-ca-tls`, `etcd-client-tls`, `etcd-peer-tls`, `etcd-server-tls`) and then deleted etcd pods, forcing cert-manager to re-issue the entire etcd CA chain. On clusters with Kamaji-managed tenant control planes this put every tenant `kube-apiserver` into CrashLoopBackOff until each DataStore was manually re-reconciled. The hook was a one-shot `2.6.0 → 2.6.1` migration that became a permanent footgun once chart versioning moved to `0.0.0+<git-hash>` (always `< 2.6.1` per semver) and after the underlying `rotationPolicy: Always` issue was fixed in `47d81f70`. The hook is now removed entirely. ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2462, backport #2511)
|
||||
|
||||
* **fix(kamaji): increase memory limits and add startup probe**: The kamaji controller frequently entered CrashLoopBackOff due to OOMKills (exit 137) within ~20–25 seconds of startup, with the readiness probe failing while the controller was still finishing initialization. Memory limit raised from 500Mi to 512Mi, request from 100Mi to 256Mi, and a 60-second startup probe (12 attempts × 5s periods) is added so the controller has room to boot before liveness/readiness probes engage. ([**@IvanHunters**](https://github.com/IvanHunters) in #2421, backport #2491)
|
||||
|
||||
## Build
|
||||
|
||||
* **build(linstor): include linstor-gui in root image build target**: The `linstor-gui` package (added in #2382) was never wired into the root `Makefile`'s `build:` target, so CI never built or published the image. `ghcr.io/cozystack/cozystack/linstor-gui` returned `NAME_UNKNOWN` and `values.yaml` stayed pinned to `tag: 2.3.0` without a digest. The missing build line is added so the next CI run publishes the image and the per-package `Makefile` digest-pins `values.yaml` automatically. ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #2498, backport #2518)
|
||||
|
||||
## Contributors
|
||||
|
||||
Thanks to everyone who contributed to this patch release:
|
||||
|
||||
* [**@IvanHunters**](https://github.com/IvanHunters)
|
||||
* [**@kvaps**](https://github.com/kvaps)
|
||||
* [**@myasnikovdaniil**](https://github.com/myasnikovdaniil)
|
||||
|
||||
**Full Changelog**: https://github.com/cozystack/cozystack/compare/v1.3.0...v1.3.1
|
||||
|
|
@ -1,145 +0,0 @@
|
|||
#!/usr/bin/env bats
|
||||
# -----------------------------------------------------------------------------
|
||||
# Chart-wide invariant for packages/apps/kubernetes:
|
||||
#
|
||||
# Every Deployment in this chart that mounts <release>-admin-kubeconfig as a
|
||||
# Secret volume MUST:
|
||||
# - declare that volume optional: true (so kubelet does not FailedMount
|
||||
# while Kamaji is still provisioning the Secret), AND
|
||||
# - include the wait-for-kubeconfig init container (so the pod becomes
|
||||
# Ready only after Kamaji publishes the Secret).
|
||||
#
|
||||
# The per-template unittests in packages/apps/kubernetes/tests/ lock in
|
||||
# today's three Deployments (cluster-autoscaler, kccm, csi controller) by
|
||||
# name. This invariant is stricter: any future Deployment added to this
|
||||
# chart that mounts the same Secret but forgets the guard will fail here.
|
||||
#
|
||||
# Requires: helm, yq (mikefarah v4+), jq. All three are available on the
|
||||
# project's CI runners and on the maintainer workstation.
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
@test "every Deployment mounting admin-kubeconfig has optional:true and wait-for-kubeconfig init" {
|
||||
values_file="packages/apps/kubernetes/tests/values-ci.yaml"
|
||||
[ -f "$values_file" ]
|
||||
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
helm template invariant packages/apps/kubernetes \
|
||||
--namespace tenant-root \
|
||||
--values "$values_file" \
|
||||
2>/dev/null > "$tmp/rendered.yaml"
|
||||
|
||||
# yq streams one JSON object per input document. jq -s slurps the stream
|
||||
# into an array so we can treat all Deployments as a single collection.
|
||||
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
|
||||
| jq -s --raw-output '
|
||||
map(select(.kind == "Deployment")) |
|
||||
map({
|
||||
name: .metadata.name,
|
||||
volumes: (.spec.template.spec.volumes // []),
|
||||
initNames: ((.spec.template.spec.initContainers // []) | map(.name)),
|
||||
}) |
|
||||
map(
|
||||
.name as $n |
|
||||
.initNames as $ins |
|
||||
(.volumes[] | select(.secret.secretName | test("-admin-kubeconfig$")?))
|
||||
| {
|
||||
name: $n,
|
||||
optional: (.secret.optional == true),
|
||||
hasInit: ($ins | index("wait-for-kubeconfig") != null),
|
||||
}
|
||||
)
|
||||
' > "$tmp/summary.json"
|
||||
|
||||
# At least one Deployment must match; if a refactor removes every
|
||||
# admin-kubeconfig volume from this chart, the test must be updated
|
||||
# deliberately rather than silently passing.
|
||||
matched=$(jq 'length' "$tmp/summary.json")
|
||||
[ "$matched" -ge 1 ]
|
||||
|
||||
offenders=$(jq --raw-output '.[] | select(.optional != true or .hasInit != true) | .name' "$tmp/summary.json")
|
||||
|
||||
if [ -n "$offenders" ]; then
|
||||
echo "Deployments mounting *-admin-kubeconfig without optional:true + wait-for-kubeconfig init:" >&2
|
||||
echo "$offenders" >&2
|
||||
echo "Full summary:" >&2
|
||||
cat "$tmp/summary.json" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Invariant holds for $matched Deployment(s)"
|
||||
}
|
||||
|
||||
@test "chart emits zero admin-kubeconfig Deployments when tenant has no etcd DataStore" {
|
||||
# Without a DataStore (parent Tenant has not populated _namespace.etcd yet)
|
||||
# the control-plane-side Deployments must NOT render at all. If they did,
|
||||
# the wait-for-kubeconfig init would CrashLoopBackOff indefinitely - there
|
||||
# would be no KamajiControlPlane to provision the Secret - consuming the
|
||||
# HelmRelease wait budget and triggering exactly the remediation cycle the
|
||||
# rest of this chart tries to avoid. This test renders the whole chart
|
||||
# with etcd empty and asserts no Deployment references the admin-kubeconfig
|
||||
# Secret.
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
helm template invariant packages/apps/kubernetes \
|
||||
--namespace tenant-root \
|
||||
--values packages/apps/kubernetes/tests/values-ci-no-etcd.yaml \
|
||||
2>/dev/null > "$tmp/rendered.yaml"
|
||||
|
||||
matched=$(
|
||||
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
|
||||
| jq -s '
|
||||
map(select(.kind == "Deployment")) |
|
||||
map(select(
|
||||
(.spec.template.spec.volumes // [])
|
||||
| any(.secret.secretName | test("-admin-kubeconfig$")?)
|
||||
)) |
|
||||
length
|
||||
'
|
||||
)
|
||||
|
||||
if [ "$matched" -ne 0 ]; then
|
||||
echo "Expected zero Deployments mounting *-admin-kubeconfig when etcd is empty, got $matched:" >&2
|
||||
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
|
||||
| jq -s 'map(select(.kind == "Deployment") | .metadata.name)' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "No admin-kubeconfig Deployments rendered for empty etcd (as expected)"
|
||||
}
|
||||
|
||||
@test "chart emits zero admin-kubeconfig HelmReleases when tenant has no etcd DataStore" {
|
||||
# Same principle as the Deployment variant above, extended to every child
|
||||
# HelmRelease (cilium, coredns, csi, cert-manager, ...). They reference
|
||||
# *-admin-kubeconfig via kubeConfig.secretRef and would otherwise sit in
|
||||
# NotReady forever on an etcd-less tenant, polluting the HelmRelease list
|
||||
# the operator sees and contradicting the "awaiting-etcd beacon only"
|
||||
# contract of the soft-skip path.
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
helm template invariant packages/apps/kubernetes \
|
||||
--namespace tenant-root \
|
||||
--values packages/apps/kubernetes/tests/values-ci-no-etcd.yaml \
|
||||
2>/dev/null > "$tmp/rendered.yaml"
|
||||
|
||||
matched=$(
|
||||
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
|
||||
| jq -s '
|
||||
map(select(.kind == "HelmRelease")) |
|
||||
map(select(.spec.kubeConfig.secretRef.name | test("-admin-kubeconfig$")?)) |
|
||||
length
|
||||
'
|
||||
)
|
||||
|
||||
if [ "$matched" -ne 0 ]; then
|
||||
echo "Expected zero HelmReleases referencing *-admin-kubeconfig when etcd is empty, got $matched:" >&2
|
||||
yq --output-format=json eval-all '.' "$tmp/rendered.yaml" \
|
||||
| jq -s 'map(select(.kind == "HelmRelease") | .metadata.name)' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "No admin-kubeconfig HelmReleases rendered for empty etcd (as expected)"
|
||||
}
|
||||
|
|
@ -1,206 +0,0 @@
|
|||
#!/usr/bin/env bats
|
||||
# -----------------------------------------------------------------------------
|
||||
# Cross-validation between GPU recording rules, the dashboards that consume
|
||||
# them, and the DCGM Exporter metric set the cluster actually scrapes. Catches:
|
||||
#
|
||||
# 1. dangling references — a dashboard query mentions a recording rule name
|
||||
# that doesn't exist in gpu-recording.rules.yaml. This is the bug the
|
||||
# pre-merge review caught: gpu-efficiency.json shipped panels keyed on
|
||||
# pod:tensor_saturation:avg5m without the rule being defined, so the
|
||||
# panel showed "No data" everywhere.
|
||||
#
|
||||
# 2. typos in rule names — same bug class, manifested as a single-character
|
||||
# difference between rule and reference.
|
||||
#
|
||||
# 3. undeclared DCGM metrics — a dashboard query or recording rule mentions
|
||||
# a DCGM_FI_* metric that is neither in the upstream default CSV nor in
|
||||
# the project's custom CSV (dcgm-custom-metrics.yaml), meaning DCGM
|
||||
# Exporter would never emit it and the panel silently shows "No data".
|
||||
# Example regression: gpu-fleet.json shipped a TDP panel referencing
|
||||
# DCGM_FI_DEV_POWER_MGMT_LIMIT before the custom CSV declared it.
|
||||
#
|
||||
# Scope: only dashboards listed in packages/system/monitoring/dashboards-infra.list
|
||||
# under the "gpu/" prefix (i.e. shipped to production), not every JSON file in
|
||||
# dashboards/gpu/. Untracked drafts stay out of scope on purpose — adding one
|
||||
# to dashboards-infra.list is what brings it under the test.
|
||||
#
|
||||
# Reverse direction (rule defined but never consumed) is intentionally NOT
|
||||
# enforced: some rules exist for ad-hoc PromQL or upcoming dashboards. Treat
|
||||
# unused rules as an editorial concern, not a regression.
|
||||
#
|
||||
# Title syntax constraints from cozytest.sh's awk parser:
|
||||
# - Titles delimited by ASCII double quotes; embedded quotes truncate.
|
||||
# - Only [A-Za-z0-9] from the title survives into the function name; titles
|
||||
# differing only in punctuation collapse to the same function.
|
||||
#
|
||||
# Run with: hack/cozytest.sh hack/check-gpu-recording-rules.bats
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
|
||||
RULES_FILE="$REPO_ROOT/packages/system/monitoring-agents/alerts/gpu-recording.rules.yaml"
|
||||
DASHBOARDS_LIST="$REPO_ROOT/packages/system/monitoring/dashboards-infra.list"
|
||||
DASHBOARDS_DIR="$REPO_ROOT/dashboards"
|
||||
DCGM_DEFAULT_CSV="$REPO_ROOT/hack/dcgm-default-counters.csv"
|
||||
DCGM_CUSTOM_CSV="$REPO_ROOT/packages/system/gpu-operator/examples/dcgm-custom-metrics.yaml"
|
||||
|
||||
# Extract the set of "- record: NAME" entries from the rules YAML.
|
||||
# Outputs one rule name per line, sorted and deduplicated.
|
||||
extract_rules() {
|
||||
awk '/^[[:space:]]*-[[:space:]]*record:[[:space:]]/ {
|
||||
sub(/^[[:space:]]*-[[:space:]]*record:[[:space:]]*/, "")
|
||||
sub(/[[:space:]]*$/, "")
|
||||
print
|
||||
}' "$RULES_FILE" | sort -u
|
||||
}
|
||||
|
||||
# Extract the set of recording-rule references from a dashboard JSON.
|
||||
# A recording-rule reference is matched by the pattern
|
||||
# <segment>:<segment>(:<segment>)+
|
||||
# where each <segment> is [a-z0-9_]. Raw DCGM metrics (DCGM_FI_*),
|
||||
# kube-state-metrics (kube_*) and similar uppercase / single-word metric
|
||||
# names do not match because the leading segment must be lowercase and the
|
||||
# whole expression must contain at least two ':' characters.
|
||||
extract_refs() {
|
||||
json_file=$1
|
||||
# Prometheus convention allows 2-segment rule names (level:metric); this
|
||||
# regex is tuned to the 3+ segment convention used in this repo
|
||||
# (level:metric:op — e.g. cluster:gpu_count:total). Update if future
|
||||
# rules use 2 segments, otherwise they will be silently skipped.
|
||||
grep -hoE '[a-z][a-z0-9_]*:[a-z0-9_]+:[a-z0-9_]+' "$json_file" | sort -u
|
||||
}
|
||||
|
||||
# Resolve "gpu/foo" -> "$DASHBOARDS_DIR/gpu/foo.json"
|
||||
list_tracked_gpu_dashboards() {
|
||||
awk '/^gpu\// { print $0 ".json" }' "$DASHBOARDS_LIST"
|
||||
}
|
||||
|
||||
# Extract the set of DCGM_FI_* metric names declared in a CSV file. Handles
|
||||
# both the upstream-style default CSV (unindented) and the ConfigMap-style
|
||||
# custom CSV (YAML-indented). A declaration line starts — after any leading
|
||||
# whitespace — with "DCGM_FI_<NAME>," ; comment lines begin with "#" and are
|
||||
# skipped. Uses POSIX awk's match()+RSTART/RLENGTH so no GNU extensions
|
||||
# are required.
|
||||
extract_csv_metrics() {
|
||||
file=$1
|
||||
awk '
|
||||
{
|
||||
line = $0
|
||||
sub(/^[[:space:]]+/, "", line)
|
||||
if (line ~ /^#/) next
|
||||
if (match(line, /^DCGM_FI_[A-Z0-9_]+/)) {
|
||||
print substr(line, RSTART, RLENGTH)
|
||||
}
|
||||
}
|
||||
' "$file" | sort -u
|
||||
}
|
||||
|
||||
# Extract the set of DCGM_FI_* metric references from a text file (dashboard
|
||||
# JSON or rules YAML). A DCGM metric name has at least two underscore-delimited
|
||||
# segments after the "DCGM_FI_" prefix (e.g. DCGM_FI_DEV_GPU_UTIL, DCGM_FI_PROF_
|
||||
# PIPE_TENSOR_ACTIVE, DCGM_FI_DRIVER_VERSION). Requiring two segments keeps
|
||||
# the matcher from latching onto glob stubs like "DCGM_FI_DEV_*_VIOLATION" that
|
||||
# appear in comments.
|
||||
extract_dcgm_refs() {
|
||||
file=$1
|
||||
grep -hoE 'DCGM_FI_[A-Z0-9]+(_[A-Z0-9]+)+' "$file" | sort -u
|
||||
}
|
||||
|
||||
@test "every recording rule reference in tracked GPU dashboards has a matching record" {
|
||||
TMP=$(mktemp -d)
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
extract_rules > "$TMP/rules.txt"
|
||||
[ -s "$TMP/rules.txt" ] || { echo "no recording rules extracted from $RULES_FILE" >&2; exit 1; }
|
||||
|
||||
list_tracked_gpu_dashboards > "$TMP/dashboards.txt"
|
||||
[ -s "$TMP/dashboards.txt" ] || { echo "no gpu/* dashboards listed in $DASHBOARDS_LIST" >&2; exit 1; }
|
||||
|
||||
failed=0
|
||||
while IFS= read -r dashboard_rel; do
|
||||
dashboard="$DASHBOARDS_DIR/$dashboard_rel"
|
||||
if [ ! -f "$dashboard" ]; then
|
||||
echo "ERROR: dashboard listed but file missing: $dashboard" >&2
|
||||
failed=1
|
||||
continue
|
||||
fi
|
||||
|
||||
extract_refs "$dashboard" > "$TMP/refs.txt"
|
||||
# comm -23: lines unique to refs.txt (referenced but not defined)
|
||||
# Both inputs must be sorted; extract_* helpers already sort.
|
||||
comm -23 "$TMP/refs.txt" "$TMP/rules.txt" > "$TMP/missing.txt"
|
||||
if [ -s "$TMP/missing.txt" ]; then
|
||||
echo "ERROR: $dashboard_rel references undefined recording rules:" >&2
|
||||
sed 's/^/ - /' "$TMP/missing.txt" >&2
|
||||
failed=1
|
||||
fi
|
||||
done < "$TMP/dashboards.txt"
|
||||
|
||||
[ "$failed" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "every DCGM metric referenced in tracked dashboards and rules is declared" {
|
||||
TMP=$(mktemp -d)
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
[ -f "$DCGM_DEFAULT_CSV" ] || { echo "missing $DCGM_DEFAULT_CSV" >&2; exit 1; }
|
||||
[ -f "$DCGM_CUSTOM_CSV" ] || { echo "missing $DCGM_CUSTOM_CSV" >&2; exit 1; }
|
||||
|
||||
{
|
||||
extract_csv_metrics "$DCGM_DEFAULT_CSV"
|
||||
extract_csv_metrics "$DCGM_CUSTOM_CSV"
|
||||
} | sort -u > "$TMP/declared.txt"
|
||||
[ -s "$TMP/declared.txt" ] || { echo "no DCGM metrics extracted from CSVs" >&2; exit 1; }
|
||||
|
||||
list_tracked_gpu_dashboards > "$TMP/dashboards.txt"
|
||||
[ -s "$TMP/dashboards.txt" ] || { echo "no gpu/* dashboards listed in $DASHBOARDS_LIST" >&2; exit 1; }
|
||||
|
||||
failed=0
|
||||
|
||||
# Dashboard coverage — every dashboard's DCGM references must resolve.
|
||||
while IFS= read -r dashboard_rel; do
|
||||
dashboard="$DASHBOARDS_DIR/$dashboard_rel"
|
||||
[ -f "$dashboard" ] || continue # handled by the existence test
|
||||
extract_dcgm_refs "$dashboard" > "$TMP/refs.txt"
|
||||
[ -s "$TMP/refs.txt" ] || continue # dashboard relies entirely on recording rules
|
||||
comm -23 "$TMP/refs.txt" "$TMP/declared.txt" > "$TMP/missing.txt"
|
||||
if [ -s "$TMP/missing.txt" ]; then
|
||||
echo "ERROR: $dashboard_rel references DCGM metrics not declared in any CSV:" >&2
|
||||
sed 's/^/ - /' "$TMP/missing.txt" >&2
|
||||
failed=1
|
||||
fi
|
||||
done < "$TMP/dashboards.txt"
|
||||
|
||||
# Rules coverage — recording rules consume DCGM directly, so their set
|
||||
# must be declared too, otherwise derived series on every dashboard
|
||||
# collapse to empty.
|
||||
extract_dcgm_refs "$RULES_FILE" > "$TMP/rule-refs.txt"
|
||||
if [ -s "$TMP/rule-refs.txt" ]; then
|
||||
comm -23 "$TMP/rule-refs.txt" "$TMP/declared.txt" > "$TMP/rule-missing.txt"
|
||||
if [ -s "$TMP/rule-missing.txt" ]; then
|
||||
echo "ERROR: gpu-recording.rules.yaml references DCGM metrics not declared in any CSV:" >&2
|
||||
sed 's/^/ - /' "$TMP/rule-missing.txt" >&2
|
||||
failed=1
|
||||
fi
|
||||
fi
|
||||
|
||||
[ "$failed" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "every tracked GPU dashboard listed in dashboards-infra.list exists on disk" {
|
||||
TMP=$(mktemp -d)
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
list_tracked_gpu_dashboards > "$TMP/dashboards.txt"
|
||||
[ -s "$TMP/dashboards.txt" ] || { echo "no gpu/* dashboards listed in $DASHBOARDS_LIST" >&2; exit 1; }
|
||||
|
||||
failed=0
|
||||
while IFS= read -r dashboard_rel; do
|
||||
dashboard="$DASHBOARDS_DIR/$dashboard_rel"
|
||||
if [ ! -f "$dashboard" ]; then
|
||||
echo "ERROR: $dashboard_rel listed in dashboards-infra.list but $dashboard does not exist" >&2
|
||||
failed=1
|
||||
fi
|
||||
done < "$TMP/dashboards.txt"
|
||||
|
||||
[ "$failed" -eq 0 ]
|
||||
}
|
||||
|
|
@ -1,104 +0,0 @@
|
|||
# Snapshot of the upstream DCGM Exporter default-counters.csv used as a
|
||||
# fixture by hack/check-gpu-recording-rules.bats. The test verifies that
|
||||
# every DCGM_FI_* metric referenced by a tracked GPU dashboard is either
|
||||
# declared here (upstream defaults) or in
|
||||
# packages/system/gpu-operator/examples/dcgm-custom-metrics.yaml
|
||||
# (the project's custom CSV).
|
||||
#
|
||||
# Source: https://github.com/NVIDIA/dcgm-exporter/blob/4.1.1-4.0.4/etc/default-counters.csv
|
||||
# Pinned to the DCGM Exporter image tag shipped by the gpu-operator
|
||||
# chart under packages/system/gpu-operator/charts/gpu-operator/values.yaml
|
||||
# (dcgmExporter.version = 4.1.1-4.0.4-ubuntu22.04). When that image is
|
||||
# bumped, refresh this file from the matching tag in the NVIDIA/dcgm-exporter
|
||||
# repo and re-run `./hack/cozytest.sh hack/check-gpu-recording-rules.bats`.
|
||||
|
||||
# Format
|
||||
# If line starts with a '#' it is considered a comment
|
||||
# DCGM FIELD, Prometheus metric type, help message
|
||||
|
||||
# Clocks
|
||||
DCGM_FI_DEV_SM_CLOCK, gauge, SM clock frequency (in MHz).
|
||||
DCGM_FI_DEV_MEM_CLOCK, gauge, Memory clock frequency (in MHz).
|
||||
|
||||
# Temperature
|
||||
DCGM_FI_DEV_MEMORY_TEMP, gauge, Memory temperature (in C).
|
||||
DCGM_FI_DEV_GPU_TEMP, gauge, GPU temperature (in C).
|
||||
|
||||
# Power
|
||||
DCGM_FI_DEV_POWER_USAGE, gauge, Power draw (in W).
|
||||
DCGM_FI_DEV_TOTAL_ENERGY_CONSUMPTION, counter, Total energy consumption since boot (in mJ).
|
||||
|
||||
# PCIE
|
||||
# DCGM_FI_PROF_PCIE_TX_BYTES, counter, Total number of bytes transmitted through PCIe TX via NVML.
|
||||
# DCGM_FI_PROF_PCIE_RX_BYTES, counter, Total number of bytes received through PCIe RX via NVML.
|
||||
DCGM_FI_DEV_PCIE_REPLAY_COUNTER, counter, Total number of PCIe retries.
|
||||
|
||||
# Utilization (the sample period varies depending on the product)
|
||||
DCGM_FI_DEV_GPU_UTIL, gauge, GPU utilization (in %).
|
||||
DCGM_FI_DEV_MEM_COPY_UTIL, gauge, Memory utilization (in %).
|
||||
DCGM_FI_DEV_ENC_UTIL, gauge, Encoder utilization (in %).
|
||||
DCGM_FI_DEV_DEC_UTIL , gauge, Decoder utilization (in %).
|
||||
|
||||
# Errors and violations
|
||||
DCGM_FI_DEV_XID_ERRORS, gauge, Value of the last XID error encountered.
|
||||
# DCGM_FI_DEV_POWER_VIOLATION, counter, Throttling duration due to power constraints (in us).
|
||||
# DCGM_FI_DEV_THERMAL_VIOLATION, counter, Throttling duration due to thermal constraints (in us).
|
||||
# DCGM_FI_DEV_SYNC_BOOST_VIOLATION, counter, Throttling duration due to sync-boost constraints (in us).
|
||||
# DCGM_FI_DEV_BOARD_LIMIT_VIOLATION, counter, Throttling duration due to board limit constraints (in us).
|
||||
# DCGM_FI_DEV_LOW_UTIL_VIOLATION, counter, Throttling duration due to low utilization (in us).
|
||||
# DCGM_FI_DEV_RELIABILITY_VIOLATION, counter, Throttling duration due to reliability constraints (in us).
|
||||
|
||||
# Memory usage
|
||||
DCGM_FI_DEV_FB_FREE, gauge, Framebuffer memory free (in MiB).
|
||||
DCGM_FI_DEV_FB_USED, gauge, Framebuffer memory used (in MiB).
|
||||
|
||||
# ECC
|
||||
# DCGM_FI_DEV_ECC_SBE_VOL_TOTAL, counter, Total number of single-bit volatile ECC errors.
|
||||
# DCGM_FI_DEV_ECC_DBE_VOL_TOTAL, counter, Total number of double-bit volatile ECC errors.
|
||||
# DCGM_FI_DEV_ECC_SBE_AGG_TOTAL, counter, Total number of single-bit persistent ECC errors.
|
||||
# DCGM_FI_DEV_ECC_DBE_AGG_TOTAL, counter, Total number of double-bit persistent ECC errors.
|
||||
|
||||
# Retired pages
|
||||
# DCGM_FI_DEV_RETIRED_SBE, counter, Total number of retired pages due to single-bit errors.
|
||||
# DCGM_FI_DEV_RETIRED_DBE, counter, Total number of retired pages due to double-bit errors.
|
||||
# DCGM_FI_DEV_RETIRED_PENDING, counter, Total number of pages pending retirement.
|
||||
|
||||
# NVLink
|
||||
# DCGM_FI_DEV_NVLINK_CRC_FLIT_ERROR_COUNT_TOTAL, counter, Total number of NVLink flow-control CRC errors.
|
||||
# DCGM_FI_DEV_NVLINK_CRC_DATA_ERROR_COUNT_TOTAL, counter, Total number of NVLink data CRC errors.
|
||||
# DCGM_FI_DEV_NVLINK_REPLAY_ERROR_COUNT_TOTAL, counter, Total number of NVLink retries.
|
||||
# DCGM_FI_DEV_NVLINK_RECOVERY_ERROR_COUNT_TOTAL, counter, Total number of NVLink recovery errors.
|
||||
DCGM_FI_DEV_NVLINK_BANDWIDTH_TOTAL, counter, Total number of NVLink bandwidth counters for all lanes.
|
||||
# DCGM_FI_DEV_NVLINK_BANDWIDTH_L0, counter, The number of bytes of active NVLink rx or tx data including both header and payload.
|
||||
|
||||
# VGPU License status
|
||||
DCGM_FI_DEV_VGPU_LICENSE_STATUS, gauge, vGPU License status
|
||||
|
||||
# Remapped rows
|
||||
DCGM_FI_DEV_UNCORRECTABLE_REMAPPED_ROWS, counter, Number of remapped rows for uncorrectable errors
|
||||
DCGM_FI_DEV_CORRECTABLE_REMAPPED_ROWS, counter, Number of remapped rows for correctable errors
|
||||
DCGM_FI_DEV_ROW_REMAP_FAILURE, gauge, Whether remapping of rows has failed
|
||||
|
||||
# Static configuration information. These appear as labels on the other metrics
|
||||
DCGM_FI_DRIVER_VERSION, label, Driver Version
|
||||
# DCGM_FI_NVML_VERSION, label, NVML Version
|
||||
# DCGM_FI_DEV_BRAND, label, Device Brand
|
||||
# DCGM_FI_DEV_SERIAL, label, Device Serial Number
|
||||
# DCGM_FI_DEV_OEM_INFOROM_VER, label, OEM inforom version
|
||||
# DCGM_FI_DEV_ECC_INFOROM_VER, label, ECC inforom version
|
||||
# DCGM_FI_DEV_POWER_INFOROM_VER, label, Power management object inforom version
|
||||
# DCGM_FI_DEV_INFOROM_IMAGE_VER, label, Inforom image version
|
||||
# DCGM_FI_DEV_VBIOS_VERSION, label, VBIOS version of the device
|
||||
|
||||
# Datacenter Profiling (DCP) metrics
|
||||
# NOTE: supported on Nvidia datacenter Volta GPUs and newer
|
||||
DCGM_FI_PROF_GR_ENGINE_ACTIVE, gauge, Ratio of time the graphics engine is active.
|
||||
# DCGM_FI_PROF_SM_ACTIVE, gauge, The ratio of cycles an SM has at least 1 warp assigned.
|
||||
# DCGM_FI_PROF_SM_OCCUPANCY, gauge, The ratio of number of warps resident on an SM.
|
||||
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE, gauge, Ratio of cycles the tensor (HMMA) pipe is active.
|
||||
DCGM_FI_PROF_DRAM_ACTIVE, gauge, Ratio of cycles the device memory interface is active sending or receiving data.
|
||||
# DCGM_FI_PROF_PIPE_FP64_ACTIVE, gauge, Ratio of cycles the fp64 pipes are active.
|
||||
# DCGM_FI_PROF_PIPE_FP32_ACTIVE, gauge, Ratio of cycles the fp32 pipes are active.
|
||||
# DCGM_FI_PROF_PIPE_FP16_ACTIVE, gauge, Ratio of cycles the fp16 pipes are active.
|
||||
DCGM_FI_PROF_PCIE_TX_BYTES, gauge, The rate of data transmitted over the PCIe bus - including both protocol headers and data payloads - in bytes per second.
|
||||
DCGM_FI_PROF_PCIE_RX_BYTES, gauge, The rate of data received over the PCIe bus - including both protocol headers and data payloads - in bytes per second.
|
||||
|
Can't render this file because it has a wrong number of fields in line 12.
|
|
|
@ -1,39 +0,0 @@
|
|||
# Helpers for asserting that a Flux HelmRelease did not fall into an
|
||||
# install/upgrade remediation cycle during an e2e run.
|
||||
#
|
||||
# Background: Flux helm-controller's ClearFailures() zeroes
|
||||
# .status.installFailures / .status.upgradeFailures on every successful
|
||||
# reconciliation (see the upstream ClearFailures method on
|
||||
# HelmReleaseStatus). That makes those counters useless for a guard that
|
||||
# runs after the HelmRelease has reached Ready - the values are always 0.
|
||||
#
|
||||
# What survives a successful reconciliation is .status.history, a bounded
|
||||
# list of release Snapshots. Each Snapshot carries a status field that
|
||||
# tracks the Helm release state: deployed, superseded, failed, uninstalled,
|
||||
# and so on. A remediation cycle leaves the footprint behind: a snapshot
|
||||
# with status "uninstalled" (from install/upgrade remediation) or "failed"
|
||||
# (Helm release failure that remediation then uninstalled). Those stay in
|
||||
# history even after a subsequent successful reinstall.
|
||||
#
|
||||
# helmrelease_has_remediation_cycle takes a newline-delimited list of
|
||||
# snapshot statuses (whatever the caller extracted via kubectl -o jsonpath
|
||||
# or equivalent) and returns 0 (detected) when any entry is "failed" or
|
||||
# "uninstalled", 1 otherwise. Empty input is treated as "no history yet,
|
||||
# no cycle observed".
|
||||
|
||||
helmrelease_has_remediation_cycle() {
|
||||
statuses="$1"
|
||||
if [ -z "${statuses}" ]; then
|
||||
return 1
|
||||
fi
|
||||
# printf + grep over the pipe, rather than a heredoc plus while read.
|
||||
# printf %s treats the status string as a literal payload, so any stray
|
||||
# $ in a future caller's input does not trigger shell expansion. grep
|
||||
# returns 0 iff at least one line matches the allowlist, which is
|
||||
# exactly the contract the caller wants, so we can return its exit
|
||||
# status directly.
|
||||
if printf '%s\n' "${statuses}" | grep --extended-regexp --quiet '^(failed|uninstalled)$'; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
|
@ -1,5 +1,3 @@
|
|||
. hack/e2e-apps/remediation-guard.sh
|
||||
|
||||
run_kubernetes_test() {
|
||||
local version_expr="$1"
|
||||
local test_name="$2"
|
||||
|
|
@ -322,35 +320,6 @@ EOF
|
|||
done
|
||||
kubectl wait hr kubernetes-${test_name}-ingress-nginx -n tenant-test --timeout=5m --for=condition=ready
|
||||
|
||||
# Guard: parent HelmRelease must not have entered an install/upgrade remediation cycle.
|
||||
# A non-zero installFailures/upgradeFailures indicates the helm-wait budget expired while
|
||||
# admin-kubeconfig was still being provisioned, which would trigger uninstall remediation
|
||||
# and churn the Cluster CR.
|
||||
# Flux helm-controller v2 retains per-revision release Snapshots in
|
||||
# .status.history; each Snapshot's .status reflects the Helm release
|
||||
# state (deployed/superseded/failed/uninstalled). A remediation cycle
|
||||
# leaves a "failed" or "uninstalled" entry behind that survives a later
|
||||
# successful reinstall, unlike the installFailures/upgradeFailures
|
||||
# counters (which ClearFailures zeroes on every successful reconcile).
|
||||
# The shape is pinned by hack/remediation-guard.bats; the upstream
|
||||
# types are github.com/fluxcd/helm-controller/api v2 Snapshot.
|
||||
history_statuses=$(kubectl get hr -n tenant-test "kubernetes-${test_name}" \
|
||||
-ojsonpath='{range .status.history[*]}{.status}{"\n"}{end}')
|
||||
# Always emit the raw value so a silent future-Flux field rename shows
|
||||
# up as "empty history on a Ready HR" in CI logs rather than vanishing.
|
||||
echo "Parent HelmRelease history statuses:"
|
||||
printf '%s\n' "${history_statuses:-<empty>}"
|
||||
if [ -z "${history_statuses}" ]; then
|
||||
echo "Unexpected empty .status.history on a Ready HelmRelease - Flux API shape may have changed." >&2
|
||||
kubectl -n tenant-test describe hr "kubernetes-${test_name}" >&2
|
||||
exit 1
|
||||
fi
|
||||
if helmrelease_has_remediation_cycle "${history_statuses}"; then
|
||||
echo "Parent HelmRelease entered remediation cycle." >&2
|
||||
kubectl -n tenant-test describe hr "kubernetes-${test_name}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Clean up
|
||||
pkill -f "port-forward.*${port}:" 2>/dev/null || true
|
||||
rm -f "tenantkubeconfig-${test_name}"
|
||||
|
|
|
|||
|
|
@ -1,127 +0,0 @@
|
|||
#!/usr/bin/env bats
|
||||
# -----------------------------------------------------------------------------
|
||||
# Unit tests for hack/e2e-apps/remediation-guard.sh
|
||||
#
|
||||
# helmrelease_has_remediation_cycle takes a newline-delimited list of
|
||||
# HelmRelease history snapshot status values (deployed/superseded/failed/
|
||||
# uninstalled/...) and returns 0 when any entry is "failed" or "uninstalled"
|
||||
# (meaning flux helm-controller performed install/upgrade remediation).
|
||||
#
|
||||
# This is used by the e2e script after the HelmRelease reaches Ready. The
|
||||
# failure/upgrade counters (.status.installFailures / .status.upgradeFailures)
|
||||
# are useless there because flux's ClearFailures zeroes them on successful
|
||||
# reconciliation; .status.history retains the snapshot trail.
|
||||
#
|
||||
# cozytest.sh's awk parser recognizes only @test blocks and a bare `}` on
|
||||
# its own line; there is no bats `run` or `$status`. Assertions are
|
||||
# expressed as direct shell tests that exit non-zero on failure.
|
||||
#
|
||||
# Run with: hack/cozytest.sh hack/remediation-guard.bats
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
@test "empty history returns not-detected" {
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
if helmrelease_has_remediation_cycle ""; then
|
||||
echo "expected not-detected for empty history" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "single deployed snapshot returns not-detected" {
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
if helmrelease_has_remediation_cycle "deployed"; then
|
||||
echo "expected not-detected for deployed-only history" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "deployed then superseded returns not-detected" {
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
statuses=$(printf 'deployed\nsuperseded\n')
|
||||
if helmrelease_has_remediation_cycle "${statuses}"; then
|
||||
echo "expected not-detected for deployed+superseded history" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "single failed snapshot returns detected" {
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
if ! helmrelease_has_remediation_cycle "failed"; then
|
||||
echo "expected detected when history contains failed snapshot" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "single uninstalled snapshot returns detected" {
|
||||
# The exact signature of the install-remediation race: the first install
|
||||
# exceeded flux's wait budget, remediation uninstalled, the next retry
|
||||
# eventually succeeded. History still carries the uninstalled snapshot.
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
if ! helmrelease_has_remediation_cycle "uninstalled"; then
|
||||
echo "expected detected when history contains uninstalled snapshot" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "uninstalled then deployed still returns detected" {
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
statuses=$(printf 'uninstalled\ndeployed\n')
|
||||
if ! helmrelease_has_remediation_cycle "${statuses}"; then
|
||||
echo "expected detected despite later successful deploy" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "deployed then failed still returns detected" {
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
statuses=$(printf 'deployed\nfailed\n')
|
||||
if ! helmrelease_has_remediation_cycle "${statuses}"; then
|
||||
echo "expected detected when any entry is failed" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "status.history extraction pins HR v2 status.history shape" {
|
||||
# Pins the Flux HelmRelease v2 .status.history[].status shape that
|
||||
# run-kubernetes.sh relies on. If a future flux release renames the
|
||||
# field, the jsonpath returns nothing, the guard reports no cycle,
|
||||
# and real remediation loops slip past the e2e assertion. This test
|
||||
# uses yq to read the exact path used in the e2e script; the upstream
|
||||
# Snapshot type lives at
|
||||
# github.com/fluxcd/helm-controller/api/v2.Snapshot (via go.mod).
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
cat > "$tmp/hr.yaml" <<'YAML'
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: kubernetes-test
|
||||
namespace: tenant-test
|
||||
status:
|
||||
history:
|
||||
- name: kubernetes-test
|
||||
namespace: tenant-test
|
||||
version: 1
|
||||
status: uninstalled
|
||||
- name: kubernetes-test
|
||||
namespace: tenant-test
|
||||
version: 2
|
||||
status: deployed
|
||||
YAML
|
||||
|
||||
# Default yq output is yaml scalar format, which for string values emits
|
||||
# bare unquoted tokens - matching what kubectl -o jsonpath produces in
|
||||
# e2e. Do not switch to JSON output here; that would quote the values
|
||||
# and break the loop in helmrelease_has_remediation_cycle.
|
||||
statuses=$(yq '.status.history[].status' "$tmp/hr.yaml")
|
||||
|
||||
[ -n "$statuses" ]
|
||||
echo "$statuses" | grep --quiet '^uninstalled$'
|
||||
|
||||
. hack/e2e-apps/remediation-guard.sh
|
||||
if ! helmrelease_has_remediation_cycle "$statuses"; then
|
||||
echo "expected detected for pinned HR snippet with uninstalled + deployed history" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
|
@ -118,19 +118,6 @@ spec:
|
|||
ReleaseName is the name of the HelmRelease resource that will be created
|
||||
If not specified, defaults to the component Name field
|
||||
type: string
|
||||
upgradeCRDs:
|
||||
description: |-
|
||||
UpgradeCRDs controls how CRDs from the chart's crds/ directory are
|
||||
handled on HelmRelease upgrades. Maps to HelmRelease.Spec.Upgrade.CRDs.
|
||||
Empty string (default) preserves the helm-controller default (Skip).
|
||||
Use "CreateReplace" for operators that evolve their CRD set between
|
||||
versions. Warning: CreateReplace overwrites CRDs and may cause data
|
||||
loss if upstream drops fields from a CRD with live objects.
|
||||
enum:
|
||||
- Skip
|
||||
- Create
|
||||
- CreateReplace
|
||||
type: string
|
||||
type: object
|
||||
libraries:
|
||||
description: |-
|
||||
|
|
|
|||
|
|
@ -45,16 +45,6 @@ const (
|
|||
SecretCozystackValues = "cozystack-values"
|
||||
)
|
||||
|
||||
// parseCRDPolicy maps ComponentInstall.UpgradeCRDs to a helmv2.CRDsPolicy.
|
||||
// Empty / nil preserves the helm-controller default (Skip on upgrade);
|
||||
// the CRD enum marker restricts the string to Skip/Create/CreateReplace.
|
||||
func parseCRDPolicy(install *cozyv1alpha1.ComponentInstall) helmv2.CRDsPolicy {
|
||||
if install == nil || install.UpgradeCRDs == "" {
|
||||
return ""
|
||||
}
|
||||
return helmv2.CRDsPolicy(install.UpgradeCRDs)
|
||||
}
|
||||
|
||||
// PackageReconciler reconciles Package resources
|
||||
type PackageReconciler struct {
|
||||
client.Client
|
||||
|
|
@ -231,7 +221,6 @@ func (r *PackageReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ct
|
|||
Remediation: &helmv2.UpgradeRemediation{
|
||||
Retries: -1,
|
||||
},
|
||||
CRDs: parseCRDPolicy(component.Install),
|
||||
},
|
||||
},
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,138 +0,0 @@
|
|||
/*
|
||||
Copyright 2025 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package operator
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
cozyv1alpha1 "github.com/cozystack/cozystack/api/v1alpha1"
|
||||
helmv2 "github.com/fluxcd/helm-controller/api/v2"
|
||||
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
|
||||
"sigs.k8s.io/yaml"
|
||||
)
|
||||
|
||||
func TestParseCRDPolicy(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
install *cozyv1alpha1.ComponentInstall
|
||||
want helmv2.CRDsPolicy
|
||||
}{
|
||||
{
|
||||
name: "nil install leaves flux default",
|
||||
install: nil,
|
||||
want: "",
|
||||
},
|
||||
{
|
||||
name: "empty upgradeCRDs leaves flux default",
|
||||
install: &cozyv1alpha1.ComponentInstall{},
|
||||
want: "",
|
||||
},
|
||||
{
|
||||
name: "Skip is passed through",
|
||||
install: &cozyv1alpha1.ComponentInstall{UpgradeCRDs: "Skip"},
|
||||
want: helmv2.Skip,
|
||||
},
|
||||
{
|
||||
name: "Create is passed through",
|
||||
install: &cozyv1alpha1.ComponentInstall{UpgradeCRDs: "Create"},
|
||||
want: helmv2.Create,
|
||||
},
|
||||
{
|
||||
name: "CreateReplace is passed through",
|
||||
install: &cozyv1alpha1.ComponentInstall{UpgradeCRDs: "CreateReplace"},
|
||||
want: helmv2.CreateReplace,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := parseCRDPolicy(tc.install)
|
||||
if got != tc.want {
|
||||
t.Errorf("parseCRDPolicy() = %q, want %q", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestPackageSourceCRDHasUpgradeCRDsEnum guards the generated CRD schema: the
|
||||
// invalid-value case from the spec is enforced at the API server via a
|
||||
// kubebuilder enum marker, not in the reconciler. If someone drops the marker
|
||||
// and forgets to regenerate, this test catches it.
|
||||
func TestPackageSourceCRDHasUpgradeCRDsEnum(t *testing.T) {
|
||||
path := filepath.Join("..", "crdinstall", "manifests", "cozystack.io_packagesources.yaml")
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read %s: %v", path, err)
|
||||
}
|
||||
|
||||
var crd apiextensionsv1.CustomResourceDefinition
|
||||
if err := yaml.Unmarshal(data, &crd); err != nil {
|
||||
t.Fatalf("unmarshal CRD: %v", err)
|
||||
}
|
||||
|
||||
var field *apiextensionsv1.JSONSchemaProps
|
||||
for i := range crd.Spec.Versions {
|
||||
v := &crd.Spec.Versions[i]
|
||||
if v.Schema == nil || v.Schema.OpenAPIV3Schema == nil {
|
||||
continue
|
||||
}
|
||||
spec, ok := v.Schema.OpenAPIV3Schema.Properties["spec"]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
variants, ok := spec.Properties["variants"]
|
||||
if !ok || variants.Items == nil || variants.Items.Schema == nil {
|
||||
continue
|
||||
}
|
||||
components, ok := variants.Items.Schema.Properties["components"]
|
||||
if !ok || components.Items == nil || components.Items.Schema == nil {
|
||||
continue
|
||||
}
|
||||
install, ok := components.Items.Schema.Properties["install"]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
f, ok := install.Properties["upgradeCRDs"]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
field = &f
|
||||
break
|
||||
}
|
||||
|
||||
if field == nil {
|
||||
t.Fatal("upgradeCRDs field not found in PackageSource CRD schema")
|
||||
}
|
||||
|
||||
got := map[string]bool{}
|
||||
for _, e := range field.Enum {
|
||||
var s string
|
||||
if err := json.Unmarshal(e.Raw, &s); err != nil {
|
||||
t.Fatalf("unmarshal enum value %q: %v", e.Raw, err)
|
||||
}
|
||||
got[s] = true
|
||||
}
|
||||
|
||||
for _, want := range []string{"Skip", "Create", "CreateReplace"} {
|
||||
if !got[want] {
|
||||
t.Errorf("enum value %q missing from upgradeCRDs; got %v", want, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -4,9 +4,6 @@ KUBERNETES_PKG_TAG = $(shell awk '$$1 == "version:" {print $$2}' Chart.yaml)
|
|||
include ../../../hack/common-envs.mk
|
||||
include ../../../hack/package.mk
|
||||
|
||||
test:
|
||||
helm unittest .
|
||||
|
||||
generate:
|
||||
cozyvalues-gen -m 'kubernetes' -v values.yaml -s values.schema.json -r README.md -g ../../../api/apps/v1alpha1/kubernetes/types.go
|
||||
../../../hack/update-crd.sh
|
||||
|
|
@ -70,4 +67,3 @@ image-cluster-autoscaler:
|
|||
echo "$(REGISTRY)/cluster-autoscaler:$(call settag,$(KUBERNETES_PKG_TAG))@$$(yq e '."containerimage.digest"' images/cluster-autoscaler.json -o json -r)" \
|
||||
> images/cluster-autoscaler.tag
|
||||
rm -f images/cluster-autoscaler.json
|
||||
|
||||
|
|
|
|||
|
|
@ -128,9 +128,6 @@ See the reference for components utilized in this service:
|
|||
| `addons.gpuOperator` | NVIDIA GPU Operator. | `object` | `{}` |
|
||||
| `addons.gpuOperator.enabled` | Enable GPU Operator. | `bool` | `false` |
|
||||
| `addons.gpuOperator.valuesOverride` | Custom Helm values overrides. | `object` | `{}` |
|
||||
| `addons.hami` | HAMi GPU virtualization middleware. | `object` | `{}` |
|
||||
| `addons.hami.enabled` | Enable HAMi (requires GPU Operator). | `bool` | `false` |
|
||||
| `addons.hami.valuesOverride` | Custom Helm values overrides. | `object` | `{}` |
|
||||
| `addons.fluxcd` | FluxCD GitOps operator. | `object` | `{}` |
|
||||
| `addons.fluxcd.enabled` | Enable FluxCD. | `bool` | `false` |
|
||||
| `addons.fluxcd.valuesOverride` | Custom Helm values overrides. | `object` | `{}` |
|
||||
|
|
@ -148,33 +145,31 @@ See the reference for components utilized in this service:
|
|||
|
||||
### Kubernetes Control Plane Configuration
|
||||
|
||||
| Name | Description | Type | Value |
|
||||
| --------------------------------------------------- | --------------------------------------------------------------------------------------------- | ---------- | ------- |
|
||||
| `controlPlane` | Kubernetes control-plane configuration. | `object` | `{}` |
|
||||
| `controlPlane.replicas` | Number of control-plane replicas. | `int` | `2` |
|
||||
| `controlPlane.apiServer` | API Server configuration. | `object` | `{}` |
|
||||
| `controlPlane.apiServer.resources` | CPU and memory resources for API Server. | `object` | `{}` |
|
||||
| `controlPlane.apiServer.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.apiServer.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.apiServer.resourcesPreset` | Preset if `resources` omitted. | `string` | `large` |
|
||||
| `controlPlane.controllerManager` | Controller Manager configuration. | `object` | `{}` |
|
||||
| `controlPlane.controllerManager.resources` | CPU and memory resources for Controller Manager. | `object` | `{}` |
|
||||
| `controlPlane.controllerManager.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.controllerManager.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.controllerManager.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
|
||||
| `controlPlane.scheduler` | Scheduler configuration. | `object` | `{}` |
|
||||
| `controlPlane.scheduler.resources` | CPU and memory resources for Scheduler. | `object` | `{}` |
|
||||
| `controlPlane.scheduler.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.scheduler.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.scheduler.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
|
||||
| `controlPlane.konnectivity` | Konnectivity configuration. | `object` | `{}` |
|
||||
| `controlPlane.konnectivity.server` | Konnectivity Server configuration. | `object` | `{}` |
|
||||
| `controlPlane.konnectivity.server.resources` | CPU and memory resources for Konnectivity. | `object` | `{}` |
|
||||
| `controlPlane.konnectivity.server.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.konnectivity.server.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.konnectivity.server.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
|
||||
| `images` | Optional image overrides for air-gapped or rate-limited registries. | `object` | `{}` |
|
||||
| `images.waitForKubeconfig` | Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag. | `string` | `""` |
|
||||
| Name | Description | Type | Value |
|
||||
| --------------------------------------------------- | ------------------------------------------------ | ---------- | ------- |
|
||||
| `controlPlane` | Kubernetes control-plane configuration. | `object` | `{}` |
|
||||
| `controlPlane.replicas` | Number of control-plane replicas. | `int` | `2` |
|
||||
| `controlPlane.apiServer` | API Server configuration. | `object` | `{}` |
|
||||
| `controlPlane.apiServer.resources` | CPU and memory resources for API Server. | `object` | `{}` |
|
||||
| `controlPlane.apiServer.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.apiServer.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.apiServer.resourcesPreset` | Preset if `resources` omitted. | `string` | `large` |
|
||||
| `controlPlane.controllerManager` | Controller Manager configuration. | `object` | `{}` |
|
||||
| `controlPlane.controllerManager.resources` | CPU and memory resources for Controller Manager. | `object` | `{}` |
|
||||
| `controlPlane.controllerManager.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.controllerManager.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.controllerManager.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
|
||||
| `controlPlane.scheduler` | Scheduler configuration. | `object` | `{}` |
|
||||
| `controlPlane.scheduler.resources` | CPU and memory resources for Scheduler. | `object` | `{}` |
|
||||
| `controlPlane.scheduler.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.scheduler.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.scheduler.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
|
||||
| `controlPlane.konnectivity` | Konnectivity configuration. | `object` | `{}` |
|
||||
| `controlPlane.konnectivity.server` | Konnectivity Server configuration. | `object` | `{}` |
|
||||
| `controlPlane.konnectivity.server.resources` | CPU and memory resources for Konnectivity. | `object` | `{}` |
|
||||
| `controlPlane.konnectivity.server.resources.cpu` | CPU available. | `quantity` | `""` |
|
||||
| `controlPlane.konnectivity.server.resources.memory` | Memory (RAM) available. | `quantity` | `""` |
|
||||
| `controlPlane.konnectivity.server.resourcesPreset` | Preset if `resources` omitted. | `string` | `micro` |
|
||||
|
||||
|
||||
## Parameter examples and reference
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
docker.io/library/busybox:1.37.0@sha256:1487d0af5f52b4ba31c7e465126ee2123fe3f2305d638e7827681e7cf6c83d5e
|
||||
|
|
@ -1 +1 @@
|
|||
ghcr.io/cozystack/cozystack/kubevirt-csi-driver:0.0.0@sha256:72154a97054e16cdf3dea6129d962b8d7e86b55cf9386095e8ac2ce7c8b69172
|
||||
ghcr.io/cozystack/cozystack/kubevirt-csi-driver:0.0.0@sha256:0e70f4391b344c71a9567e1ed22ca24910116a619cb681381059b8c39f382219
|
||||
|
|
|
|||
|
|
@ -49,52 +49,3 @@ Selector labels
|
|||
app.kubernetes.io/name: {{ include "kubernetes.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
wait-for-kubeconfig init container shared by the control-plane-side
|
||||
Deployments (cluster-autoscaler, kccm, kcsi-controller) that mount the
|
||||
*-admin-kubeconfig Secret provisioned asynchronously by Kamaji. The
|
||||
Secret volume is declared optional so kubelet does not FailedMount while
|
||||
Kamaji is still bootstrapping; this container polls the mounted path and
|
||||
exits only when super-admin.svc appears, which happens after kubelet's
|
||||
optional-Secret refresh cycle.
|
||||
|
||||
The 10m deadline stays strictly below the 15m HelmRelease
|
||||
Install.Timeout set by cozystack-api for the Kubernetes kind (via the
|
||||
release.cozystack.io/helm-install-timeout annotation) so the
|
||||
CrashLoopBackOff surfaces before flux remediation fires and uninstalls
|
||||
the Cluster CR.
|
||||
|
||||
The default image lives in images/busybox.tag and points directly at
|
||||
docker.io by digest (not mirrored to ghcr.io like the other .tag files
|
||||
here): the payload is a one-shot sh loop and the digest pin makes the
|
||||
pull immutable. Operators in air-gapped or rate-limited environments
|
||||
can override it via .Values.images.waitForKubeconfig (any registry
|
||||
reference kubelet can pull). When the value is empty the chart falls
|
||||
back to the bundled digest pin, preserving the prior default.
|
||||
|
||||
Call site owns the surrounding volumes block; the kubeconfig volume
|
||||
must exist on the pod and mount at /etc/kubernetes/kubeconfig.
|
||||
*/}}
|
||||
{{- define "kubernetes.waitForAdminKubeconfig" -}}
|
||||
- name: wait-for-kubeconfig
|
||||
image: "{{ default (.Files.Get "images/busybox.tag" | trim) .Values.images.waitForKubeconfig }}"
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -eu
|
||||
deadline=$(( $(date +%s) + 600 ))
|
||||
until [ -s /etc/kubernetes/kubeconfig/super-admin.svc ]; do
|
||||
if [ "$(date +%s)" -ge "$deadline" ]; then
|
||||
echo "admin kubeconfig was not provisioned within 10m; exiting so the pod goes CrashLoopBackOff and surfaces in dashboards" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "waiting for admin kubeconfig (provisioned by Kamaji, visible after kubelet Secret refresh)..."
|
||||
sleep 5
|
||||
done
|
||||
volumeMounts:
|
||||
- name: kubeconfig
|
||||
mountPath: /etc/kubernetes/kubeconfig
|
||||
readOnly: true
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,14 +1,3 @@
|
|||
{{- /*
|
||||
Gate the control-plane-side workloads on the parent tenant having an etcd
|
||||
DataStore. Without it no KamajiControlPlane is ever created, Kamaji never
|
||||
provisions -admin-kubeconfig, and rendering these Deployments would cause
|
||||
the wait-for-kubeconfig init to CrashLoopBackOff indefinitely, consuming
|
||||
the parent HelmRelease install timeout and triggering the very uninstall
|
||||
remediation cycle this chart is supposed to avoid. Rendering them only
|
||||
when $etcd is set keeps the HelmRelease Ready while flux retries on its
|
||||
interval and picks up the DataStore as soon as the Tenant chart finishes.
|
||||
*/}}
|
||||
{{- if .Values._namespace.etcd }}
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
|
|
@ -34,8 +23,6 @@ spec:
|
|||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: "NoSchedule"
|
||||
initContainers:
|
||||
{{- include "kubernetes.waitForAdminKubeconfig" $ | nindent 6 }}
|
||||
containers:
|
||||
- image: "{{ $.Files.Get "images/cluster-autoscaler.tag" | trim }}"
|
||||
name: cluster-autoscaler
|
||||
|
|
@ -69,7 +56,6 @@ spec:
|
|||
name: cloud-config
|
||||
- secret:
|
||||
secretName: {{ .Release.Name }}-admin-kubeconfig
|
||||
optional: true
|
||||
name: kubeconfig
|
||||
serviceAccountName: {{ .Release.Name }}-cluster-autoscaler
|
||||
terminationGracePeriodSeconds: 10
|
||||
|
|
@ -119,4 +105,3 @@ rules:
|
|||
- list
|
||||
- update
|
||||
- watch
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,15 +1,4 @@
|
|||
{{- $etcd := .Values._namespace.etcd }}
|
||||
{{- /*
|
||||
When $etcd is empty, the parent Tenant application has not populated
|
||||
_namespace.etcd in cozystack-values yet - either the operator forgot to
|
||||
set etcd: true on an ancestor Tenant, or the Tenant HelmRelease is still
|
||||
reconciling. Either way, rendering a KamajiControlPlane with an empty
|
||||
dataStoreName would be rejected by Kamaji's admission webhook and the
|
||||
HelmRelease would fail to install, triggering remediation. Instead, emit
|
||||
a single ConfigMap as a user-visible status beacon and skip the rest so
|
||||
flux marks the HelmRelease Ready and retries its 5m reconcile loop until
|
||||
the Tenant chart catches up.
|
||||
*/}}
|
||||
{{- $ingress := .Values._namespace.ingress }}
|
||||
{{- $host := .Values._namespace.host }}
|
||||
{{- $kubevirtmachinetemplateNames := list }}
|
||||
|
|
@ -95,26 +84,6 @@ spec:
|
|||
- name: default
|
||||
pod: {}
|
||||
{{- end }}
|
||||
{{- if not $etcd }}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-awaiting-etcd
|
||||
namespace: {{ .Release.Namespace }}
|
||||
labels:
|
||||
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||
app.kubernetes.io/name: kubernetes
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
data:
|
||||
status: "awaiting-etcd"
|
||||
message: |
|
||||
No DataStore is available for this tenant Kubernetes cluster. The parent
|
||||
Tenant application has not populated _namespace.etcd. Set spec.etcd: true
|
||||
on an ancestor Tenant (usually tenant-root) and wait for its HelmRelease
|
||||
to reconcile - this HelmRelease will pick up the DataStore on its next
|
||||
5m reconcile loop and provision the cluster.
|
||||
{{- else }}
|
||||
---
|
||||
apiVersion: cluster.x-k8s.io/v1beta1
|
||||
kind: Cluster
|
||||
|
|
@ -435,4 +404,3 @@ metadata:
|
|||
spec:
|
||||
{{- .spec | toYaml | nindent 2 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
{{- if .Values._namespace.etcd }}
|
||||
kind: Deployment
|
||||
apiVersion: apps/v1
|
||||
metadata:
|
||||
|
|
@ -25,8 +24,6 @@ spec:
|
|||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: "NoSchedule"
|
||||
initContainers:
|
||||
{{- include "kubernetes.waitForAdminKubeconfig" $ | nindent 6 }}
|
||||
containers:
|
||||
- name: csi-driver
|
||||
imagePullPolicy: Always
|
||||
|
|
@ -237,6 +234,4 @@ spec:
|
|||
emptyDir: {}
|
||||
- secret:
|
||||
secretName: {{ .Release.Name }}-admin-kubeconfig
|
||||
optional: true
|
||||
name: kubeconfig
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{{- if and .Values.addons.certManager.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.certManager.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ cert-manager:
|
|||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{- if and .Values.addons.certManager.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.certManager.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -14,7 +14,6 @@ cilium:
|
|||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{- if .Values._namespace.etcd }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -56,4 +55,3 @@ spec:
|
|||
- name: {{ .Release.Name }}-gateway-api-crds
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ coredns:
|
|||
clusterIP: "10.95.0.10"
|
||||
{{- end }}
|
||||
|
||||
{{- if .Values._namespace.etcd }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -43,4 +42,3 @@ spec:
|
|||
{{- end }}
|
||||
- name: {{ .Release.Name }}-cilium
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
{{- if .Values._namespace.etcd }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -40,4 +39,3 @@ spec:
|
|||
- name: {{ .Release.Name }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{{- if and .Values.addons.fluxcd.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.fluxcd.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{{- if and $.Values.addons.gatewayAPI.enabled $.Values._namespace.etcd }}
|
||||
{{- if $.Values.addons.gatewayAPI.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -1,12 +1,4 @@
|
|||
{{- define "cozystack.defaultGpuOperatorValues" -}}
|
||||
{{- if .Values.addons.hami.enabled }}
|
||||
gpu-operator:
|
||||
devicePlugin:
|
||||
enabled: false
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{- if and .Values.addons.gpuOperator.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.gpuOperator.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -37,12 +29,9 @@ spec:
|
|||
force: true
|
||||
remediation:
|
||||
retries: -1
|
||||
{{- $defaults := fromYaml (include "cozystack.defaultGpuOperatorValues" .) }}
|
||||
{{- $overrides := deepCopy (default (dict) .Values.addons.gpuOperator.valuesOverride) }}
|
||||
{{- $merged := mergeOverwrite (default (dict) $defaults) $overrides }}
|
||||
{{- if $merged }}
|
||||
{{- with .Values.addons.gpuOperator.valuesOverride }}
|
||||
values:
|
||||
{{- toYaml $merged | nindent 4 }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
|
||||
dependsOn:
|
||||
|
|
|
|||
|
|
@ -1,49 +0,0 @@
|
|||
{{- if and .Values.addons.hami.enabled .Values._namespace.etcd }}
|
||||
{{- if not .Values.addons.gpuOperator.enabled }}
|
||||
{{- fail "addons.hami requires addons.gpuOperator to be enabled" }}
|
||||
{{- end }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-hami
|
||||
labels:
|
||||
cozystack.io/repository: system
|
||||
cozystack.io/target-cluster-name: {{ .Release.Name }}
|
||||
sharding.fluxcd.io/key: tenants
|
||||
spec:
|
||||
releaseName: hami
|
||||
chartRef:
|
||||
kind: ExternalArtifact
|
||||
name: cozystack-kubernetes-application-kubevirt-kubernetes-hami
|
||||
namespace: cozy-system
|
||||
kubeConfig:
|
||||
secretRef:
|
||||
name: {{ .Release.Name }}-admin-kubeconfig
|
||||
key: super-admin.svc
|
||||
targetNamespace: cozy-hami
|
||||
storageNamespace: cozy-hami
|
||||
interval: 5m
|
||||
timeout: 10m
|
||||
install:
|
||||
createNamespace: true
|
||||
remediation:
|
||||
retries: -1
|
||||
upgrade:
|
||||
force: true
|
||||
remediation:
|
||||
retries: -1
|
||||
{{- with .Values.addons.hami.valuesOverride }}
|
||||
values:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
|
||||
dependsOn:
|
||||
{{- if lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" .Release.Namespace .Release.Name }}
|
||||
- name: {{ .Release.Name }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
- name: {{ .Release.Name }}-cilium
|
||||
namespace: {{ .Release.Namespace }}
|
||||
- name: {{ .Release.Name }}-gpu-operator
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
|
|
@ -20,7 +20,7 @@ ingress-nginx:
|
|||
node-role.kubernetes.io/ingress-nginx: ""
|
||||
{{- end }}
|
||||
|
||||
{{- if and .Values.addons.ingressNginx.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.ingressNginx.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
{{- if .Values._namespace.etcd }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -37,4 +36,3 @@ spec:
|
|||
namespace: {{ .Release.Namespace }}
|
||||
- name: {{ .Release.Name }}-prometheus-operator-crds
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{{- $targetTenant := .Values._namespace.monitoring }}
|
||||
{{- $clusterDomain := (index .Values._cluster "cluster-domain") | default "cozy.local" }}
|
||||
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.monitoringAgents.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
{{- if .Values._namespace.etcd }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -32,4 +31,3 @@ spec:
|
|||
- name: {{ .Release.Name }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{{- if and .Values.addons.velero.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.velero.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.monitoringAgents.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ vertical-pod-autoscaler:
|
|||
memory: 1600Mi
|
||||
{{- end }}
|
||||
|
||||
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.monitoringAgents.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{{- if and .Values.addons.monitoringAgents.enabled .Values._namespace.etcd }}
|
||||
{{- if .Values.addons.monitoringAgents.enabled }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
{{- if .Values._namespace.etcd }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -34,4 +33,3 @@ spec:
|
|||
- name: {{ .Release.Name }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
{{- if .Values._namespace.etcd }}
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
|
|
@ -23,8 +22,6 @@ spec:
|
|||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: "NoSchedule"
|
||||
initContainers:
|
||||
{{- include "kubernetes.waitForAdminKubeconfig" $ | nindent 6 }}
|
||||
containers:
|
||||
- name: kubevirt-cloud-controller-manager
|
||||
args:
|
||||
|
|
@ -58,7 +55,5 @@ spec:
|
|||
name: cloud-config
|
||||
- secret:
|
||||
secretName: {{ .Release.Name }}-admin-kubeconfig
|
||||
optional: true
|
||||
name: kubeconfig
|
||||
serviceAccountName: {{ .Release.Name }}-kccm
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -1,155 +0,0 @@
|
|||
suite: admin-kubeconfig wait guards
|
||||
|
||||
release:
|
||||
name: test
|
||||
namespace: tenant-root
|
||||
|
||||
values:
|
||||
- values-ci.yaml
|
||||
|
||||
tests:
|
||||
- it: cluster-autoscaler mounts admin-kubeconfig as optional
|
||||
template: templates/cluster-autoscaler/deployment.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.secretName
|
||||
value: test-admin-kubeconfig
|
||||
- equal:
|
||||
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.optional
|
||||
value: true
|
||||
|
||||
- it: cluster-autoscaler waits for admin-kubeconfig via initContainer
|
||||
template: templates/cluster-autoscaler/deployment.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.initContainers[0].name
|
||||
value: wait-for-kubeconfig
|
||||
- contains:
|
||||
path: spec.template.spec.initContainers[0].volumeMounts
|
||||
content:
|
||||
name: kubeconfig
|
||||
mountPath: /etc/kubernetes/kubeconfig
|
||||
readOnly: true
|
||||
|
||||
- it: kccm mounts admin-kubeconfig as optional
|
||||
template: templates/kccm/manager.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.secretName
|
||||
value: test-admin-kubeconfig
|
||||
- equal:
|
||||
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.optional
|
||||
value: true
|
||||
|
||||
- it: kccm waits for admin-kubeconfig via initContainer
|
||||
template: templates/kccm/manager.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.initContainers[0].name
|
||||
value: wait-for-kubeconfig
|
||||
- contains:
|
||||
path: spec.template.spec.initContainers[0].volumeMounts
|
||||
content:
|
||||
name: kubeconfig
|
||||
mountPath: /etc/kubernetes/kubeconfig
|
||||
readOnly: true
|
||||
|
||||
- it: csi controller mounts admin-kubeconfig as optional
|
||||
template: templates/csi/deploy.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.secretName
|
||||
value: test-admin-kubeconfig
|
||||
- equal:
|
||||
path: spec.template.spec.volumes[?(@.name=="kubeconfig")].secret.optional
|
||||
value: true
|
||||
|
||||
- it: csi controller waits for admin-kubeconfig via initContainer
|
||||
template: templates/csi/deploy.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.initContainers[0].name
|
||||
value: wait-for-kubeconfig
|
||||
- contains:
|
||||
path: spec.template.spec.initContainers[0].volumeMounts
|
||||
content:
|
||||
name: kubeconfig
|
||||
mountPath: /etc/kubernetes/kubeconfig
|
||||
readOnly: true
|
||||
|
||||
- it: wait-for-kubeconfig defaults to bundled busybox digest pin
|
||||
template: templates/cluster-autoscaler/deployment.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
asserts:
|
||||
- matchRegex:
|
||||
path: spec.template.spec.initContainers[0].image
|
||||
pattern: '^docker\.io/library/busybox:[^@]+@sha256:[0-9a-f]{64}$'
|
||||
|
||||
- it: wait-for-kubeconfig honours images.waitForKubeconfig override
|
||||
template: templates/cluster-autoscaler/deployment.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Deployment
|
||||
set:
|
||||
images:
|
||||
waitForKubeconfig: "registry.example.test/library/busybox:1.37.0@sha256:0000000000000000000000000000000000000000000000000000000000000000"
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.template.spec.initContainers[0].image
|
||||
value: "registry.example.test/library/busybox:1.37.0@sha256:0000000000000000000000000000000000000000000000000000000000000000"
|
||||
|
||||
- it: cluster.yaml renders and wires dataStoreName when tenant has etcd
|
||||
template: templates/cluster.yaml
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: KamajiControlPlane
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.dataStoreName
|
||||
value: tenant-root
|
||||
|
||||
- it: cluster.yaml skips Cluster resources when tenant has no etcd DataStore
|
||||
# Must NOT fail rendering - the parent Tenant chart populates
|
||||
# _namespace.etcd asynchronously, so rendering failures here would cause
|
||||
# flux install remediation on every cold bootstrap. Instead, emit only a
|
||||
# ConfigMap status beacon so the HelmRelease reports Ready while flux
|
||||
# retries on its interval until the DataStore appears.
|
||||
template: templates/cluster.yaml
|
||||
set:
|
||||
_namespace:
|
||||
etcd: ""
|
||||
monitoring: ""
|
||||
ingress: ""
|
||||
seaweedfs: ""
|
||||
host: ""
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 1
|
||||
- isKind:
|
||||
of: ConfigMap
|
||||
- equal:
|
||||
path: metadata.name
|
||||
value: test-awaiting-etcd
|
||||
- equal:
|
||||
path: data.status
|
||||
value: awaiting-etcd
|
||||
|
|
@ -1,99 +0,0 @@
|
|||
suite: GPU Operator HelmRelease HAMi integration tests
|
||||
templates:
|
||||
- templates/helmreleases/gpu-operator.yaml
|
||||
values:
|
||||
- values-ci.yaml
|
||||
tests:
|
||||
- it: should disable devicePlugin when hami is enabled
|
||||
set:
|
||||
addons:
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.values.gpu-operator.devicePlugin.enabled
|
||||
value: false
|
||||
|
||||
- it: should not have values when hami is disabled and no overrides
|
||||
set:
|
||||
addons:
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
hami:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- notExists:
|
||||
path: spec.values
|
||||
|
||||
- it: should apply hami defaults when valuesOverride key is omitted
|
||||
set:
|
||||
addons:
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
hami:
|
||||
enabled: true
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.values.gpu-operator.devicePlugin.enabled
|
||||
value: false
|
||||
|
||||
- it: should allow user overrides to merge with hami defaults
|
||||
set:
|
||||
addons:
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride:
|
||||
gpu-operator:
|
||||
driver:
|
||||
enabled: false
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.values.gpu-operator.devicePlugin.enabled
|
||||
value: false
|
||||
- equal:
|
||||
path: spec.values.gpu-operator.driver.enabled
|
||||
value: false
|
||||
|
||||
- it: should let user explicitly override devicePlugin.enabled to true with hami enabled
|
||||
set:
|
||||
addons:
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride:
|
||||
gpu-operator:
|
||||
devicePlugin:
|
||||
enabled: true
|
||||
driver:
|
||||
enabled: false
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.values.gpu-operator.devicePlugin.enabled
|
||||
value: true
|
||||
- equal:
|
||||
path: spec.values.gpu-operator.driver.enabled
|
||||
value: false
|
||||
|
||||
- it: should not render when gpuOperator is disabled
|
||||
set:
|
||||
addons:
|
||||
gpuOperator:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
hami:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
|
@ -1,153 +0,0 @@
|
|||
suite: HAMi HelmRelease tests
|
||||
templates:
|
||||
- templates/helmreleases/hami.yaml
|
||||
values:
|
||||
- values-ci.yaml
|
||||
tests:
|
||||
- it: should not render when hami is disabled
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
- it: should render HelmRelease when hami is enabled
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 1
|
||||
- isKind:
|
||||
of: HelmRelease
|
||||
|
||||
- it: should fail when gpuOperator is not enabled
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- failedTemplate:
|
||||
errorMessage: "addons.hami requires addons.gpuOperator to be enabled"
|
||||
|
||||
- it: should have correct metadata labels
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- equal:
|
||||
path: metadata.labels["cozystack.io/repository"]
|
||||
value: system
|
||||
- equal:
|
||||
path: metadata.labels["sharding.fluxcd.io/key"]
|
||||
value: tenants
|
||||
|
||||
- it: should use ExternalArtifact chartRef
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.chartRef.kind
|
||||
value: ExternalArtifact
|
||||
- equal:
|
||||
path: spec.chartRef.namespace
|
||||
value: cozy-system
|
||||
|
||||
- it: should target cozy-hami namespace
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.targetNamespace
|
||||
value: cozy-hami
|
||||
- equal:
|
||||
path: spec.storageNamespace
|
||||
value: cozy-hami
|
||||
|
||||
- it: should depend on gpu-operator and cilium
|
||||
release:
|
||||
name: test
|
||||
namespace: test-ns
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- contains:
|
||||
path: spec.dependsOn
|
||||
content:
|
||||
name: test-cilium
|
||||
namespace: test-ns
|
||||
- contains:
|
||||
path: spec.dependsOn
|
||||
content:
|
||||
name: test-gpu-operator
|
||||
namespace: test-ns
|
||||
|
||||
- it: should not render spec.values when valuesOverride is empty
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 1
|
||||
- notExists:
|
||||
path: spec.values
|
||||
|
||||
- it: should pass through valuesOverride
|
||||
set:
|
||||
addons:
|
||||
hami:
|
||||
enabled: true
|
||||
valuesOverride:
|
||||
hami:
|
||||
devicePlugin:
|
||||
deviceSplitCount: 5
|
||||
gpuOperator:
|
||||
enabled: true
|
||||
valuesOverride: {}
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.values.hami.devicePlugin.deviceSplitCount
|
||||
value: 5
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
_namespace:
|
||||
etcd: ""
|
||||
monitoring: ""
|
||||
ingress: ""
|
||||
seaweedfs: ""
|
||||
host: ""
|
||||
_cluster:
|
||||
cluster-domain: cozy.local
|
||||
nodeGroups: null
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
_namespace:
|
||||
etcd: tenant-root
|
||||
monitoring: ""
|
||||
ingress: ""
|
||||
seaweedfs: ""
|
||||
host: ""
|
||||
_cluster:
|
||||
cluster-domain: cozy.local
|
||||
nodeGroups: null
|
||||
|
|
@ -149,7 +149,6 @@
|
|||
"fluxcd",
|
||||
"gatewayAPI",
|
||||
"gpuOperator",
|
||||
"hami",
|
||||
"ingressNginx",
|
||||
"monitoringAgents",
|
||||
"velero",
|
||||
|
|
@ -269,28 +268,6 @@
|
|||
}
|
||||
}
|
||||
},
|
||||
"hami": {
|
||||
"description": "HAMi GPU virtualization middleware.",
|
||||
"type": "object",
|
||||
"default": {},
|
||||
"required": [
|
||||
"enabled",
|
||||
"valuesOverride"
|
||||
],
|
||||
"properties": {
|
||||
"enabled": {
|
||||
"description": "Enable HAMi (requires GPU Operator).",
|
||||
"type": "boolean",
|
||||
"default": false
|
||||
},
|
||||
"valuesOverride": {
|
||||
"description": "Custom Helm values overrides.",
|
||||
"type": "object",
|
||||
"default": {},
|
||||
"x-kubernetes-preserve-unknown-fields": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"ingressNginx": {
|
||||
"description": "Ingress-NGINX controller.",
|
||||
"type": "object",
|
||||
|
|
@ -653,18 +630,6 @@
|
|||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"images": {
|
||||
"description": "Optional image overrides for air-gapped or rate-limited registries.",
|
||||
"type": "object",
|
||||
"default": {},
|
||||
"properties": {
|
||||
"waitForKubeconfig": {
|
||||
"description": "Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag.",
|
||||
"type": "string",
|
||||
"default": ""
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -94,10 +94,6 @@ host: ""
|
|||
## @field {bool} enabled - Enable FluxCD.
|
||||
## @field {object} valuesOverride - Custom Helm values overrides.
|
||||
|
||||
## @typedef {struct} HAMiAddon - HAMi GPU virtualization middleware.
|
||||
## @field {bool} enabled - Enable HAMi (requires GPU Operator).
|
||||
## @field {object} valuesOverride - Custom Helm values overrides.
|
||||
|
||||
## @typedef {struct} MonitoringAgentsAddon - Monitoring agents (Fluent Bit, VMAgents).
|
||||
## @field {bool} enabled - Enable monitoring agents.
|
||||
## @field {object} valuesOverride - Custom Helm values overrides.
|
||||
|
|
@ -118,7 +114,6 @@ host: ""
|
|||
## @field {GatewayAPIAddon} gatewayAPI - Gateway API addon.
|
||||
## @field {IngressNginxAddon} ingressNginx - Ingress-NGINX controller.
|
||||
## @field {GPUOperatorAddon} gpuOperator - NVIDIA GPU Operator.
|
||||
## @field {HAMiAddon} hami - HAMi GPU virtualization middleware.
|
||||
## @field {FluxCDAddon} fluxcd - FluxCD GitOps operator.
|
||||
## @field {MonitoringAgentsAddon} monitoringAgents - Monitoring agents.
|
||||
## @field {VerticalPodAutoscalerAddon} verticalPodAutoscaler - Vertical Pod Autoscaler.
|
||||
|
|
@ -142,9 +137,6 @@ addons:
|
|||
gpuOperator:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
hami:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
fluxcd:
|
||||
enabled: false
|
||||
valuesOverride: {}
|
||||
|
|
@ -205,10 +197,3 @@ controlPlane:
|
|||
server:
|
||||
resources: {}
|
||||
resourcesPreset: "micro"
|
||||
|
||||
## @typedef {struct} Images - Optional image overrides for chart-internal helpers.
|
||||
## @field {string} [waitForKubeconfig] - Image used by the wait-for-kubeconfig init container. Empty falls back to images/busybox.tag.
|
||||
|
||||
## @param {Images} images - Optional image overrides for air-gapped or rate-limited registries.
|
||||
images:
|
||||
waitForKubeconfig: ""
|
||||
|
|
|
|||
|
|
@ -133,13 +133,12 @@ See:
|
|||
|
||||
### Bootstrap (recovery) parameters
|
||||
|
||||
| Name | Description | Type | Value |
|
||||
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------- |
|
||||
| `bootstrap` | Bootstrap configuration. | `object` | `{}` |
|
||||
| `bootstrap.enabled` | Whether to restore from a backup. | `bool` | `false` |
|
||||
| `bootstrap.recoveryTime` | Timestamp (RFC3339) for point-in-time recovery; empty means latest. | `string` | `""` |
|
||||
| `bootstrap.oldName` | Previous cluster name before deletion. | `string` | `""` |
|
||||
| `bootstrap.serverName` | Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name. | `string` | `""` |
|
||||
| Name | Description | Type | Value |
|
||||
| ------------------------ | ------------------------------------------------------------------- | -------- | ------- |
|
||||
| `bootstrap` | Bootstrap configuration. | `object` | `{}` |
|
||||
| `bootstrap.enabled` | Whether to restore from a backup. | `bool` | `false` |
|
||||
| `bootstrap.recoveryTime` | Timestamp (RFC3339) for point-in-time recovery; empty means latest. | `string` | `""` |
|
||||
| `bootstrap.oldName` | Previous cluster name before deletion. | `string` | `""` |
|
||||
|
||||
|
||||
## Parameter examples and reference
|
||||
|
|
|
|||
|
|
@ -32,9 +32,6 @@ spec:
|
|||
- name: {{ .Values.bootstrap.oldName }}
|
||||
barmanObjectStore:
|
||||
destinationPath: {{ .Values.backup.destinationPath }}
|
||||
{{- if .Values.bootstrap.serverName }}
|
||||
serverName: {{ .Values.bootstrap.serverName }}
|
||||
{{- end }}
|
||||
endpointURL: {{ .Values.backup.endpointURL }}
|
||||
s3Credentials:
|
||||
accessKeyId:
|
||||
|
|
|
|||
|
|
@ -254,11 +254,6 @@
|
|||
"description": "Timestamp (RFC3339) for point-in-time recovery; empty means latest.",
|
||||
"type": "string",
|
||||
"default": ""
|
||||
},
|
||||
"serverName": {
|
||||
"description": "Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name.",
|
||||
"type": "string",
|
||||
"default": ""
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -154,7 +154,6 @@ backup:
|
|||
## @field {bool} enabled - Whether to restore from a backup.
|
||||
## @field {string} [recoveryTime] - Timestamp (RFC3339) for point-in-time recovery; empty means latest.
|
||||
## @field {string} oldName - Previous cluster name before deletion.
|
||||
## @field {string} [serverName] - Barman server name (S3 path prefix) used by the original cluster when writing backups. Set this only when the original cluster had an explicit barmanObjectStore.serverName that differed from its Kubernetes resource name.
|
||||
|
||||
## @param {Bootstrap} bootstrap - Bootstrap configuration.
|
||||
bootstrap:
|
||||
|
|
@ -162,4 +161,3 @@ bootstrap:
|
|||
# example: 2020-11-26 15:22:00.00000+00
|
||||
recoveryTime: ""
|
||||
oldName: ""
|
||||
serverName: ""
|
||||
|
|
|
|||
|
|
@ -3,6 +3,3 @@ include ../../../hack/package.mk
|
|||
generate:
|
||||
cozyvalues-gen -m 'tenant' -v values.yaml -s values.schema.json -r README.md -g ../../../api/apps/v1alpha1/tenant/types.go
|
||||
../../../hack/update-crd.sh
|
||||
|
||||
test:
|
||||
helm unittest .
|
||||
|
|
|
|||
|
|
@ -1,28 +0,0 @@
|
|||
{{- $exposeMode := (index .Values._cluster "expose-mode") | default "externalIPs" }}
|
||||
{{- $exposeIngress := (index .Values._cluster "expose-ingress") | default "tenant-root" }}
|
||||
{{- $exposeIPs := (index .Values._cluster "expose-external-ips") | default "" | nospace }}
|
||||
{{- $ipsList := list }}
|
||||
{{- range splitList "," $exposeIPs }}
|
||||
{{- $ip := . | trim }}
|
||||
{{- if $ip }}
|
||||
{{- $ipsList = append $ipsList $ip }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- $isPublishingIngressLB := and
|
||||
(eq $exposeMode "loadBalancer")
|
||||
(eq $exposeIngress .Release.Namespace)
|
||||
.Values.ingress }}
|
||||
{{- if and $isPublishingIngressLB $ipsList }}
|
||||
apiVersion: cilium.io/v2
|
||||
kind: CiliumLoadBalancerIPPool
|
||||
metadata:
|
||||
name: {{ trimPrefix "tenant-" .Release.Namespace }}-exposure
|
||||
spec:
|
||||
blocks:
|
||||
{{- range $ipsList }}
|
||||
- cidr: {{ . }}{{ if not (contains "/" .) }}/{{ if contains ":" . }}128{{ else }}32{{ end }}{{ end }}
|
||||
{{- end }}
|
||||
serviceSelector:
|
||||
matchLabels:
|
||||
"io.kubernetes.service.namespace": {{ .Release.Namespace | quote }}
|
||||
{{- end }}
|
||||
|
|
@ -1,141 +0,0 @@
|
|||
suite: tenant CiliumLoadBalancerIPPool rendering for publishing.exposure=loadBalancer
|
||||
templates:
|
||||
- templates/cilium-lb-pool.yaml
|
||||
|
||||
release:
|
||||
name: tenant-root
|
||||
namespace: tenant-root
|
||||
|
||||
tests:
|
||||
- it: default exposure (externalIPs) renders no pool
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10,192.0.2.11"
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
- it: loadBalancer mode in publishing tenant with ingress=true renders v2 pool with namespace-only selector
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10,192.0.2.11"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 1
|
||||
- equal:
|
||||
path: apiVersion
|
||||
value: cilium.io/v2
|
||||
- equal:
|
||||
path: kind
|
||||
value: CiliumLoadBalancerIPPool
|
||||
- equal:
|
||||
path: metadata.name
|
||||
value: root-exposure
|
||||
- equal:
|
||||
path: spec.blocks
|
||||
value:
|
||||
- cidr: 192.0.2.10/32
|
||||
- cidr: 192.0.2.11/32
|
||||
- equal:
|
||||
path: spec.serviceSelector.matchLabels["io.kubernetes.service.namespace"]
|
||||
value: tenant-root
|
||||
- notExists:
|
||||
path: spec.serviceSelector.matchLabels["app.kubernetes.io/name"]
|
||||
|
||||
- it: loadBalancer mode with IPv6 emits /128 CIDR
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "2001:db8::1"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.blocks
|
||||
value:
|
||||
- cidr: 2001:db8::1/128
|
||||
|
||||
- it: loadBalancer mode with mixed IPv4 and IPv6 emits correct CIDR per family
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10,2001:db8::1"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.blocks
|
||||
value:
|
||||
- cidr: 192.0.2.10/32
|
||||
- cidr: 2001:db8::1/128
|
||||
|
||||
- it: loadBalancer mode accepts pre-CIDR input without double-suffixing
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10/32,2001:db8::1/128"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.blocks
|
||||
value:
|
||||
- cidr: 192.0.2.10/32
|
||||
- cidr: 2001:db8::1/128
|
||||
|
||||
- it: loadBalancer mode filters out empty entries from externalIPs (trailing, leading, repeated commas)
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10,,192.0.2.11,"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 1
|
||||
- equal:
|
||||
path: spec.blocks
|
||||
value:
|
||||
- cidr: 192.0.2.10/32
|
||||
- cidr: 192.0.2.11/32
|
||||
|
||||
- it: loadBalancer mode with ingress=false in publishing tenant renders no pool
|
||||
set:
|
||||
ingress: false
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
- it: loadBalancer mode in a non-publishing tenant renders no pool
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10"
|
||||
expose-mode: loadBalancer
|
||||
release:
|
||||
name: tenant-u1
|
||||
namespace: tenant-u1
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
- it: loadBalancer mode in publishing tenant with empty externalIPs renders no pool
|
||||
set:
|
||||
ingress: true
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: ""
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
|
@ -36,32 +36,31 @@ virtctl ssh <user>@<vm>
|
|||
|
||||
### Common parameters
|
||||
|
||||
| Name | Description | Type | Value |
|
||||
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----------- |
|
||||
| `external` | Enable external access from outside the cluster. | `bool` | `false` |
|
||||
| `externalMethod` | Method to pass through traffic to the VM. | `string` | `PortList` |
|
||||
| `externalPorts` | Ports to forward from outside the cluster. | `[]int` | `[22]` |
|
||||
| `externalAllowICMP` | Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect. | `bool` | `true` |
|
||||
| `runStrategy` | Requested running state of the VirtualMachineInstance | `string` | `Always` |
|
||||
| `instanceType` | Virtual Machine instance type. | `string` | `u1.medium` |
|
||||
| `instanceProfile` | Virtual Machine preferences profile. | `string` | `ubuntu` |
|
||||
| `disks` | List of disks to attach. | `[]object` | `[]` |
|
||||
| `disks[i].name` | Disk name. | `string` | `""` |
|
||||
| `disks[i].bus` | Disk bus type (e.g. "sata"). | `string` | `""` |
|
||||
| `networks` | Networks to attach the VM to. | `[]object` | `[]` |
|
||||
| `networks[i].name` | Network attachment name. | `string` | `""` |
|
||||
| `subnets` | Deprecated: use networks instead. | `[]object` | `[]` |
|
||||
| `subnets[i].name` | Network attachment name. | `string` | `""` |
|
||||
| `gpus` | List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM). | `[]object` | `[]` |
|
||||
| `gpus[i].name` | The name of the GPU resource to attach. | `string` | `""` |
|
||||
| `cpuModel` | Model specifies the CPU model inside the VMI. List of available models https://github.com/libvirt/libvirt/tree/master/src/cpu_map | `string` | `""` |
|
||||
| `resources` | Resource configuration for the virtual machine. | `object` | `{}` |
|
||||
| `resources.cpu` | Number of CPU cores allocated. | `quantity` | `""` |
|
||||
| `resources.memory` | Amount of memory allocated. | `quantity` | `""` |
|
||||
| `resources.sockets` | Number of CPU sockets (vCPU topology). | `quantity` | `""` |
|
||||
| `sshKeys` | List of SSH public keys for authentication. | `[]string` | `[]` |
|
||||
| `cloudInit` | Cloud-init user data. | `string` | `""` |
|
||||
| `cloudInitSeed` | Seed string to generate SMBIOS UUID for the VM. | `string` | `""` |
|
||||
| Name | Description | Type | Value |
|
||||
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----------- |
|
||||
| `external` | Enable external access from outside the cluster. | `bool` | `false` |
|
||||
| `externalMethod` | Method to pass through traffic to the VM. | `string` | `PortList` |
|
||||
| `externalPorts` | Ports to forward from outside the cluster. | `[]int` | `[22]` |
|
||||
| `runStrategy` | Requested running state of the VirtualMachineInstance | `string` | `Always` |
|
||||
| `instanceType` | Virtual Machine instance type. | `string` | `u1.medium` |
|
||||
| `instanceProfile` | Virtual Machine preferences profile. | `string` | `ubuntu` |
|
||||
| `disks` | List of disks to attach. | `[]object` | `[]` |
|
||||
| `disks[i].name` | Disk name. | `string` | `""` |
|
||||
| `disks[i].bus` | Disk bus type (e.g. "sata"). | `string` | `""` |
|
||||
| `networks` | Networks to attach the VM to. | `[]object` | `[]` |
|
||||
| `networks[i].name` | Network attachment name. | `string` | `""` |
|
||||
| `subnets` | Deprecated: use networks instead. | `[]object` | `[]` |
|
||||
| `subnets[i].name` | Network attachment name. | `string` | `""` |
|
||||
| `gpus` | List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM). | `[]object` | `[]` |
|
||||
| `gpus[i].name` | The name of the GPU resource to attach. | `string` | `""` |
|
||||
| `cpuModel` | Model specifies the CPU model inside the VMI. List of available models https://github.com/libvirt/libvirt/tree/master/src/cpu_map | `string` | `""` |
|
||||
| `resources` | Resource configuration for the virtual machine. | `object` | `{}` |
|
||||
| `resources.cpu` | Number of CPU cores allocated. | `quantity` | `""` |
|
||||
| `resources.memory` | Amount of memory allocated. | `quantity` | `""` |
|
||||
| `resources.sockets` | Number of CPU sockets (vCPU topology). | `quantity` | `""` |
|
||||
| `sshKeys` | List of SSH public keys for authentication. | `[]string` | `[]` |
|
||||
| `cloudInit` | Cloud-init user data. | `string` | `""` |
|
||||
| `cloudInitSeed` | Seed string to generate SMBIOS UUID for the VM. | `string` | `""` |
|
||||
|
||||
|
||||
## U Series
|
||||
|
|
|
|||
|
|
@ -9,10 +9,7 @@ metadata:
|
|||
{{- if .Values.external }}
|
||||
service.kubernetes.io/service-proxy-name: "cozy-proxy"
|
||||
annotations:
|
||||
networking.cozystack.io/wholeIP: {{ ternary "true" "false" (eq .Values.externalMethod "WholeIP") | quote }}
|
||||
{{- if eq .Values.externalMethod "PortList" }}
|
||||
networking.cozystack.io/allowICMP: {{ ternary "true" "false" (ne .Values.externalAllowICMP false) | quote }}
|
||||
{{- end }}
|
||||
networking.cozystack.io/wholeIP: "true"
|
||||
{{- end }}
|
||||
spec:
|
||||
type: {{ ternary "LoadBalancer" "ClusterIP" .Values.external }}
|
||||
|
|
|
|||
|
|
@ -26,11 +26,6 @@
|
|||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"externalAllowICMP": {
|
||||
"description": "Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect.",
|
||||
"type": "boolean",
|
||||
"default": true
|
||||
},
|
||||
"runStrategy": {
|
||||
"description": "Requested running state of the VirtualMachineInstance",
|
||||
"type": "string",
|
||||
|
|
|
|||
|
|
@ -31,9 +31,6 @@ externalMethod: PortList
|
|||
externalPorts:
|
||||
- 22
|
||||
|
||||
## @param {bool} externalAllowICMP - Whether to accept ICMP traffic to the VM in PortList mode (preserves ping and PMTU discovery). No effect in WholeIP mode. Default true so ping behaves as users expect even when port filtering is in effect.
|
||||
externalAllowICMP: true
|
||||
|
||||
## @enum {string} RunStrategy - Requested running state of the VirtualMachineInstance
|
||||
## @value Always - VMI should always be running
|
||||
## @value Halted - VMI should never be running
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
cozystackOperator:
|
||||
# Deployment variant: talos, generic, hosted
|
||||
variant: talos
|
||||
image: ghcr.io/cozystack/cozystack/cozystack-operator:v1.3.0@sha256:62574f12486bb40c901cf5ed484cca264405ce5810196d86555cbb27cce1ba48
|
||||
image: ghcr.io/cozystack/cozystack/cozystack-operator:v1.3.1@sha256:6cf2c5eb432193b5fa8ab202cbc56c8cd7971436639ceefa5be99aede74b3061
|
||||
platformSourceUrl: 'oci://ghcr.io/cozystack/cozystack/cozystack-packages'
|
||||
platformSourceRef: 'digest=sha256:a0b9ef938446b3132d3d22ad2262beb1027c48c9037b6c2346fdc2f19acd3036'
|
||||
platformSourceRef: 'digest=sha256:7b0d95638195317f0d84f33e0cbd77542229c9df42de41ee22e977987043631f'
|
||||
# Generic variant configuration (only used when cozystackOperator.variant=generic)
|
||||
cozystack:
|
||||
# Kubernetes API server host (IP only, no protocol/port)
|
||||
|
|
|
|||
|
|
@ -1,24 +0,0 @@
|
|||
---
|
||||
apiVersion: cozystack.io/v1alpha1
|
||||
kind: PackageSource
|
||||
metadata:
|
||||
name: cozystack.hami
|
||||
spec:
|
||||
sourceRef:
|
||||
kind: OCIRepository
|
||||
name: cozystack-packages
|
||||
namespace: cozy-system
|
||||
path: /
|
||||
variants:
|
||||
- name: default
|
||||
dependsOn:
|
||||
- cozystack.gpu-operator
|
||||
components:
|
||||
- name: hami
|
||||
path: system/hami
|
||||
valuesFiles:
|
||||
- values.yaml
|
||||
install:
|
||||
privileged: true
|
||||
namespace: cozy-hami
|
||||
releaseName: hami
|
||||
|
|
@ -52,8 +52,6 @@ spec:
|
|||
path: system/cilium
|
||||
- name: kubernetes-gpu-operator
|
||||
path: system/gpu-operator
|
||||
- name: kubernetes-hami
|
||||
path: system/hami
|
||||
- name: kubernetes-vertical-pod-autoscaler
|
||||
path: system/vertical-pod-autoscaler
|
||||
- name: kubernetes-prometheus-operator-crds
|
||||
|
|
|
|||
|
|
@ -30,7 +30,6 @@ stringData:
|
|||
expose-services: {{ .Values.publishing.exposedServices | join "," | quote }}
|
||||
expose-ingress: {{ .Values.publishing.ingressName | quote }}
|
||||
expose-external-ips: {{ .Values.publishing.externalIPs | join "," | quote }}
|
||||
expose-mode: {{ .Values.publishing.exposure | default "externalIPs" | quote }}
|
||||
cluster-domain: {{ .Values.networking.clusterDomain | quote }}
|
||||
api-server-endpoint: {{ .Values.publishing.apiServerEndpoint | quote }}
|
||||
{{- with .Values.branding }}
|
||||
|
|
|
|||
|
|
@ -10,7 +10,6 @@
|
|||
{{include "cozystack.platform.package.default" (list "cozystack.kubevirt-cdi" $) }}
|
||||
{{include "cozystack.platform.package.optional.default" (list "cozystack.vm-default-images" $) }}
|
||||
{{include "cozystack.platform.package.optional.default" (list "cozystack.gpu-operator" $) }}
|
||||
{{include "cozystack.platform.package.optional.default" (list "cozystack.hami" $) }}
|
||||
{{include "cozystack.platform.package.default" (list "cozystack.kamaji" $) }}
|
||||
{{include "cozystack.platform.package.default" (list "cozystack.capi-operator" $) }}
|
||||
{{include "cozystack.platform.package.default" (list "cozystack.capi-provider-bootstrap-kubeadm" $) }}
|
||||
|
|
|
|||
|
|
@ -105,6 +105,9 @@
|
|||
{{- /* cozystack-api DaemonSet */ -}}
|
||||
{{- $apiValues := dict "cozystackAPI" (dict "nodeSelector" $genericNodeSelector) -}}
|
||||
{{- $_ := set $cozystackEngineComponents "cozystack-api" (dict "values" $apiValues) -}}
|
||||
{{- /* lineage-controller-webhook DaemonSet */ -}}
|
||||
{{- $lineageValues := dict "lineageControllerWebhook" (dict "nodeSelector" $genericNodeSelector) -}}
|
||||
{{- $_ := set $cozystackEngineComponents "lineage-controller-webhook" (dict "values" $lineageValues) -}}
|
||||
{{- end -}}
|
||||
{{- if .Values.authentication.oidc.enabled }}
|
||||
{{include "cozystack.platform.package" (list "cozystack.cozystack-engine" "oidc" $ $cozystackEngineComponents) }}
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ sourceRef:
|
|||
path: /
|
||||
migrations:
|
||||
enabled: false
|
||||
image: ghcr.io/cozystack/cozystack/platform-migrations:v1.3.0@sha256:555e4b76421361805a84bc9088b01b23a9c4a9430bd8ebd2db82ef9677d7008c
|
||||
image: ghcr.io/cozystack/cozystack/platform-migrations:v1.3.1@sha256:555e4b76421361805a84bc9088b01b23a9c4a9430bd8ebd2db82ef9677d7008c
|
||||
targetVersion: 39
|
||||
# Bundle deployment configuration
|
||||
bundles:
|
||||
|
|
@ -45,41 +45,6 @@ publishing:
|
|||
- cdi-uploadproxy
|
||||
apiServerEndpoint: "" # example: "https://api.example.org"
|
||||
externalIPs: []
|
||||
# Exposure mode for the ingress-nginx Service. When "externalIPs" (current
|
||||
# default) is selected, the Service is created as ClusterIP with
|
||||
# Service.spec.externalIPs set from publishing.externalIPs. When
|
||||
# "loadBalancer" is selected, the Service is type: LoadBalancer and a
|
||||
# CiliumLoadBalancerIPPool makes those same addresses allocatable via LB IPAM.
|
||||
#
|
||||
# Service.spec.externalIPs is deprecated upstream in Kubernetes v1.36
|
||||
# (KEP-5707). The AllowServiceExternalIPs feature gate is expected to default
|
||||
# to false around v1.40 and the implementation removed around v1.43 — switch
|
||||
# to "loadBalancer" before upgrading past v1.40.
|
||||
#
|
||||
# Caveats for the "loadBalancer" mode:
|
||||
# - publishing.externalIPs must contain at least one non-empty address,
|
||||
# otherwise the chart render fails with an explicit error (a LoadBalancer
|
||||
# Service without a pool would sit in <pending> forever).
|
||||
# - The ingress-nginx Service is created with externalTrafficPolicy: Local
|
||||
# to preserve the client source IP. Traffic arriving on a node that does
|
||||
# not host an ingress-nginx pod is dropped, so the external IP must be
|
||||
# routed to a node that runs the ingress pod (floating IP / keepalived /
|
||||
# upstream router / podAntiAffinity).
|
||||
# - Cilium does NOT announce the IP on its own unless L2 announcements or
|
||||
# BGP are enabled in the Cilium values (disabled by default in Cozystack).
|
||||
# This mode assumes the operator already routes the externalIPs to a
|
||||
# cluster node; enabling announcements is out of scope for this setting.
|
||||
# - Switching this value on a running cluster causes the ingress-nginx
|
||||
# Service to be recreated (the HelmRelease has upgrade.force: true and
|
||||
# the Service kind changes between ClusterIP and LoadBalancer). Expect a
|
||||
# brief interruption of ingress traffic during the flip.
|
||||
#
|
||||
# Scope: this setting only controls the ingress-nginx Service. Other
|
||||
# cozystack components that currently write Service.spec.externalIPs directly
|
||||
# (e.g. the vpn app at packages/apps/vpn/templates/service.yaml) are NOT
|
||||
# migrated by flipping this value and must be addressed separately before
|
||||
# the AllowServiceExternalIPs feature gate flips to off in ~v1.40.
|
||||
exposure: externalIPs # "externalIPs" or "loadBalancer"
|
||||
certificates:
|
||||
solver: http01 # "http01" or "dns01"
|
||||
issuerName: letsencrypt-prod
|
||||
|
|
|
|||
|
|
@ -1,2 +1,2 @@
|
|||
e2e:
|
||||
image: ghcr.io/cozystack/cozystack/e2e-sandbox:v1.3.0@sha256:0367a03b981df2a3ea13f411d4cb7869c2bf2c89c07d3d5c8971b9a28921ccef
|
||||
image: ghcr.io/cozystack/cozystack/e2e-sandbox:v1.3.1@sha256:0367a03b981df2a3ea13f411d4cb7869c2bf2c89c07d3d5c8971b9a28921ccef
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
ghcr.io/cozystack/cozystack/matchbox:v1.3.0@sha256:85b8e04bf6f0690612dd63e80475df269f4a436d16680f8a40f2860cf16e2f74
|
||||
ghcr.io/cozystack/cozystack/matchbox:v1.3.1@sha256:f61425c515e7c8b49d634cef4f050faeeaa8c455ffbbe77eee669a04dc1ed305
|
||||
|
|
|
|||
|
|
@ -10,6 +10,3 @@ get-cloudflare-ips:
|
|||
generate:
|
||||
cozyvalues-gen -v values.yaml -s values.schema.json -r README.md
|
||||
../../../hack/update-crd.sh
|
||||
|
||||
test:
|
||||
helm unittest .
|
||||
|
|
|
|||
|
|
@ -14,17 +14,3 @@
|
|||
| `resources.memory` | Memory (RAM) available to each replica. | `quantity` | `""` |
|
||||
| `resourcesPreset` | Default sizing preset used when `resources` is omitted. | `string` | `micro` |
|
||||
|
||||
|
||||
## Exposure mode
|
||||
|
||||
The ingress Service type is driven by the cluster-wide `publishing.exposure` value in the platform chart, not by any key in this package. Two modes exist:
|
||||
|
||||
- `externalIPs` (default) has three rendered shapes:
|
||||
- Release namespace matches `publishing.ingressName` AND `publishing.externalIPs` is non-empty → Service is `ClusterIP` with `Service.spec.externalIPs` set from that list and `externalTrafficPolicy: Cluster`.
|
||||
- Release namespace matches `publishing.ingressName` but `publishing.externalIPs` is empty → Service falls back to `type: LoadBalancer` with `externalTrafficPolicy: Local`.
|
||||
- Release namespace does not match `publishing.ingressName` (non-root tenants) → Service is `type: LoadBalancer` with `externalTrafficPolicy: Local`.
|
||||
`Service.spec.externalIPs` is deprecated upstream in Kubernetes v1.36 (KEP-5707); plan migration before v1.40.
|
||||
- `loadBalancer` — Service is `type: LoadBalancer` with `externalTrafficPolicy: Local`, and a `CiliumLoadBalancerIPPool` makes the addresses in `publishing.externalIPs` allocatable via Cilium LB IPAM. Requires `publishing.externalIPs` to contain at least one non-empty address (render fails otherwise) and assumes the addresses are already routed to a cluster node (floating IP / upstream router). See the inline comment on `publishing.exposure` in the platform chart for full caveats, including the note that switching the value on a running cluster causes the ingress Service to be recreated.
|
||||
|
||||
This setting only migrates ingress-nginx away from `Service.spec.externalIPs`. Other cozystack components that use the same deprecated field (e.g. the `vpn` app) must be migrated separately before Kubernetes v1.40 flips the `AllowServiceExternalIPs` feature gate off.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,19 +1,5 @@
|
|||
{{- $exposeIngress := (index .Values._cluster "expose-ingress") | default "tenant-root" }}
|
||||
{{- $exposeExternalIPs := (index .Values._cluster "expose-external-ips") | default "" | nospace }}
|
||||
{{- $exposeMode := (index .Values._cluster "expose-mode") | default "externalIPs" }}
|
||||
{{- $exposeIPsList := list }}
|
||||
{{- range splitList "," $exposeExternalIPs }}
|
||||
{{- $ip := . | trim }}
|
||||
{{- if $ip }}
|
||||
{{- $exposeIPsList = append $exposeIPsList $ip }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if not (has $exposeMode (list "externalIPs" "loadBalancer")) }}
|
||||
{{- fail (printf "unknown publishing.exposure mode %q: must be \"externalIPs\" or \"loadBalancer\"" $exposeMode) }}
|
||||
{{- end }}
|
||||
{{- if and (eq $exposeMode "loadBalancer") (eq $exposeIngress .Release.Namespace) (not $exposeIPsList) }}
|
||||
{{- fail "publishing.exposure=loadBalancer requires publishing.externalIPs to contain at least one non-empty address: the CiliumLoadBalancerIPPool has nothing to advertise and the Service would stay in <pending>." }}
|
||||
{{- end }}
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
|
|
@ -55,12 +41,9 @@ spec:
|
|||
enabled: false
|
||||
{{- end }}
|
||||
service:
|
||||
{{- if and (eq $exposeIngress .Release.Namespace) (eq $exposeMode "loadBalancer") }}
|
||||
type: LoadBalancer
|
||||
externalTrafficPolicy: Local
|
||||
{{- else if and (eq $exposeIngress .Release.Namespace) $exposeIPsList }}
|
||||
{{- if and (eq $exposeIngress .Release.Namespace) $exposeExternalIPs }}
|
||||
externalIPs:
|
||||
{{- toYaml $exposeIPsList | nindent 12 }}
|
||||
{{- toYaml (splitList "," $exposeExternalIPs) | nindent 12 }}
|
||||
type: ClusterIP
|
||||
externalTrafficPolicy: Cluster
|
||||
{{- else }}
|
||||
|
|
|
|||
|
|
@ -1,165 +0,0 @@
|
|||
suite: ingress exposure modes
|
||||
templates:
|
||||
- templates/nginx-ingress.yaml
|
||||
|
||||
release:
|
||||
name: ingress
|
||||
namespace: tenant-root
|
||||
|
||||
tests:
|
||||
- it: default exposure (externalIPs) renders ClusterIP Service with spec.externalIPs
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10,192.0.2.11"
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.type
|
||||
value: ClusterIP
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.externalTrafficPolicy
|
||||
value: Cluster
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.externalIPs
|
||||
value:
|
||||
- 192.0.2.10
|
||||
- 192.0.2.11
|
||||
- template: templates/nginx-ingress.yaml
|
||||
notExists:
|
||||
path: spec.values.ingress-nginx.controller.service.labels
|
||||
|
||||
- it: legacy config without expose-mode falls back to externalIPs behavior
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10"
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.type
|
||||
value: ClusterIP
|
||||
|
||||
- it: externalIPs mode in a namespace other than publishing.ingressName renders LoadBalancer fallback without externalIPs
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10"
|
||||
release:
|
||||
namespace: tenant-other
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.type
|
||||
value: LoadBalancer
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.externalTrafficPolicy
|
||||
value: Local
|
||||
- template: templates/nginx-ingress.yaml
|
||||
notExists:
|
||||
path: spec.values.ingress-nginx.controller.service.externalIPs
|
||||
|
||||
- it: loadBalancer mode renders LoadBalancer Service without externalIPs on the Service
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10,192.0.2.11"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.type
|
||||
value: LoadBalancer
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.externalTrafficPolicy
|
||||
value: Local
|
||||
- template: templates/nginx-ingress.yaml
|
||||
notExists:
|
||||
path: spec.values.ingress-nginx.controller.service.labels
|
||||
- template: templates/nginx-ingress.yaml
|
||||
notExists:
|
||||
path: spec.values.ingress-nginx.controller.service.externalIPs
|
||||
|
||||
- it: loadBalancer mode without externalIPs fails chart render with explicit message
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: ""
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
failedTemplate:
|
||||
errorMessage: "publishing.exposure=loadBalancer requires publishing.externalIPs to contain at least one non-empty address: the CiliumLoadBalancerIPPool has nothing to advertise and the Service would stay in <pending>."
|
||||
|
||||
- it: unknown exposure mode is rejected with a clear error (case-sensitive enum)
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10"
|
||||
expose-mode: LoadBalancer
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
failedTemplate:
|
||||
errorMessage: "unknown publishing.exposure mode \"LoadBalancer\": must be \"externalIPs\" or \"loadBalancer\""
|
||||
|
||||
- it: another typo in exposure mode also fails
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10"
|
||||
expose-mode: loadbalancer
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
failedTemplate:
|
||||
errorMessage: "unknown publishing.exposure mode \"loadbalancer\": must be \"externalIPs\" or \"loadBalancer\""
|
||||
|
||||
- it: loadBalancer mode in a namespace other than publishing.ingressName falls back to LoadBalancer Service without pool
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10"
|
||||
expose-mode: loadBalancer
|
||||
release:
|
||||
namespace: tenant-other
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.type
|
||||
value: LoadBalancer
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.externalTrafficPolicy
|
||||
value: Local
|
||||
- template: templates/nginx-ingress.yaml
|
||||
notExists:
|
||||
path: spec.values.ingress-nginx.controller.service.labels
|
||||
- template: templates/nginx-ingress.yaml
|
||||
notExists:
|
||||
path: spec.values.ingress-nginx.controller.service.externalIPs
|
||||
|
||||
- it: loadBalancer mode with only-empty externalIPs fails chart render (comma-only input)
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: ",,"
|
||||
expose-mode: loadBalancer
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
failedTemplate:
|
||||
errorMessage: "publishing.exposure=loadBalancer requires publishing.externalIPs to contain at least one non-empty address: the CiliumLoadBalancerIPPool has nothing to advertise and the Service would stay in <pending>."
|
||||
|
||||
- it: externalIPs mode also filters out empty entries (trailing comma)
|
||||
set:
|
||||
_cluster:
|
||||
expose-ingress: tenant-root
|
||||
expose-external-ips: "192.0.2.10,"
|
||||
asserts:
|
||||
- template: templates/nginx-ingress.yaml
|
||||
equal:
|
||||
path: spec.values.ingress-nginx.controller.service.externalIPs
|
||||
value:
|
||||
- 192.0.2.10
|
||||
|
|
@ -1 +1 @@
|
|||
ghcr.io/cozystack/cozystack/objectstorage-sidecar:v1.3.0@sha256:5bcccbdb13979a16cee535eb5fbcdf0d87973689010a10b7b25e55c5be3edaa6
|
||||
ghcr.io/cozystack/cozystack/objectstorage-sidecar:v1.3.1@sha256:3895676b15cdf186b03f231dcfdfe40812ecd76ab00c3b4e2952daccbacd57f3
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
backupController:
|
||||
image: "ghcr.io/cozystack/cozystack/backup-controller:v1.3.0@sha256:e1a083dc92f26dfef004f47c1cd20a6357174aad835004f58e751c494b76649a"
|
||||
image: "ghcr.io/cozystack/cozystack/backup-controller:v1.3.1@sha256:b0ed6db5cf738b774de62a2af51c9800c23dd6e977e95728d3b0098d18c8927e"
|
||||
replicas: 2
|
||||
debug: false
|
||||
metrics:
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
backupStrategyController:
|
||||
image: "ghcr.io/cozystack/cozystack/backupstrategy-controller:v1.3.0@sha256:be0a9ec1f4307064b16388a24628aee46e06252738338add80b99ea1e04e62bf"
|
||||
image: "ghcr.io/cozystack/cozystack/backupstrategy-controller:v1.3.1@sha256:8b0500a68c4dd027b6499d5309ccebddc56bb29f070722538cc306827e71da93"
|
||||
replicas: 2
|
||||
debug: false
|
||||
metrics:
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
ghcr.io/cozystack/cozystack/s3manager:v0.5.0@sha256:fb65e734bbdbdaef2238769cc2ecfb54dbddaf1e0952ad438a9b3e26b9dbb4b5
|
||||
ghcr.io/cozystack/cozystack/s3manager:v0.5.0@sha256:7473f0acd04d2e14393b4fc3638fe9e0d3dc15a5778b05dac9dfe1a1957d2647
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ annotations:
|
|||
Cilium Gateway Class Config\n description: |\n CiliumGatewayClassConfig defines
|
||||
a configuration for Gateway API GatewayClass.\n"
|
||||
apiVersion: v2
|
||||
appVersion: 1.19.3
|
||||
appVersion: 1.19.1
|
||||
description: eBPF-based Networking, Security, and Observability
|
||||
home: https://cilium.io/
|
||||
icon: https://cdn.jsdelivr.net/gh/cilium/cilium@main/Documentation/images/logo-solo.svg
|
||||
|
|
@ -92,4 +92,4 @@ kubeVersion: '>= 1.21.0-0'
|
|||
name: cilium
|
||||
sources:
|
||||
- https://github.com/cilium/cilium
|
||||
version: 1.19.3
|
||||
version: 1.19.1
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# cilium
|
||||
|
||||
 
|
||||
 
|
||||
|
||||
Cilium is open source software for providing and transparently securing
|
||||
network connectivity and loadbalancing between application workloads such as
|
||||
|
|
@ -89,7 +89,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| authentication.mutual.spire.install.agent.tolerations | list | `[{"effect":"NoSchedule","key":"node.kubernetes.io/not-ready"},{"effect":"NoSchedule","key":"node-role.kubernetes.io/master"},{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane"},{"effect":"NoSchedule","key":"node.cloudprovider.kubernetes.io/uninitialized","value":"true"},{"key":"CriticalAddonsOnly","operator":"Exists"}]` | SPIRE agent tolerations configuration By default it follows the same tolerations as the agent itself to allow the Cilium agent on this node to connect to SPIRE. ref: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ |
|
||||
| authentication.mutual.spire.install.enabled | bool | `true` | Enable SPIRE installation. This will only take effect only if authentication.mutual.spire.enabled is true |
|
||||
| authentication.mutual.spire.install.existingNamespace | bool | `false` | SPIRE namespace already exists. Set to true if Helm should not create, manage, and import the SPIRE namespace. |
|
||||
| authentication.mutual.spire.install.initImage | object | `{"digest":"sha256:1487d0af5f52b4ba31c7e465126ee2123fe3f2305d638e7827681e7cf6c83d5e","override":null,"pullPolicy":"IfNotPresent","repository":"docker.io/library/busybox","tag":"1.37.0","useDigest":true}` | init container image of SPIRE agent and server |
|
||||
| authentication.mutual.spire.install.initImage | object | `{"digest":"sha256:b3255e7dfbcd10cb367af0d409747d511aeb66dfac98cf30e97e87e4207dd76f","override":null,"pullPolicy":"IfNotPresent","repository":"docker.io/library/busybox","tag":"1.37.0","useDigest":true}` | init container image of SPIRE agent and server |
|
||||
| authentication.mutual.spire.install.namespace | string | `"cilium-spire"` | SPIRE namespace to install into |
|
||||
| authentication.mutual.spire.install.server.affinity | object | `{}` | SPIRE server affinity configuration |
|
||||
| authentication.mutual.spire.install.server.annotations | object | `{}` | SPIRE server annotations |
|
||||
|
|
@ -175,7 +175,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| bpf.tproxy | bool | `false` | Configure the eBPF-based TPROXY (beta) to reduce reliance on iptables rules for implementing Layer 7 policy. Note this is incompatible with netkit (`bpf.datapathMode=netkit`, `bpf.datapathMode=netkit-l2`). |
|
||||
| bpf.vlanBypass | list | `[]` | Configure explicitly allowed VLAN id's for bpf logic bypass. [0] will allow all VLAN id's without any filtering. |
|
||||
| bpfClockProbe | bool | `false` | Enable BPF clock source probing for more efficient tick retrieval. |
|
||||
| certgen | object | `{"affinity":{},"annotations":{"cronJob":{},"job":{}},"cronJob":{"failedJobsHistoryLimit":1,"successfulJobsHistoryLimit":3},"extraVolumeMounts":[],"extraVolumes":[],"generateCA":true,"image":{"digest":"sha256:f0c656830e856d26b24b0e144df1f8b327d3b46748d76a630514111fc365b697","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/certgen","tag":"v0.4.1","useDigest":true},"nodeSelector":{},"podLabels":{},"priorityClassName":"","resources":{},"tolerations":[],"ttlSecondsAfterFinished":null}` | Configure certificate generation for Hubble integration. If hubble.tls.auto.method=cronJob, these values are used for the Kubernetes CronJob which will be scheduled regularly to (re)generate any certificates not provided manually. |
|
||||
| certgen | object | `{"affinity":{},"annotations":{"cronJob":{},"job":{}},"cronJob":{"failedJobsHistoryLimit":1,"successfulJobsHistoryLimit":3},"extraVolumeMounts":[],"extraVolumes":[],"generateCA":true,"image":{"digest":"sha256:19921f48ee7e2295ea4dca955878a6cd8d70e6d4219d08f688e866ece9d95d4d","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/certgen","tag":"v0.3.2","useDigest":true},"nodeSelector":{},"podLabels":{},"priorityClassName":"","resources":{},"tolerations":[],"ttlSecondsAfterFinished":null}` | Configure certificate generation for Hubble integration. If hubble.tls.auto.method=cronJob, these values are used for the Kubernetes CronJob which will be scheduled regularly to (re)generate any certificates not provided manually. |
|
||||
| certgen.affinity | object | `{}` | Affinity for certgen |
|
||||
| certgen.annotations | object | `{"cronJob":{},"job":{}}` | Annotations to be added to the hubble-certgen initial Job and CronJob |
|
||||
| certgen.cronJob.failedJobsHistoryLimit | int | `1` | The number of failed finished jobs to keep |
|
||||
|
|
@ -214,7 +214,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| clustermesh.apiserver.extraVolumeMounts | list | `[]` | Additional clustermesh-apiserver volumeMounts. |
|
||||
| clustermesh.apiserver.extraVolumes | list | `[]` | Additional clustermesh-apiserver volumes. |
|
||||
| clustermesh.apiserver.healthPort | int | `9880` | TCP port for the clustermesh-apiserver health API. |
|
||||
| clustermesh.apiserver.image | object | `{"digest":"sha256:a8136a7615d6c6041d3aa6f2674d17beaec238170d669507ccc05328a778e2b7","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/clustermesh-apiserver","tag":"v1.19.3","useDigest":true}` | Clustermesh API server image. |
|
||||
| clustermesh.apiserver.image | object | `{"digest":"sha256:56d6c3dc13b50126b80ecb571707a0ea97f6db694182b9d61efd386d04e5bb28","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/clustermesh-apiserver","tag":"v1.19.1","useDigest":true}` | Clustermesh API server image. |
|
||||
| clustermesh.apiserver.kvstoremesh.enabled | bool | `true` | Enable KVStoreMesh. KVStoreMesh caches the information retrieved from the remote clusters in the local etcd instance (deprecated - KVStoreMesh will always be enabled once the option is removed). |
|
||||
| clustermesh.apiserver.kvstoremesh.extraArgs | list | `[]` | Additional KVStoreMesh arguments. |
|
||||
| clustermesh.apiserver.kvstoremesh.extraEnv | list | `[]` | Additional KVStoreMesh environment variables. |
|
||||
|
|
@ -340,10 +340,6 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| cni.resources | object | `{"limits":{"cpu":1,"memory":"1Gi"},"requests":{"cpu":"100m","memory":"10Mi"}}` | Specifies the resources for the cni initContainer |
|
||||
| cni.uninstall | bool | `false` | Remove the CNI configuration and binary files on agent shutdown. Enable this if you're removing Cilium from the cluster. Disable this to prevent the CNI configuration file from being removed during agent upgrade, which can cause nodes to go unmanageable. |
|
||||
| commonLabels | object | `{}` | commonLabels allows users to add common labels for all Cilium resources. |
|
||||
| configDriftDetection | object | `{"driftChecker":true,"enabled":true,"ignoredKeys":[]}` | Configuration for the ConfigMap drift detection feature. When enabled, the agent continuously watches the cilium-config ConfigMap and exposes a cilium_drift_checker_config_delta Prometheus metric reporting the number of keys that differ between the ConfigMap and the agent's active settings. A non-zero value indicates that the agent has not yet applied all current ConfigMap changes and needs to be restarted. |
|
||||
| configDriftDetection.driftChecker | bool | `true` | Enable the drift checker which compares the DynamicConfig table against the agent's active settings and publishes the cilium_drift_checker_config_delta metric. |
|
||||
| configDriftDetection.enabled | bool | `true` | Enable watching of the cilium-config ConfigMap and reflecting its contents into the agent's internal DynamicConfig table. |
|
||||
| configDriftDetection.ignoredKeys | list | `[]` | List of config-map keys to ignore when computing the drift delta. |
|
||||
| connectivityProbeFrequencyRatio | float64 | `0.5` | Ratio of the connectivity probe frequency vs resource usage, a float in [0, 1]. 0 will give more frequent probing, 1 will give less frequent probing. Probing frequency is dynamically adjusted based on the cluster size. |
|
||||
| conntrackGCInterval | string | `"0s"` | Configure how frequently garbage collection should occur for the datapath connection tracking table. |
|
||||
| conntrackGCMaxInterval | string | `""` | Configure the maximum frequency for the garbage collection of the connection tracking table. Only affects the automatic computation for the frequency and has no effect when 'conntrackGCInterval' is set. This can be set to more frequently clean up unused identities created from ToFQDN policies. |
|
||||
|
|
@ -384,6 +380,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| enableMasqueradeRouteSource | bool | `false` | Enables masquerading to the source of the route for traffic leaving the node from endpoints. |
|
||||
| enableNoServiceEndpointsRoutable | bool | `true` | Enable routing to a service that has zero endpoints |
|
||||
| enableNonDefaultDenyPolicies | bool | `true` | Enable Non-Default-Deny policies |
|
||||
| enableTunnelBIGTCP | bool | `false` | Enable BIG TCP in tunneling mode and increase maximum GRO/GSO limits for VXLAN/GENEVE tunnels |
|
||||
| enableXTSocketFallback | bool | `true` | Enables the fallback compatibility solution for when the xt_socket kernel module is missing and it is needed for the datapath L7 redirection to work properly. See documentation for details on when this can be disabled: https://docs.cilium.io/en/stable/operations/system_requirements/#linux-kernel. |
|
||||
| encryption.enabled | bool | `false` | Enable transparent network encryption. |
|
||||
| encryption.ipsec.encryptedOverlay | bool | `false` | Enable IPsec encrypted overlay |
|
||||
|
|
@ -404,29 +401,8 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| encryption.strictMode.ingress.enabled | bool | `false` | Enable strict ingress encryption. When enabled, all unencrypted overlay ingress traffic will be dropped. This option is only applicable when WireGuard and tunneling are enabled. |
|
||||
| encryption.type | string | `"ipsec"` | Encryption method. Can be one of ipsec, wireguard or ztunnel. |
|
||||
| encryption.wireguard.persistentKeepalive | string | `"0s"` | Controls WireGuard PersistentKeepalive option. Set 0s to disable. |
|
||||
| encryption.ztunnel | object | `{"affinity":{},"annotations":{},"caAddress":"https://localhost:15012","extraEnv":[],"extraVolumeMounts":[],"extraVolumes":[],"healthPort":15021,"image":{"digest":null,"override":null,"pullPolicy":"IfNotPresent","repository":"docker.io/istio/ztunnel","tag":"1.28.0-distroless","useDigest":false},"nodeSelector":{"kubernetes.io/os":"linux"},"podAnnotations":{},"podLabels":{},"priorityClassName":null,"readinessProbe":{"failureThreshold":3,"initialDelaySeconds":0,"periodSeconds":10},"resources":{"requests":{"cpu":"200m","memory":"512Mi"}},"secrets":{"bootstrapRootCert":null},"terminationGracePeriodSeconds":30,"tolerations":[{"effect":"NoSchedule","operator":"Exists"},{"key":"CriticalAddonsOnly","operator":"Exists"},{"effect":"NoExecute","operator":"Exists"}],"updateStrategy":{"rollingUpdate":{"maxSurge":1,"maxUnavailable":0},"type":"RollingUpdate"}}` | ztunnel encryption configuration. ztunnel is Istio's purpose-built, per-node proxy for handling L4 traffic in ambient mesh mode. These settings only apply when encryption.type is set to "ztunnel". |
|
||||
| encryption.ztunnel.affinity | object | `{}` | Affinity for ztunnel pods. |
|
||||
| encryption.ztunnel.annotations | object | `{}` | Annotations to be added to all ztunnel resources. |
|
||||
| encryption.ztunnel.caAddress | string | `"https://localhost:15012"` | CA server address for certificate requests. |
|
||||
| encryption.ztunnel.extraEnv | list | `[]` | Additional ztunnel container environment variables. |
|
||||
| encryption.ztunnel.extraVolumeMounts | list | `[]` | Additional ztunnel volumeMounts. |
|
||||
| encryption.ztunnel.extraVolumes | list | `[]` | Additional ztunnel volumes. |
|
||||
| encryption.ztunnel.healthPort | int | `15021` | TCP port for the health API. |
|
||||
| encryption.ztunnel.image | object | `{"digest":null,"override":null,"pullPolicy":"IfNotPresent","repository":"docker.io/istio/ztunnel","tag":"1.28.0-distroless","useDigest":false}` | ztunnel container image. |
|
||||
| encryption.ztunnel.nodeSelector | object | `{"kubernetes.io/os":"linux"}` | Node selector for ztunnel pods. |
|
||||
| encryption.ztunnel.podAnnotations | object | `{}` | Annotations to be added to ztunnel pods. |
|
||||
| encryption.ztunnel.podLabels | object | `{}` | Labels to be added to ztunnel pods. |
|
||||
| encryption.ztunnel.priorityClassName | string | `nil` | The priority class to use for ztunnel pods. |
|
||||
| encryption.ztunnel.readinessProbe | object | `{"failureThreshold":3,"initialDelaySeconds":0,"periodSeconds":10}` | Readiness probe configuration. |
|
||||
| encryption.ztunnel.resources | object | `{"requests":{"cpu":"200m","memory":"512Mi"}}` | ztunnel resource limits & requests. |
|
||||
| encryption.ztunnel.secrets | object | `{"bootstrapRootCert":null}` | ztunnel secrets configuration. |
|
||||
| encryption.ztunnel.secrets.bootstrapRootCert | string | `nil` | Base64-encoded bootstrap root certificate content. If not provided, the secret must be created manually before deploying. @schema type: [null, string] @schema |
|
||||
| encryption.ztunnel.terminationGracePeriodSeconds | int | `30` | Configure termination grace period for ztunnel DaemonSet. |
|
||||
| encryption.ztunnel.tolerations | list | `[{"effect":"NoSchedule","operator":"Exists"},{"key":"CriticalAddonsOnly","operator":"Exists"},{"effect":"NoExecute","operator":"Exists"}]` | Node tolerations for ztunnel scheduling. |
|
||||
| encryption.ztunnel.updateStrategy | object | `{"rollingUpdate":{"maxSurge":1,"maxUnavailable":0},"type":"RollingUpdate"}` | ztunnel update strategy. |
|
||||
| endpointHealthChecking.enabled | bool | `true` | Enable connectivity health checking between virtual endpoints. |
|
||||
| endpointLockdownOnMapOverflow | bool | `false` | Enable endpoint lockdown on policy map overflow. |
|
||||
| endpointPolicyUpdateTimeoutDuration | string | `nil` | Max duration to wait for envoy to respond to configuration changes. Default "10s". |
|
||||
| endpointRoutes.enabled | bool | `false` | Enable use of per endpoint routes instead of routing via the cilium_host interface. |
|
||||
| eni.awsEnablePrefixDelegation | bool | `false` | Enable ENI prefix delegation |
|
||||
| eni.awsReleaseExcessIPs | bool | `false` | Release IPs not used from the ENI |
|
||||
|
|
@ -470,7 +446,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| envoy.httpRetryCount | int | `3` | Maximum number of retries for each HTTP request |
|
||||
| envoy.httpUpstreamLingerTimeout | string | `nil` | Time in seconds to block Envoy worker thread while an upstream HTTP connection is closing. If set to 0, the connection is closed immediately (with TCP RST). If set to -1, the connection is closed asynchronously in the background. |
|
||||
| envoy.idleTimeoutDurationSeconds | int | `60` | Set Envoy upstream HTTP idle connection timeout seconds. Does not apply to connections with pending requests. Default 60s |
|
||||
| envoy.image | object | `{"digest":"sha256:ba0ab8adac082d50d525fd2c5ba096c8facea3a471561b7c61c7a5b9c2e0de0d","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium-envoy","tag":"v1.36.6-1776000132-2437d2edeaf4d9b56ef279bd0d71127440c067aa","useDigest":true}` | Envoy container image. |
|
||||
| envoy.image | object | `{"digest":"sha256:8188114a2768b5f49d6ce58e168b20d765e0fbc64eee0d83241aa2b150ccd788","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium-envoy","tag":"v1.35.9-1770979049-232ed4a26881e4ab4f766f251f258ed424fff663","useDigest":true}` | Envoy container image. |
|
||||
| envoy.initContainers | list | `[]` | Init containers added to the cilium Envoy DaemonSet. |
|
||||
| envoy.initialFetchTimeoutSeconds | int | `30` | Time in seconds after which the initial fetch on an xDS stream is considered timed out |
|
||||
| envoy.livenessProbe.enabled | bool | `true` | Enable liveness probe for cilium-envoy |
|
||||
|
|
@ -615,7 +591,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| hubble.relay.extraVolumes | list | `[]` | Additional hubble-relay volumes. |
|
||||
| hubble.relay.gops.enabled | bool | `true` | Enable gops for hubble-relay |
|
||||
| hubble.relay.gops.port | int | `9893` | Configure gops listen port for hubble-relay |
|
||||
| hubble.relay.image | object | `{"digest":"sha256:5ee21d57b6ef2aa6db67e603a735fdceb162454b352b7335b651456e308f681b","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/hubble-relay","tag":"v1.19.3","useDigest":true}` | Hubble-relay container image. |
|
||||
| hubble.relay.image | object | `{"digest":"sha256:d8c4e13bc36a56179292bb52bc6255379cb94cb873700d316ea3139b1bdb8165","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/hubble-relay","tag":"v1.19.1","useDigest":true}` | Hubble-relay container image. |
|
||||
| hubble.relay.listenHost | string | `""` | Host to listen to. Specify an empty string to bind to all the interfaces. |
|
||||
| hubble.relay.listenPort | string | `"4245"` | Port to listen to. |
|
||||
| hubble.relay.logOptions | object | `{"format":null,"level":null}` | Logging configuration for hubble-relay. |
|
||||
|
|
@ -733,7 +709,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| identityAllocationMode | string | `"crd"` | Method to use for identity allocation (`crd`, `kvstore` or `doublewrite-readkvstore` / `doublewrite-readcrd` for migrating between identity backends). |
|
||||
| identityChangeGracePeriod | string | `"5s"` | Time to wait before using new identity on endpoint identity change. |
|
||||
| identityManagementMode | string | `"agent"` | Control whether CiliumIdentities are created by the agent ("agent"), the operator ("operator") or both ("both"). "Both" should be used only to migrate between "agent" and "operator". Operator-managed identities is a beta feature. |
|
||||
| image | object | `{"digest":"sha256:2e61680593cddca8b6c055f6d4c849d87a26a1c91c7e3b8b56c7fb76ab7b7b10","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium","tag":"v1.19.3","useDigest":true}` | Agent container image. |
|
||||
| image | object | `{"digest":"sha256:41f1f74a0000de8656f1de4088ea00c8f2d49d6edea579034c73c5fd5fe01792","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium","tag":"v1.19.1","useDigest":true}` | Agent container image. |
|
||||
| imagePullSecrets | list | `[]` | Configure image pull secrets for pulling container images |
|
||||
| ingressController.default | bool | `false` | Set cilium ingress controller to be the default ingress controller This will let cilium ingress controller route entries without ingress class set |
|
||||
| ingressController.defaultSecretName | string | `nil` | Default secret name for ingresses without .spec.tls[].secretName set. |
|
||||
|
|
@ -821,13 +797,12 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| livenessProbe.failureThreshold | int | `10` | failure threshold of liveness probe |
|
||||
| livenessProbe.periodSeconds | int | `30` | interval between checks of the liveness probe |
|
||||
| livenessProbe.requireK8sConnectivity | bool | `false` | whether to require k8s connectivity as part of the check. |
|
||||
| loadBalancer | object | `{"acceleration":"disabled","l7":{"algorithm":"round_robin","backend":"disabled","ports":[]},"serviceTopology":false}` | Configure service load balancing |
|
||||
| loadBalancer | object | `{"acceleration":"disabled","l7":{"algorithm":"round_robin","backend":"disabled","ports":[]}}` | Configure service load balancing |
|
||||
| loadBalancer.acceleration | string | `"disabled"` | acceleration is the option to accelerate service handling via XDP Applicable values can be: disabled (do not use XDP), native (XDP BPF program is run directly out of the networking driver's early receive path), or best-effort (use native mode XDP acceleration on devices that support it). |
|
||||
| loadBalancer.l7 | object | `{"algorithm":"round_robin","backend":"disabled","ports":[]}` | L7 LoadBalancer |
|
||||
| loadBalancer.l7.algorithm | string | `"round_robin"` | Default LB algorithm The default LB algorithm to be used for services, which can be overridden by the service annotation (e.g. service.cilium.io/lb-l7-algorithm) Applicable values: round_robin, least_request, random |
|
||||
| loadBalancer.l7.backend | string | `"disabled"` | Enable L7 service load balancing via envoy proxy. The request to a k8s service, which has specific annotation e.g. service.cilium.io/lb-l7, will be forwarded to the local backend proxy to be load balanced to the service endpoints. Please refer to docs for supported annotations for more configuration. Applicable values: - envoy: Enable L7 load balancing via envoy proxy. This will automatically set enable-envoy-config as well. - disabled: Disable L7 load balancing by way of service annotation. |
|
||||
| loadBalancer.l7.ports | list | `[]` | List of ports from service to be automatically redirected to above backend. Any service exposing one of these ports will be automatically redirected. Fine-grained control can be achieved by using the service annotation. |
|
||||
| loadBalancer.serviceTopology | bool | `false` | serviceTopology enables K8s Topology Aware Hints -based service endpoints filtering |
|
||||
| localRedirectPolicies.addressMatcherCIDRs | string | `nil` | Limit the allowed addresses in Address Matcher rule of Local Redirect Policies to the given CIDRs. @schema@ type: [null, array] @schema@ |
|
||||
| localRedirectPolicies.enabled | bool | `false` | Enable local redirect policies. |
|
||||
| localRedirectPolicy | bool | `false` | Enable Local Redirect Policy (deprecated, please use 'localRedirectPolicies.enabled' instead) |
|
||||
|
|
@ -885,7 +860,7 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| operator.hostNetwork | bool | `true` | HostNetwork setting |
|
||||
| operator.identityGCInterval | string | `"15m0s"` | Interval for identity garbage collection. |
|
||||
| operator.identityHeartbeatTimeout | string | `"30m0s"` | Timeout for identity heartbeats. |
|
||||
| operator.image | object | `{"alibabacloudDigest":"sha256:176321a65123373ff8c7823b25183102cbad98375e8d6c80b96d68b6e8491103","awsDigest":"sha256:a53dcbfb77282bf2ddd3abbe60f6d49762e7c1389a36cb35b71d504644a56640","azureDigest":"sha256:699c1571a3df1a98882ee13610d47cffb7b34ee7e8d276096db798a5f6c7e4cb","genericDigest":"sha256:205b09b0ed6accbf9fe688d312a9f0fcfc6a316fc081c23fbffb472af5dd62cd","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/operator","suffix":"","tag":"v1.19.3","useDigest":true}` | cilium-operator image. |
|
||||
| operator.image | object | `{"alibabacloudDigest":"sha256:837b12f4239e88ea5b4b5708ab982c319a94ee05edaecaafe5fd0e5b1962f554","awsDigest":"sha256:18913d05a6c4d205f0b7126c4723bb9ccbd4dc24403da46ed0f9f4bf2a142804","azureDigest":"sha256:82bce78603056e709d4c4e9f9ebb25c222c36d8a07f8c05381c2372d9078eca8","genericDigest":"sha256:e7278d763e448bf6c184b0682cf98cdca078d58a27e1b2f3c906792670aa211a","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/operator","suffix":"","tag":"v1.19.1","useDigest":true}` | cilium-operator image. |
|
||||
| operator.nodeGCInterval | string | `"5m0s"` | Interval for cilium node garbage collection. |
|
||||
| operator.nodeSelector | object | `{"kubernetes.io/os":"linux"}` | Node labels for cilium-operator pod assignment ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector |
|
||||
| operator.podAnnotations | object | `{}` | Annotations to be added to cilium-operator pods |
|
||||
|
|
@ -943,11 +918,11 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| preflight.affinity | object | `{"podAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":[{"labelSelector":{"matchLabels":{"k8s-app":"cilium"}},"topologyKey":"kubernetes.io/hostname"}]}}` | Affinity for cilium-preflight |
|
||||
| preflight.annotations | object | `{}` | Annotations to be added to all top-level preflight objects (resources under templates/cilium-preflight) |
|
||||
| preflight.enabled | bool | `false` | Enable Cilium pre-flight resources (required for upgrade) |
|
||||
| preflight.envoy.image | object | `{"digest":"sha256:ba0ab8adac082d50d525fd2c5ba096c8facea3a471561b7c61c7a5b9c2e0de0d","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium-envoy","tag":"v1.36.6-1776000132-2437d2edeaf4d9b56ef279bd0d71127440c067aa","useDigest":true}` | Envoy pre-flight image. |
|
||||
| preflight.envoy.image | object | `{"digest":"sha256:8188114a2768b5f49d6ce58e168b20d765e0fbc64eee0d83241aa2b150ccd788","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium-envoy","tag":"v1.35.9-1770979049-232ed4a26881e4ab4f766f251f258ed424fff663","useDigest":true}` | Envoy pre-flight image. |
|
||||
| preflight.extraEnv | list | `[]` | Additional preflight environment variables. |
|
||||
| preflight.extraVolumeMounts | list | `[]` | Additional preflight volumeMounts. |
|
||||
| preflight.extraVolumes | list | `[]` | Additional preflight volumes. |
|
||||
| preflight.image | object | `{"digest":"sha256:2e61680593cddca8b6c055f6d4c849d87a26a1c91c7e3b8b56c7fb76ab7b7b10","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium","tag":"v1.19.3","useDigest":true}` | Cilium pre-flight image. |
|
||||
| preflight.image | object | `{"digest":"sha256:41f1f74a0000de8656f1de4088ea00c8f2d49d6edea579034c73c5fd5fe01792","override":null,"pullPolicy":"IfNotPresent","repository":"quay.io/cilium/cilium","tag":"v1.19.1","useDigest":true}` | Cilium pre-flight image. |
|
||||
| preflight.nodeSelector | object | `{"kubernetes.io/os":"linux"}` | Node labels for preflight pod assignment ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector |
|
||||
| preflight.podAnnotations | object | `{}` | Annotations to be added to preflight pods |
|
||||
| preflight.podDisruptionBudget.enabled | bool | `false` | enable PodDisruptionBudget ref: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/ |
|
||||
|
|
@ -1004,7 +979,6 @@ contributors across the globe, there is almost always someone available to help.
|
|||
| serviceAccounts.corednsMCSAPI | object | `{"annotations":{},"automount":true,"create":true,"name":"cilium-coredns-mcsapi-autoconfig"}` | CorednsMCSAPI is used if clustermesh.mcsapi.corednsAutoConfigure.enabled=true |
|
||||
| serviceAccounts.hubblecertgen | object | `{"annotations":{},"automount":true,"create":true,"name":"hubble-generate-certs"}` | Hubblecertgen is used if hubble.tls.auto.method=cronJob |
|
||||
| serviceAccounts.nodeinit.enabled | bool | `false` | Enabled is temporary until https://github.com/cilium/cilium-cli/issues/1396 is implemented. Cilium CLI doesn't create the SAs for node-init, thus the workaround. Helm is not affected by this issue. Name and automount can be configured, if enabled is set to true. Otherwise, they are ignored. Enabled can be removed once the issue is fixed. Cilium-nodeinit DS must also be fixed. |
|
||||
| serviceAccounts.ztunnel | object | `{"annotations":{},"automount":false,"create":true,"name":"ztunnel-cilium"}` | Ztunnel is used if encryption.type=ztunnel |
|
||||
| serviceNoBackendResponse | string | `"reject"` | Configure what the response should be to traffic for a service without backends. Possible values: - reject (default) - drop |
|
||||
| sleepAfterInit | bool | `false` | Do not run Cilium agent when running with clean mode. Useful to completely uninstall Cilium as it will stop Cilium from starting and create artifacts in the node. |
|
||||
| socketLB | object | `{"enabled":false}` | Configure socket LB |
|
||||
|
|
|
|||
|
|
@ -167,8 +167,6 @@ staticResources:
|
|||
circuitBreakers:
|
||||
thresholds:
|
||||
- maxRetries: {{ .Values.envoy.maxConcurrentRetries }}
|
||||
maxConnections: {{ .Values.envoy.clusterMaxConnections }}
|
||||
maxRequests: {{ .Values.envoy.clusterMaxRequests }}
|
||||
lbPolicy: "CLUSTER_PROVIDED"
|
||||
typedExtensionProtocolOptions:
|
||||
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
|
||||
|
|
@ -185,8 +183,6 @@ staticResources:
|
|||
circuitBreakers:
|
||||
thresholds:
|
||||
- maxRetries: {{ .Values.envoy.maxConcurrentRetries }}
|
||||
maxConnections: {{ .Values.envoy.clusterMaxConnections }}
|
||||
maxRequests: {{ .Values.envoy.clusterMaxRequests }}
|
||||
lbPolicy: "CLUSTER_PROVIDED"
|
||||
typedExtensionProtocolOptions:
|
||||
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
|
||||
|
|
@ -208,8 +204,6 @@ staticResources:
|
|||
circuitBreakers:
|
||||
thresholds:
|
||||
- maxRetries: {{ .Values.envoy.maxConcurrentRetries }}
|
||||
maxConnections: {{ .Values.envoy.clusterMaxConnections }}
|
||||
maxRequests: {{ .Values.envoy.clusterMaxRequests }}
|
||||
lbPolicy: "CLUSTER_PROVIDED"
|
||||
typedExtensionProtocolOptions:
|
||||
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
|
||||
|
|
@ -226,8 +220,6 @@ staticResources:
|
|||
circuitBreakers:
|
||||
thresholds:
|
||||
- maxRetries: {{ .Values.envoy.maxConcurrentRetries }}
|
||||
maxConnections: {{ .Values.envoy.clusterMaxConnections }}
|
||||
maxRequests: {{ .Values.envoy.clusterMaxRequests }}
|
||||
lbPolicy: "CLUSTER_PROVIDED"
|
||||
typedExtensionProtocolOptions:
|
||||
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
|
||||
|
|
@ -312,4 +304,3 @@ admin:
|
|||
address:
|
||||
pipe:
|
||||
path: "/var/run/cilium/envoy/sockets/admin.sock"
|
||||
mode: 0660
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue