Compare commits
5 commits
main
...
feat/updat
| Author | SHA1 | Date | |
|---|---|---|---|
|
5a61a33b85 | ||
|
|
dbe37e7ed4 | ||
|
|
6a45f0e65d | ||
|
|
8c86693408 | ||
|
|
9ee5f5d2f4 |
33 changed files with 1482 additions and 391 deletions
172
docs/gpu-vgpu.md
Normal file
172
docs/gpu-vgpu.md
Normal file
|
|
@ -0,0 +1,172 @@
|
|||
# GPU Operator: vGPU Support
|
||||
|
||||
This document describes how to configure the GPU Operator package with NVIDIA vGPU support for sharing a single physical GPU across multiple virtual machines using mediated devices.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- NVIDIA GPU with vGPU support (e.g., NVIDIA L40S, A100, A30, etc.)
|
||||
- Talos Linux as the host OS
|
||||
- NVIDIA vGPU Software license (NVIDIA AI Enterprise or vGPU subscription)
|
||||
- Access to the NVIDIA Licensing Portal ([ui.licensing.nvidia.com](https://ui.licensing.nvidia.com))
|
||||
|
||||
## Variants
|
||||
|
||||
The GPU Operator package supports two variants:
|
||||
|
||||
- **`default`** — GPU passthrough mode (vfio-pci). The entire GPU is passed through to a single VM.
|
||||
- **`vgpu`** — vGPU mode. A physical GPU is shared between multiple VMs using NVIDIA mediated devices.
|
||||
|
||||
## Building the vGPU Manager Image
|
||||
|
||||
The vGPU Manager driver is proprietary and must be obtained from NVIDIA. The GPU Operator expects a pre-built driver container image — it does not install the driver from a raw `.run` file at runtime.
|
||||
|
||||
1. Log in to the [NVIDIA Licensing Portal](https://ui.licensing.nvidia.com)
|
||||
2. Navigate to **Software Downloads** and download the NVIDIA vGPU Software package for your GPU
|
||||
3. Build the driver container image using NVIDIA's Makefile-based build system:
|
||||
|
||||
```bash
|
||||
# Clone the NVIDIA driver container repository
|
||||
git clone https://gitlab.com/nvidia/container-images/driver.git
|
||||
cd driver
|
||||
|
||||
# Place the downloaded .run file in the appropriate directory
|
||||
cp NVIDIA-Linux-x86_64-550.90.05-vgpu-kvm.run vgpu/
|
||||
|
||||
# Build using the provided Makefile
|
||||
make OS_TAG=ubuntu22.04 \
|
||||
VGPU_DRIVER_VERSION=550.90.05 \
|
||||
PRIVATE_REGISTRY=registry.example.com/nvidia
|
||||
|
||||
# Push to your private registry
|
||||
docker push registry.example.com/nvidia/vgpu-manager:550.90.05
|
||||
```
|
||||
|
||||
> **Important:** The build process compiles kernel modules against the host kernel version. Refer to the [NVIDIA GPU Operator vGPU documentation](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/install-gpu-operator-vgpu.html) for the complete build procedure and supported OS/kernel combinations.
|
||||
>
|
||||
> Uploading the vGPU driver to a publicly available registry is a violation of the NVIDIA vGPU EULA.
|
||||
|
||||
## Deploying with vGPU Variant
|
||||
|
||||
Create a Package CR with the `vgpu` variant and provide your vGPU Manager image coordinates:
|
||||
|
||||
```yaml
|
||||
apiVersion: cozystack.io/v1alpha1
|
||||
kind: Package
|
||||
metadata:
|
||||
name: cozystack.gpu-operator
|
||||
spec:
|
||||
variant: vgpu
|
||||
components:
|
||||
gpu-operator:
|
||||
values:
|
||||
gpu-operator:
|
||||
vgpuManager:
|
||||
repository: registry.example.com/nvidia
|
||||
version: "550.90.05"
|
||||
```
|
||||
|
||||
If your registry requires authentication, create an `imagePullSecret` in the `cozy-gpu-operator` namespace and reference it:
|
||||
|
||||
```yaml
|
||||
gpu-operator:
|
||||
vgpuManager:
|
||||
repository: registry.example.com/nvidia
|
||||
version: "550.90.05"
|
||||
imagePullSecrets:
|
||||
- name: nvidia-registry-secret
|
||||
```
|
||||
|
||||
## NVIDIA License Server (NLS) Configuration
|
||||
|
||||
vGPU requires a license server. Configure NLS by passing the license server address via a Secret:
|
||||
|
||||
1. Create a Secret with the NLS client configuration in the `cozy-gpu-operator` namespace:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: licensing-config
|
||||
namespace: cozy-gpu-operator
|
||||
stringData:
|
||||
gridd.conf: |
|
||||
ServerAddress=nls.example.com
|
||||
ServerPort=443
|
||||
FeatureType=1 # 1 for vGPU (vPC/vWS), 2 for Virtual Compute Server (vCS)
|
||||
# ServerPort depends on your NLS deployment (commonly 443 for DLS or 7070 for legacy NLS)
|
||||
```
|
||||
|
||||
2. Reference the Secret in the Package values:
|
||||
|
||||
```yaml
|
||||
gpu-operator:
|
||||
vgpuManager:
|
||||
repository: registry.example.com/nvidia
|
||||
version: "550.90.05"
|
||||
driver:
|
||||
licensingConfig:
|
||||
secretName: licensing-config
|
||||
```
|
||||
|
||||
## vGPU Profiles
|
||||
|
||||
Each GPU model supports specific vGPU profiles that determine how the GPU is partitioned. To list available profiles for your GPU, consult the [NVIDIA vGPU User Guide](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/).
|
||||
|
||||
Example profiles for NVIDIA L40S:
|
||||
|
||||
| Profile | Frame Buffer | Max Instances | Use Case |
|
||||
| --- | --- | --- | --- |
|
||||
| NVIDIA L40S-1Q | 1 GB | 48 | Light 3D/VDI |
|
||||
| NVIDIA L40S-2Q | 2 GB | 24 | Medium 3D/VDI |
|
||||
| NVIDIA L40S-4Q | 4 GB | 12 | Heavy 3D/VDI |
|
||||
| NVIDIA L40S-6Q | 6 GB | 8 | Professional 3D |
|
||||
| NVIDIA L40S-8Q | 8 GB | 6 | AI/ML inference |
|
||||
| NVIDIA L40S-12Q | 12 GB | 4 | AI/ML training |
|
||||
| NVIDIA L40S-24Q | 24 GB | 2 | Large AI workloads |
|
||||
| NVIDIA L40S-48Q | 48 GB | 1 | Full GPU equivalent |
|
||||
|
||||
Custom vGPU device configuration can be provided via a ConfigMap:
|
||||
|
||||
```yaml
|
||||
gpu-operator:
|
||||
vgpuDeviceManager:
|
||||
enabled: true
|
||||
config:
|
||||
name: vgpu-devices-config
|
||||
default: default
|
||||
```
|
||||
|
||||
## KubeVirt Integration
|
||||
|
||||
To use vGPU with KubeVirt VMs, configure `mediatedDeviceTypes` in the KubeVirt CR. This maps vGPU profiles to node selectors:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubevirt.io/v1
|
||||
kind: KubeVirt
|
||||
metadata:
|
||||
name: kubevirt
|
||||
spec:
|
||||
configuration:
|
||||
mediatedDevicesConfiguration:
|
||||
mediatedDeviceTypes:
|
||||
- nvidia-592 # NVIDIA L40S-24Q
|
||||
permittedHostDevices:
|
||||
mediatedDevices:
|
||||
- mdevNameSelector: NVIDIA L40S-24Q
|
||||
resourceName: nvidia.com/NVIDIA_L40S-24Q
|
||||
```
|
||||
|
||||
Then reference the vGPU resource in a VirtualMachine spec:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubevirt.io/v1
|
||||
kind: VirtualMachine
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
domain:
|
||||
devices:
|
||||
gpus:
|
||||
- name: gpu1
|
||||
deviceName: nvidia.com/NVIDIA_L40S-24Q
|
||||
```
|
||||
|
|
@ -23,3 +23,16 @@ spec:
|
|||
privileged: true
|
||||
namespace: cozy-gpu-operator
|
||||
releaseName: gpu-operator
|
||||
- name: vgpu
|
||||
dependsOn:
|
||||
- cozystack.networking
|
||||
components:
|
||||
- name: gpu-operator
|
||||
path: system/gpu-operator
|
||||
valuesFiles:
|
||||
- values.yaml
|
||||
- values-talos-vgpu.yaml
|
||||
install:
|
||||
privileged: true
|
||||
namespace: cozy-gpu-operator
|
||||
releaseName: gpu-operator
|
||||
|
|
|
|||
|
|
@ -8,4 +8,4 @@ update:
|
|||
rm -rf charts
|
||||
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
|
||||
helm repo update nvidia
|
||||
helm pull nvidia/gpu-operator --untar --untardir charts
|
||||
helm pull nvidia/gpu-operator --untar --untardir charts --version v26.3.1
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
dependencies:
|
||||
- name: node-feature-discovery
|
||||
repository: https://kubernetes-sigs.github.io/node-feature-discovery/charts
|
||||
version: 0.17.2
|
||||
digest: sha256:4c55d30d958027ef8997a2976449326de3c90049025c3ebb9bee017cad32cc3f
|
||||
generated: "2025-02-25T09:08:49.128088-08:00"
|
||||
repository: oci://registry.k8s.io/nfd/charts
|
||||
version: 0.18.3
|
||||
digest: sha256:1bce3b80da1e6d47a0befbaa6a2f758aeac6d20382e900f0c301f45020afbebc
|
||||
generated: "2026-01-23T15:08:56.617442-08:00"
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
apiVersion: v2
|
||||
appVersion: v25.3.0
|
||||
appVersion: v26.3.1
|
||||
dependencies:
|
||||
- condition: nfd.enabled
|
||||
name: node-feature-discovery
|
||||
repository: https://kubernetes-sigs.github.io/node-feature-discovery/charts
|
||||
version: v0.17.2
|
||||
repository: oci://registry.k8s.io/nfd/charts
|
||||
version: 0.18.3
|
||||
description: NVIDIA GPU Operator creates/configures/manages GPUs atop Kubernetes
|
||||
home: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/overview.html
|
||||
icon: https://assets.nvidiagrid.net/ngc/logos/GPUoperator.png
|
||||
|
|
@ -20,4 +20,4 @@ kubeVersion: '>= 1.16.0-0'
|
|||
name: gpu-operator
|
||||
sources:
|
||||
- https://github.com/NVIDIA/gpu-operator
|
||||
version: v25.3.0
|
||||
version: v26.3.1
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
apiVersion: v2
|
||||
appVersion: v0.17.2
|
||||
appVersion: v0.18.3
|
||||
description: 'Detects hardware features available on each node in a Kubernetes cluster,
|
||||
and advertises those features using node labels. '
|
||||
home: https://github.com/kubernetes-sigs/node-feature-discovery
|
||||
icon: https://kubernetes-sigs.github.io/node-feature-discovery/v0.18/assets/images/nfd/favicon.svg
|
||||
keywords:
|
||||
- feature-discovery
|
||||
- feature-detection
|
||||
|
|
@ -11,4 +12,4 @@ name: node-feature-discovery
|
|||
sources:
|
||||
- https://github.com/kubernetes-sigs/node-feature-discovery
|
||||
type: application
|
||||
version: 0.17.2
|
||||
version: 0.18.3
|
||||
|
|
|
|||
|
|
@ -6,5 +6,5 @@ labels. NFD provides flexible configuration and extension points for a wide
|
|||
range of vendor and application specific node labeling needs.
|
||||
|
||||
See
|
||||
[NFD documentation](https://kubernetes-sigs.github.io/node-feature-discovery/v0.17/deployment/helm.html)
|
||||
[NFD documentation](https://kubernetes-sigs.github.io/node-feature-discovery/v0.18/deployment/helm.html)
|
||||
for deployment instructions.
|
||||
|
|
|
|||
|
|
@ -204,11 +204,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
@ -240,11 +248,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
@ -295,11 +311,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
@ -331,11 +355,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
@ -356,6 +388,19 @@ spec:
|
|||
name:
|
||||
description: Name of the rule.
|
||||
type: string
|
||||
vars:
|
||||
additionalProperties:
|
||||
type: string
|
||||
description: |-
|
||||
Vars is the variables to store if the rule matches. Variables can be
|
||||
referenced from other rules enabling more complex rule hierarchies.
|
||||
type: object
|
||||
varsTemplate:
|
||||
description: |-
|
||||
VarsTemplate specifies a template to expand for dynamically generating
|
||||
multiple variables. Data (after template expansion) must be keys with an
|
||||
optional value (<key>[=<value>]) separated by newlines.
|
||||
type: string
|
||||
required:
|
||||
- name
|
||||
type: object
|
||||
|
|
@ -498,11 +543,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
@ -534,11 +587,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
@ -589,11 +650,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
@ -625,11 +694,19 @@ spec:
|
|||
- Exists
|
||||
- DoesNotExist
|
||||
- Gt
|
||||
- Ge
|
||||
- Lt
|
||||
- Le
|
||||
- GtLt
|
||||
- GeLe
|
||||
- IsTrue
|
||||
- IsFalse
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
Type defines the value type for specific operators.
|
||||
The currently supported type is 'version' for Gt,Ge,Lt,Le,GtLt,GeLe operators.
|
||||
type: string
|
||||
value:
|
||||
description: |-
|
||||
Value is the list of values that the operand evaluates the input
|
||||
|
|
|
|||
|
|
@ -105,3 +105,22 @@ Create the name of the service account which nfd-gc will use
|
|||
{{ default "default" .Values.gc.serviceAccount.name }}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
imagePullSecrets helper - uses local values or falls back to global values
|
||||
*/}}
|
||||
{{- define "node-feature-discovery.imagePullSecrets" -}}
|
||||
{{- $imagePullSecrets := list -}}
|
||||
{{- if .Values.imagePullSecrets -}}
|
||||
{{- range .Values.imagePullSecrets -}}
|
||||
{{- $imagePullSecrets = append $imagePullSecrets . -}}
|
||||
{{- end -}}
|
||||
{{- else if and .Values.global .Values.global.imagePullSecrets -}}
|
||||
{{- range .Values.global.imagePullSecrets -}}
|
||||
{{- $imagePullSecrets = append $imagePullSecrets . -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
{{- if $imagePullSecrets -}}
|
||||
{{- $imagePullSecrets | toJson }}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,17 @@
|
|||
{{- if .Values.master.enable }}
|
||||
{{- if .Values.master.podDisruptionBudget.enable -}}
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: {{ include "node-feature-discovery.fullname" . }}-master
|
||||
namespace: {{ include "node-feature-discovery.namespace" . }}
|
||||
labels:
|
||||
{{- include "node-feature-discovery.labels" . | nindent 4 }}
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "node-feature-discovery.selectorLabels" . | nindent 6 }}
|
||||
role: master
|
||||
{{- toYaml (omit .Values.master.podDisruptionBudget "enable") | nindent 2 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
@ -29,13 +29,11 @@ spec:
|
|||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
dnsPolicy: {{ .Values.master.dnsPolicy }}
|
||||
{{- with .Values.priorityClassName }}
|
||||
priorityClassName: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
imagePullSecrets: {{ include "node-feature-discovery.imagePullSecrets" . }}
|
||||
serviceAccountName: {{ include "node-feature-discovery.master.serviceAccountName" . }}
|
||||
enableServiceLinks: false
|
||||
securityContext:
|
||||
|
|
@ -48,8 +46,9 @@ spec:
|
|||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
startupProbe:
|
||||
grpc:
|
||||
port: {{ .Values.master.healthPort | default "8082" }}
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.master.startupProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
|
|
@ -63,8 +62,9 @@ spec:
|
|||
timeoutSeconds: {{ . }}
|
||||
{{- end }}
|
||||
livenessProbe:
|
||||
grpc:
|
||||
port: {{ .Values.master.healthPort | default "8082" }}
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.master.livenessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
|
|
@ -78,8 +78,9 @@ spec:
|
|||
timeoutSeconds: {{ . }}
|
||||
{{- end }}
|
||||
readinessProbe:
|
||||
grpc:
|
||||
port: {{ .Values.master.healthPort | default "8082" }}
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.master.readinessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
|
|
@ -96,17 +97,15 @@ spec:
|
|||
successThreshold: {{ . }}
|
||||
{{- end }}
|
||||
ports:
|
||||
- containerPort: {{ .Values.master.metricsPort | default "8081" }}
|
||||
name: metrics
|
||||
- containerPort: {{ .Values.master.healthPort | default "8082" }}
|
||||
name: health
|
||||
- containerPort: {{ .Values.master.port | default "8080" }}
|
||||
name: http
|
||||
env:
|
||||
- name: NODE_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: spec.nodeName
|
||||
{{- with .Values.master.extraEnvs }}
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- toYaml . | nindent 10 }}
|
||||
{{- end}}
|
||||
command:
|
||||
- "nfd-master"
|
||||
|
|
@ -126,9 +125,6 @@ spec:
|
|||
{{- if .Values.master.enableTaints }}
|
||||
- "-enable-taints"
|
||||
{{- end }}
|
||||
{{- if .Values.master.featureRulesController | kindIs "invalid" | not }}
|
||||
- "-featurerules-controller={{ .Values.master.featureRulesController }}"
|
||||
{{- end }}
|
||||
{{- if .Values.master.resyncPeriod }}
|
||||
- "-resync-period={{ .Values.master.resyncPeriod }}"
|
||||
{{- end }}
|
||||
|
|
@ -139,8 +135,7 @@ spec:
|
|||
{{- range $key, $value := .Values.featureGates }}
|
||||
- "-feature-gates={{ $key }}={{ $value }}"
|
||||
{{- end }}
|
||||
- "-metrics={{ .Values.master.metricsPort | default "8081" }}"
|
||||
- "-grpc-health={{ .Values.master.healthPort | default "8082" }}"
|
||||
- "-port={{ .Values.master.port | default "8080" }}"
|
||||
{{- with .Values.master.extraArgs }}
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,17 @@
|
|||
{{- if .Values.gc.enable }}
|
||||
{{- if .Values.gc.podDisruptionBudget.enable -}}
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: {{ include "node-feature-discovery.fullname" . }}-gc
|
||||
namespace: {{ include "node-feature-discovery.namespace" . }}
|
||||
labels:
|
||||
{{- include "node-feature-discovery.labels" . | nindent 4 }}
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "node-feature-discovery.selectorLabels" . | nindent 6 }}
|
||||
role: gc
|
||||
{{- toYaml (omit .Values.gc.podDisruptionBudget "enable") | nindent 2 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
@ -29,14 +29,11 @@ spec:
|
|||
{{- end }}
|
||||
spec:
|
||||
serviceAccountName: {{ include "node-feature-discovery.gc.serviceAccountName" . }}
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
dnsPolicy: {{ .Values.gc.dnsPolicy }}
|
||||
{{- with .Values.priorityClassName }}
|
||||
priorityClassName: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
imagePullSecrets: {{ include "node-feature-discovery.imagePullSecrets" . }}
|
||||
securityContext:
|
||||
{{- toYaml .Values.gc.podSecurityContext | nindent 8 }}
|
||||
hostNetwork: {{ .Values.gc.hostNetwork }}
|
||||
|
|
@ -44,6 +41,41 @@ spec:
|
|||
- name: gc
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
|
||||
imagePullPolicy: "{{ .Values.image.pullPolicy }}"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.gc.livenessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.gc.livenessProbe.failureThreshold }}
|
||||
failureThreshold: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.gc.livenessProbe.periodSeconds }}
|
||||
periodSeconds: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.gc.livenessProbe.timeoutSeconds }}
|
||||
timeoutSeconds: {{ . }}
|
||||
{{- end }}
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.gc.readinessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.gc.readinessProbe.failureThreshold }}
|
||||
failureThreshold: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.gc.readinessProbe.periodSeconds }}
|
||||
periodSeconds: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.gc.readinessProbe.timeoutSeconds }}
|
||||
timeoutSeconds: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.gc.readinessProbe.successThreshold }}
|
||||
successThreshold: {{ . }}
|
||||
{{- end }}
|
||||
env:
|
||||
- name: NODE_NAME
|
||||
valueFrom:
|
||||
|
|
@ -70,8 +102,8 @@ spec:
|
|||
readOnlyRootFilesystem: true
|
||||
runAsNonRoot: true
|
||||
ports:
|
||||
- name: metrics
|
||||
containerPort: {{ .Values.gc.metricsPort | default "8081"}}
|
||||
- name: http
|
||||
containerPort: {{ .Values.gc.port | default "8080"}}
|
||||
|
||||
{{- with .Values.gc.nodeSelector }}
|
||||
nodeSelector:
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
{{- if .Values.postDeleteCleanup }}
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
|
|
@ -66,6 +67,7 @@ spec:
|
|||
role: prune
|
||||
spec:
|
||||
serviceAccountName: {{ include "node-feature-discovery.fullname" . }}-prune
|
||||
imagePullSecrets: {{ include "node-feature-discovery.imagePullSecrets" . }}
|
||||
containers:
|
||||
- name: nfd-master
|
||||
securityContext:
|
||||
|
|
@ -92,3 +94,9 @@ spec:
|
|||
tolerations:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.master.resources }}
|
||||
resources:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ spec:
|
|||
- honorLabels: true
|
||||
interval: {{ .Values.prometheus.scrapeInterval }}
|
||||
path: /metrics
|
||||
port: metrics
|
||||
port: http
|
||||
scheme: http
|
||||
namespaceSelector:
|
||||
matchNames:
|
||||
|
|
|
|||
|
|
@ -29,14 +29,11 @@ spec:
|
|||
{{- end }}
|
||||
spec:
|
||||
serviceAccountName: {{ include "node-feature-discovery.topologyUpdater.serviceAccountName" . }}
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
dnsPolicy: {{ .Values.topologyUpdater.dnsPolicy }}
|
||||
{{- with .Values.priorityClassName }}
|
||||
priorityClassName: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
imagePullSecrets: {{ include "node-feature-discovery.imagePullSecrets" . }}
|
||||
securityContext:
|
||||
{{- toYaml .Values.topologyUpdater.podSecurityContext | nindent 8 }}
|
||||
hostNetwork: {{ .Values.topologyUpdater.hostNetwork }}
|
||||
|
|
@ -45,8 +42,9 @@ spec:
|
|||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
|
||||
imagePullPolicy: "{{ .Values.image.pullPolicy }}"
|
||||
livenessProbe:
|
||||
grpc:
|
||||
port: {{ .Values.topologyUpdater.healthPort | default "8082" }}
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.topologyUpdater.livenessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
|
|
@ -60,8 +58,9 @@ spec:
|
|||
timeoutSeconds: {{ . }}
|
||||
{{- end }}
|
||||
readinessProbe:
|
||||
grpc:
|
||||
port: {{ .Values.topologyUpdater.healthPort | default "8082" }}
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.topologyUpdater.readinessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
|
|
@ -113,16 +112,13 @@ spec:
|
|||
# Disable kubelet state tracking by giving an empty path
|
||||
- "-kubelet-state-dir="
|
||||
{{- end }}
|
||||
- "-metrics={{ .Values.topologyUpdater.metricsPort | default "8081"}}"
|
||||
- "-grpc-health={{ .Values.topologyUpdater.healthPort | default "8082" }}"
|
||||
- "-port={{ .Values.topologyUpdater.port | default "8080"}}"
|
||||
{{- with .Values.topologyUpdater.extraArgs }}
|
||||
{{- toYaml . | nindent 10 }}
|
||||
{{- end }}
|
||||
ports:
|
||||
- containerPort: {{ .Values.topologyUpdater.metricsPort | default "8081"}}
|
||||
name: metrics
|
||||
- containerPort: {{ .Values.topologyUpdater.healthPort | default "8082" }}
|
||||
name: health
|
||||
- containerPort: {{ .Values.topologyUpdater.port | default "8080"}}
|
||||
name: http
|
||||
volumeMounts:
|
||||
{{- if .Values.topologyUpdater.kubeletConfigPath | empty | not }}
|
||||
- name: kubelet-config
|
||||
|
|
|
|||
|
|
@ -13,6 +13,10 @@ metadata:
|
|||
{{- end }}
|
||||
spec:
|
||||
revisionHistoryLimit: {{ .Values.worker.revisionHistoryLimit }}
|
||||
{{- with .Values.worker.updateStrategy }}
|
||||
updateStrategy:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end}}
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "node-feature-discovery.selectorLabels" . | nindent 6 }}
|
||||
|
|
@ -28,14 +32,11 @@ spec:
|
|||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
dnsPolicy: {{ .Values.worker.dnsPolicy }}
|
||||
{{- with .Values.priorityClassName }}
|
||||
priorityClassName: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
imagePullSecrets: {{ include "node-feature-discovery.imagePullSecrets" . }}
|
||||
serviceAccountName: {{ include "node-feature-discovery.worker.serviceAccountName" . }}
|
||||
securityContext:
|
||||
{{- toYaml .Values.worker.podSecurityContext | nindent 8 }}
|
||||
|
|
@ -47,8 +48,9 @@ spec:
|
|||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
livenessProbe:
|
||||
grpc:
|
||||
port: {{ .Values.worker.healthPort | default "8082" }}
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.worker.livenessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
|
|
@ -62,8 +64,9 @@ spec:
|
|||
timeoutSeconds: {{ . }}
|
||||
{{- end }}
|
||||
readinessProbe:
|
||||
grpc:
|
||||
port: {{ .Values.worker.healthPort | default "8082" }}
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
{{- with .Values.worker.readinessProbe.initialDelaySeconds }}
|
||||
initialDelaySeconds: {{ . }}
|
||||
{{- end }}
|
||||
|
|
@ -104,16 +107,13 @@ spec:
|
|||
{{- range $key, $value := .Values.featureGates }}
|
||||
- "-feature-gates={{ $key }}={{ $value }}"
|
||||
{{- end }}
|
||||
- "-metrics={{ .Values.worker.metricsPort | default "8081"}}"
|
||||
- "-grpc-health={{ .Values.worker.healthPort | default "8082" }}"
|
||||
- "-port={{ .Values.worker.port | default "8080"}}"
|
||||
{{- with .Values.worker.extraArgs }}
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
ports:
|
||||
- containerPort: {{ .Values.worker.metricsPort | default "8081"}}
|
||||
name: metrics
|
||||
- containerPort: {{ .Values.worker.healthPort | default "8082" }}
|
||||
name: health
|
||||
- containerPort: {{ .Values.worker.port | default "8080"}}
|
||||
name: http
|
||||
volumeMounts:
|
||||
- name: host-boot
|
||||
mountPath: "/host-boot"
|
||||
|
|
|
|||
|
|
@ -15,17 +15,20 @@ featureGates:
|
|||
|
||||
priorityClassName: ""
|
||||
|
||||
postDeleteCleanup: true
|
||||
|
||||
master:
|
||||
enable: true
|
||||
extraArgs: []
|
||||
extraEnvs: []
|
||||
hostNetwork: false
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
config: ### <NFD-MASTER-CONF-START-DO-NOT-REMOVE>
|
||||
# noPublish: false
|
||||
# autoDefaultNs: true
|
||||
# extraLabelNs: ["added.ns.io","added.kubernets.io"]
|
||||
# denyLabelNs: ["denied.ns.io","denied.kubernetes.io"]
|
||||
# enableTaints: false
|
||||
# informerPageSize: 200
|
||||
# labelWhiteList: "foo"
|
||||
# resyncPeriod: "2h"
|
||||
# restrictions:
|
||||
|
|
@ -66,15 +69,12 @@ master:
|
|||
# retryPeriod: 2s
|
||||
# nfdApiParallelism: 10
|
||||
### <NFD-MASTER-CONF-END-DO-NOT-REMOVE>
|
||||
metricsPort: 8081
|
||||
healthPort: 8082
|
||||
port: 8080
|
||||
instance:
|
||||
featureApi:
|
||||
resyncPeriod:
|
||||
denyLabelNs: []
|
||||
extraLabelNs: []
|
||||
enableTaints: false
|
||||
featureRulesController: null
|
||||
nfdApiParallelism: null
|
||||
deploymentAnnotations: {}
|
||||
replicaCount: 1
|
||||
|
|
@ -120,26 +120,21 @@ master:
|
|||
nodeSelector: {}
|
||||
|
||||
tolerations:
|
||||
- key: "node-role.kubernetes.io/master"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
effect: "NoSchedule"
|
||||
- key: "node-role.kubernetes.io/control-plane"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
effect: "NoSchedule"
|
||||
|
||||
podDisruptionBudget:
|
||||
enable: false
|
||||
minAvailable: 1
|
||||
unhealthyPodEvictionPolicy: AlwaysAllow
|
||||
|
||||
annotations: {}
|
||||
|
||||
affinity:
|
||||
nodeAffinity:
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- weight: 1
|
||||
preference:
|
||||
matchExpressions:
|
||||
- key: "node-role.kubernetes.io/master"
|
||||
operator: In
|
||||
values: [""]
|
||||
- weight: 1
|
||||
preference:
|
||||
matchExpressions:
|
||||
|
|
@ -148,20 +143,14 @@ master:
|
|||
values: [""]
|
||||
|
||||
startupProbe:
|
||||
grpc:
|
||||
port: 8082
|
||||
failureThreshold: 30
|
||||
# periodSeconds: 10
|
||||
livenessProbe:
|
||||
grpc:
|
||||
port: 8082
|
||||
livenessProbe: {}
|
||||
# failureThreshold: 3
|
||||
# initialDelaySeconds: 0
|
||||
# periodSeconds: 10
|
||||
# timeoutSeconds: 1
|
||||
readinessProbe:
|
||||
grpc:
|
||||
port: 8082
|
||||
failureThreshold: 10
|
||||
# initialDelaySeconds: 0
|
||||
# periodSeconds: 10
|
||||
|
|
@ -173,6 +162,7 @@ worker:
|
|||
extraArgs: []
|
||||
extraEnvs: []
|
||||
hostNetwork: false
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
config: ### <NFD-WORKER-CONF-START-DO-NOT-REMOVE>
|
||||
#core:
|
||||
# labelWhiteList:
|
||||
|
|
@ -416,8 +406,7 @@ worker:
|
|||
# matchName: {op: In, value: ["SWAP", "X86", "ARM"]}
|
||||
### <NFD-WORKER-CONF-END-DO-NOT-REMOVE>
|
||||
|
||||
metricsPort: 8081
|
||||
healthPort: 8082
|
||||
port: 8080
|
||||
daemonsetAnnotations: {}
|
||||
podSecurityContext: {}
|
||||
# fsGroup: 2000
|
||||
|
|
@ -431,15 +420,11 @@ worker:
|
|||
# runAsUser: 1000
|
||||
|
||||
livenessProbe:
|
||||
grpc:
|
||||
port: 8082
|
||||
initialDelaySeconds: 10
|
||||
# failureThreshold: 3
|
||||
# periodSeconds: 10
|
||||
# timeoutSeconds: 1
|
||||
readinessProbe:
|
||||
grpc:
|
||||
port: 8082
|
||||
initialDelaySeconds: 5
|
||||
failureThreshold: 10
|
||||
# periodSeconds: 10
|
||||
|
|
@ -483,6 +468,12 @@ worker:
|
|||
|
||||
priorityClassName: ""
|
||||
|
||||
updateStrategy: {}
|
||||
# type: RollingUpdate
|
||||
# rollingUpdate:
|
||||
# maxSurge: 0
|
||||
# maxUnavailable: "10%"
|
||||
|
||||
topologyUpdater:
|
||||
config: ### <NFD-TOPOLOGY-UPDATER-CONF-START-DO-NOT-REMOVE>
|
||||
## key = node name, value = list of resources to be excluded.
|
||||
|
|
@ -499,6 +490,7 @@ topologyUpdater:
|
|||
extraArgs: []
|
||||
extraEnvs: []
|
||||
hostNetwork: false
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
|
||||
serviceAccount:
|
||||
create: true
|
||||
|
|
@ -511,8 +503,7 @@ topologyUpdater:
|
|||
rbac:
|
||||
create: true
|
||||
|
||||
metricsPort: 8081
|
||||
healthPort: 8082
|
||||
port: 8080
|
||||
kubeletConfigPath:
|
||||
kubeletPodResourcesSockPath:
|
||||
updateInterval: 60s
|
||||
|
|
@ -526,17 +517,13 @@ topologyUpdater:
|
|||
drop: [ "ALL" ]
|
||||
readOnlyRootFilesystem: true
|
||||
runAsUser: 0
|
||||
|
||||
|
||||
livenessProbe:
|
||||
grpc:
|
||||
port: 8082
|
||||
initialDelaySeconds: 10
|
||||
# failureThreshold: 3
|
||||
# periodSeconds: 10
|
||||
# timeoutSeconds: 1
|
||||
readinessProbe:
|
||||
grpc:
|
||||
port: 8082
|
||||
initialDelaySeconds: 5
|
||||
failureThreshold: 10
|
||||
# periodSeconds: 10
|
||||
|
|
@ -563,6 +550,7 @@ gc:
|
|||
extraEnvs: []
|
||||
hostNetwork: false
|
||||
replicaCount: 1
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
|
||||
serviceAccount:
|
||||
create: true
|
||||
|
|
@ -575,6 +563,18 @@ gc:
|
|||
|
||||
podSecurityContext: {}
|
||||
|
||||
livenessProbe:
|
||||
initialDelaySeconds: 10
|
||||
# failureThreshold: 3
|
||||
# periodSeconds: 10
|
||||
# timeoutSeconds: 1
|
||||
readinessProbe:
|
||||
initialDelaySeconds: 5
|
||||
# failureThreshold: 3
|
||||
# periodSeconds: 10
|
||||
# timeoutSeconds: 1
|
||||
# successThreshold: 1
|
||||
|
||||
resources:
|
||||
limits:
|
||||
memory: 1Gi
|
||||
|
|
@ -582,7 +582,7 @@ gc:
|
|||
cpu: 10m
|
||||
memory: 128Mi
|
||||
|
||||
metricsPort: 8081
|
||||
port: 8080
|
||||
|
||||
nodeSelector: {}
|
||||
tolerations: []
|
||||
|
|
@ -590,6 +590,11 @@ gc:
|
|||
deploymentAnnotations: {}
|
||||
affinity: {}
|
||||
|
||||
podDisruptionBudget:
|
||||
enable: false
|
||||
minAvailable: 1
|
||||
unhealthyPodEvictionPolicy: AlwaysAllow
|
||||
|
||||
# specify how many old ReplicaSets for the Deployment to retain.
|
||||
revisionHistoryLimit:
|
||||
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
|
|||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
annotations:
|
||||
controller-gen.kubebuilder.io/version: v0.17.2
|
||||
controller-gen.kubebuilder.io/version: v0.20.1
|
||||
name: clusterpolicies.nvidia.com
|
||||
spec:
|
||||
group: nvidia.com
|
||||
|
|
@ -82,6 +82,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the CC Manager pod
|
||||
uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: CC Manager image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -136,13 +140,21 @@ spec:
|
|||
properties:
|
||||
default:
|
||||
default: false
|
||||
description: Default indicates whether to use CDI as the default
|
||||
mechanism for providing GPU access to containers.
|
||||
description: 'Deprecated: This field is no longer used. Setting
|
||||
cdi.enabled=true will configure CDI as the default mechanism
|
||||
for making GPUs accessible to containers.'
|
||||
type: boolean
|
||||
enabled:
|
||||
default: true
|
||||
description: Enabled indicates whether the Container Device Interface
|
||||
(CDI) should be used as the mechanism for making GPUs accessible
|
||||
to containers.
|
||||
type: boolean
|
||||
nriPluginEnabled:
|
||||
default: false
|
||||
description: Enabled indicates whether CDI can be used to make
|
||||
GPUs accessible to containers.
|
||||
description: NRIPluginEnabled indicates whether an NRI Plugin
|
||||
should be run as a means of injecting CDI devices to gpu management
|
||||
containers.
|
||||
type: boolean
|
||||
type: object
|
||||
daemonsets:
|
||||
|
|
@ -164,6 +176,239 @@ spec:
|
|||
(scope and select) objects. May match selectors of replication controllers
|
||||
and services.
|
||||
type: object
|
||||
podSecurityContext:
|
||||
description: 'Optional: Set pod-level security context for all
|
||||
DaemonSet pods (applies as defaults to all containers)'
|
||||
properties:
|
||||
appArmorProfile:
|
||||
description: |-
|
||||
appArmorProfile is the AppArmor options to use by the containers in this pod.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
properties:
|
||||
localhostProfile:
|
||||
description: |-
|
||||
localhostProfile indicates a profile loaded on the node that should be used.
|
||||
The profile must be preconfigured on the node to work.
|
||||
Must match the loaded name of the profile.
|
||||
Must be set if and only if type is "Localhost".
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
type indicates which kind of AppArmor profile will be applied.
|
||||
Valid options are:
|
||||
Localhost - a profile pre-loaded on the node.
|
||||
RuntimeDefault - the container runtime's default profile.
|
||||
Unconfined - no AppArmor enforcement.
|
||||
type: string
|
||||
required:
|
||||
- type
|
||||
type: object
|
||||
fsGroup:
|
||||
description: |-
|
||||
A special supplemental group that applies to all containers in a pod.
|
||||
Some volume types allow the Kubelet to change the ownership of that volume
|
||||
to be owned by the pod:
|
||||
|
||||
1. The owning GID will be the FSGroup
|
||||
2. The setgid bit is set (new files created in the volume will be owned by FSGroup)
|
||||
3. The permission bits are OR'd with rw-rw----
|
||||
|
||||
If unset, the Kubelet will not modify the ownership and permissions of any volume.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
format: int64
|
||||
type: integer
|
||||
fsGroupChangePolicy:
|
||||
description: |-
|
||||
fsGroupChangePolicy defines behavior of changing ownership and permission of the volume
|
||||
before being exposed inside Pod. This field will only apply to
|
||||
volume types which support fsGroup based ownership(and permissions).
|
||||
It will have no effect on ephemeral volume types such as: secret, configmaps
|
||||
and emptydir.
|
||||
Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
type: string
|
||||
runAsGroup:
|
||||
description: |-
|
||||
The GID to run the entrypoint of the container process.
|
||||
Uses runtime default if unset.
|
||||
May also be set in SecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence
|
||||
for that container.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
format: int64
|
||||
type: integer
|
||||
runAsNonRoot:
|
||||
description: |-
|
||||
Indicates that the container must run as a non-root user.
|
||||
If true, the Kubelet will validate the image at runtime to ensure that it
|
||||
does not run as UID 0 (root) and fail to start the container if it does.
|
||||
If unset or false, no such validation will be performed.
|
||||
May also be set in SecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence.
|
||||
type: boolean
|
||||
runAsUser:
|
||||
description: |-
|
||||
The UID to run the entrypoint of the container process.
|
||||
Defaults to user specified in image metadata if unspecified.
|
||||
May also be set in SecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence
|
||||
for that container.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
format: int64
|
||||
type: integer
|
||||
seLinuxChangePolicy:
|
||||
description: |-
|
||||
seLinuxChangePolicy defines how the container's SELinux label is applied to all volumes used by the Pod.
|
||||
It has no effect on nodes that do not support SELinux or to volumes does not support SELinux.
|
||||
Valid values are "MountOption" and "Recursive".
|
||||
|
||||
"Recursive" means relabeling of all files on all Pod volumes by the container runtime.
|
||||
This may be slow for large volumes, but allows mixing privileged and unprivileged Pods sharing the same volume on the same node.
|
||||
|
||||
"MountOption" mounts all eligible Pod volumes with `-o context` mount option.
|
||||
This requires all Pods that share the same volume to use the same SELinux label.
|
||||
It is not possible to share the same volume among privileged and unprivileged Pods.
|
||||
Eligible volumes are in-tree FibreChannel and iSCSI volumes, and all CSI volumes
|
||||
whose CSI driver announces SELinux support by setting spec.seLinuxMount: true in their
|
||||
CSIDriver instance. Other volumes are always re-labelled recursively.
|
||||
"MountOption" value is allowed only when SELinuxMount feature gate is enabled.
|
||||
|
||||
If not specified and SELinuxMount feature gate is enabled, "MountOption" is used.
|
||||
If not specified and SELinuxMount feature gate is disabled, "MountOption" is used for ReadWriteOncePod volumes
|
||||
and "Recursive" for all other volumes.
|
||||
|
||||
This field affects only Pods that have SELinux label set, either in PodSecurityContext or in SecurityContext of all containers.
|
||||
|
||||
All Pods that use the same volume should use the same seLinuxChangePolicy, otherwise some pods can get stuck in ContainerCreating state.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
type: string
|
||||
seLinuxOptions:
|
||||
description: |-
|
||||
The SELinux context to be applied to all containers.
|
||||
If unspecified, the container runtime will allocate a random SELinux context for each
|
||||
container. May also be set in SecurityContext. If set in
|
||||
both SecurityContext and PodSecurityContext, the value specified in SecurityContext
|
||||
takes precedence for that container.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
properties:
|
||||
level:
|
||||
description: Level is SELinux level label that applies
|
||||
to the container.
|
||||
type: string
|
||||
role:
|
||||
description: Role is a SELinux role label that applies
|
||||
to the container.
|
||||
type: string
|
||||
type:
|
||||
description: Type is a SELinux type label that applies
|
||||
to the container.
|
||||
type: string
|
||||
user:
|
||||
description: User is a SELinux user label that applies
|
||||
to the container.
|
||||
type: string
|
||||
type: object
|
||||
seccompProfile:
|
||||
description: |-
|
||||
The seccomp options to use by the containers in this pod.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
properties:
|
||||
localhostProfile:
|
||||
description: |-
|
||||
localhostProfile indicates a profile defined in a file on the node should be used.
|
||||
The profile must be preconfigured on the node to work.
|
||||
Must be a descending path, relative to the kubelet's configured seccomp profile location.
|
||||
Must be set if type is "Localhost". Must NOT be set for any other type.
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
type indicates which kind of seccomp profile will be applied.
|
||||
Valid options are:
|
||||
|
||||
Localhost - a profile defined in a file on the node should be used.
|
||||
RuntimeDefault - the container runtime default profile should be used.
|
||||
Unconfined - no profile should be applied.
|
||||
type: string
|
||||
required:
|
||||
- type
|
||||
type: object
|
||||
supplementalGroups:
|
||||
description: |-
|
||||
A list of groups applied to the first process run in each container, in
|
||||
addition to the container's primary GID and fsGroup (if specified). If
|
||||
the SupplementalGroupsPolicy feature is enabled, the
|
||||
supplementalGroupsPolicy field determines whether these are in addition
|
||||
to or instead of any group memberships defined in the container image.
|
||||
If unspecified, no additional groups are added, though group memberships
|
||||
defined in the container image may still be used, depending on the
|
||||
supplementalGroupsPolicy field.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
items:
|
||||
format: int64
|
||||
type: integer
|
||||
type: array
|
||||
x-kubernetes-list-type: atomic
|
||||
supplementalGroupsPolicy:
|
||||
description: |-
|
||||
Defines how supplemental groups of the first container processes are calculated.
|
||||
Valid values are "Merge" and "Strict". If not specified, "Merge" is used.
|
||||
(Alpha) Using the field requires the SupplementalGroupsPolicy feature gate to be enabled
|
||||
and the container runtime must implement support for this feature.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
type: string
|
||||
sysctls:
|
||||
description: |-
|
||||
Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported
|
||||
sysctls (by the container runtime) might fail to launch.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
items:
|
||||
description: Sysctl defines a kernel parameter to be set
|
||||
properties:
|
||||
name:
|
||||
description: Name of a property to set
|
||||
type: string
|
||||
value:
|
||||
description: Value of a property to set
|
||||
type: string
|
||||
required:
|
||||
- name
|
||||
- value
|
||||
type: object
|
||||
type: array
|
||||
x-kubernetes-list-type: atomic
|
||||
windowsOptions:
|
||||
description: |-
|
||||
The Windows specific settings applied to all containers.
|
||||
If unspecified, the options within a container's SecurityContext will be used.
|
||||
If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.
|
||||
Note that this field cannot be set when spec.os.name is linux.
|
||||
properties:
|
||||
gmsaCredentialSpec:
|
||||
description: |-
|
||||
GMSACredentialSpec is where the GMSA admission webhook
|
||||
(https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the
|
||||
GMSA credential spec named by the GMSACredentialSpecName field.
|
||||
type: string
|
||||
gmsaCredentialSpecName:
|
||||
description: GMSACredentialSpecName is the name of the
|
||||
GMSA credential spec to use.
|
||||
type: string
|
||||
hostProcess:
|
||||
description: |-
|
||||
HostProcess determines if a container should be run as a 'Host Process' container.
|
||||
All of a Pod's containers must have the same effective HostProcess value
|
||||
(it is not allowed to have a mix of HostProcess containers and non-HostProcess containers).
|
||||
In addition, if HostProcess is true then HostNetwork must also be set to true.
|
||||
type: boolean
|
||||
runAsUserName:
|
||||
description: |-
|
||||
The UserName in Windows to run the entrypoint of the container process.
|
||||
Defaults to the user specified in image metadata if unspecified.
|
||||
May also be set in PodSecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence.
|
||||
type: string
|
||||
type: object
|
||||
type: object
|
||||
priorityClassName:
|
||||
type: string
|
||||
rollingUpdate:
|
||||
|
|
@ -193,9 +438,10 @@ spec:
|
|||
operator:
|
||||
description: |-
|
||||
Operator represents a key's relationship to the value.
|
||||
Valid operators are Exists and Equal. Defaults to Equal.
|
||||
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
|
||||
Exists is equivalent to wildcard for value, so that a pod can
|
||||
tolerate all taints of a particular category.
|
||||
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).
|
||||
type: string
|
||||
tolerationSeconds:
|
||||
description: |-
|
||||
|
|
@ -247,6 +493,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the DCGM pod uses the
|
||||
host's network namespace.
|
||||
type: boolean
|
||||
hostPort:
|
||||
description: 'Deprecated: HostPort represents host port that needs
|
||||
to be bound for DCGM engine (Default: 5555)'
|
||||
|
|
@ -337,6 +587,27 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork allows the DCGM-Exporter daemon set to
|
||||
expose metrics port on the host's network namespace.
|
||||
type: boolean
|
||||
hostPID:
|
||||
description: HostPID allows the DCGM-Exporter daemon set to access
|
||||
the host's PID namespace
|
||||
type: boolean
|
||||
hpcJobMapping:
|
||||
description: 'Optional: HPC job mapping configuration for NVIDIA
|
||||
DCGM Exporter'
|
||||
properties:
|
||||
directory:
|
||||
description: |-
|
||||
Directory path where HPC job mapping files are created by the workload manager
|
||||
Defaults to /var/lib/dcgm-exporter/job-mapping if not specified
|
||||
type: string
|
||||
enabled:
|
||||
description: Enable HPC job mapping for DCGM Exporter
|
||||
type: boolean
|
||||
type: object
|
||||
image:
|
||||
description: NVIDIA DCGM Exporter image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -381,6 +652,19 @@ spec:
|
|||
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
type: object
|
||||
type: object
|
||||
service:
|
||||
description: 'Optional: Service configuration for NVIDIA DCGM
|
||||
Exporter'
|
||||
properties:
|
||||
internalTrafficPolicy:
|
||||
description: InternalTrafficPolicy describes how nodes distribute
|
||||
service traffic they receive on the ClusterIP.
|
||||
type: string
|
||||
type:
|
||||
description: Type represents the ServiceType which describes
|
||||
ingress methods for a service
|
||||
type: string
|
||||
type: object
|
||||
serviceMonitor:
|
||||
description: 'Optional: ServiceMonitor configuration for NVIDIA
|
||||
DCGM Exporter'
|
||||
|
|
@ -418,7 +702,7 @@ spec:
|
|||
action:
|
||||
default: replace
|
||||
description: |-
|
||||
Action to perform based on the regex matching.
|
||||
action to perform based on the regex matching.
|
||||
|
||||
`Uppercase` and `Lowercase` actions require Prometheus >= v2.36.0.
|
||||
`DropEqual` and `KeepEqual` actions require Prometheus >= v2.41.0.
|
||||
|
|
@ -450,41 +734,41 @@ spec:
|
|||
type: string
|
||||
modulus:
|
||||
description: |-
|
||||
Modulus to take of the hash of the source label values.
|
||||
modulus to take of the hash of the source label values.
|
||||
|
||||
Only applicable when the action is `HashMod`.
|
||||
format: int64
|
||||
type: integer
|
||||
regex:
|
||||
description: Regular expression against which the extracted
|
||||
value is matched.
|
||||
description: regex defines the regular expression against
|
||||
which the extracted value is matched.
|
||||
type: string
|
||||
replacement:
|
||||
description: |-
|
||||
Replacement value against which a Replace action is performed if the
|
||||
replacement value against which a Replace action is performed if the
|
||||
regular expression matches.
|
||||
|
||||
Regex capture groups are available.
|
||||
type: string
|
||||
separator:
|
||||
description: Separator is the string between concatenated
|
||||
description: separator defines the string between concatenated
|
||||
SourceLabels.
|
||||
type: string
|
||||
sourceLabels:
|
||||
description: |-
|
||||
The source labels select values from existing labels. Their content is
|
||||
sourceLabels defines the source labels select values from existing labels. Their content is
|
||||
concatenated using the configured Separator and matched against the
|
||||
configured regular expression.
|
||||
items:
|
||||
description: |-
|
||||
LabelName is a valid Prometheus label name which may only contain ASCII
|
||||
letters, numbers, as well as underscores.
|
||||
pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$
|
||||
LabelName is a valid Prometheus label name.
|
||||
For Prometheus 3.x, a label name is valid if it contains UTF-8 characters.
|
||||
For Prometheus 2.x, a label name is only valid if it contains ASCII characters, letters, numbers, as well as underscores.
|
||||
type: string
|
||||
type: array
|
||||
targetLabel:
|
||||
description: |-
|
||||
Label to which the resulting string is written in a replacement.
|
||||
targetLabel defines the label to which the resulting string is written in a replacement.
|
||||
|
||||
It is mandatory for `Replace`, `HashMod`, `Lowercase`, `Uppercase`,
|
||||
`KeepEqual` and `DropEqual` actions.
|
||||
|
|
@ -539,6 +823,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Device Plugin pod
|
||||
uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA Device Plugin image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -631,6 +919,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Driver pod uses
|
||||
the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA Driver image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -666,11 +958,18 @@ spec:
|
|||
licensing'
|
||||
properties:
|
||||
configMapName:
|
||||
description: 'Deprecated: ConfigMapName has been deprecated
|
||||
in favour of SecretName. Please use secrets to handle the
|
||||
licensing server configuration more securely'
|
||||
type: string
|
||||
nlsEnabled:
|
||||
description: NLSEnabled indicates if NVIDIA Licensing System
|
||||
is used for licensing.
|
||||
type: boolean
|
||||
secretName:
|
||||
description: SecretName indicates the name of the secret containing
|
||||
the licensing token
|
||||
type: string
|
||||
type: object
|
||||
livenessProbe:
|
||||
description: NVIDIA Driver container liveness probe settings
|
||||
|
|
@ -844,6 +1143,10 @@ spec:
|
|||
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
type: object
|
||||
type: object
|
||||
secretEnv:
|
||||
description: 'Optional: SecretEnv represents the name of the Kubernetes
|
||||
Secret with secret environment variables for the NVIDIA Driver'
|
||||
type: string
|
||||
startupProbe:
|
||||
description: NVIDIA Driver container startup probe settings
|
||||
properties:
|
||||
|
|
@ -1132,6 +1435,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the GPU Feature Discovery
|
||||
pod uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: GFD image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -1275,6 +1582,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Kata Manager pod
|
||||
uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: Kata Manager image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -1323,6 +1634,86 @@ spec:
|
|||
description: Kata Manager image tag
|
||||
type: string
|
||||
type: object
|
||||
kataSandboxDevicePlugin:
|
||||
description: KataSandboxDevicePlugin component spec
|
||||
properties:
|
||||
args:
|
||||
description: 'Optional: List of arguments'
|
||||
items:
|
||||
type: string
|
||||
type: array
|
||||
enabled:
|
||||
description: Enabled indicates if deployment of NVIDIA component
|
||||
through operator is enabled
|
||||
type: boolean
|
||||
env:
|
||||
description: 'Optional: List of environment variables'
|
||||
items:
|
||||
description: EnvVar represents an environment variable present
|
||||
in a Container.
|
||||
properties:
|
||||
name:
|
||||
description: Name of the environment variable.
|
||||
type: string
|
||||
value:
|
||||
description: Value of the environment variable.
|
||||
type: string
|
||||
required:
|
||||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Kata Sandbox Device
|
||||
Plugin pod uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA component image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
type: string
|
||||
imagePullPolicy:
|
||||
description: Image pull policy
|
||||
type: string
|
||||
imagePullSecrets:
|
||||
description: Image pull secrets
|
||||
items:
|
||||
type: string
|
||||
type: array
|
||||
repository:
|
||||
description: NVIDIA component image repository
|
||||
type: string
|
||||
resources:
|
||||
description: 'Optional: Define resources requests and limits for
|
||||
each pod'
|
||||
properties:
|
||||
limits:
|
||||
additionalProperties:
|
||||
anyOf:
|
||||
- type: integer
|
||||
- type: string
|
||||
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
|
||||
x-kubernetes-int-or-string: true
|
||||
description: |-
|
||||
Limits describes the maximum amount of compute resources allowed.
|
||||
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
type: object
|
||||
requests:
|
||||
additionalProperties:
|
||||
anyOf:
|
||||
- type: integer
|
||||
- type: string
|
||||
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
|
||||
x-kubernetes-int-or-string: true
|
||||
description: |-
|
||||
Requests describes the minimum amount of compute resources required.
|
||||
If Requests is omitted for a container, it defaults to Limits if that is explicitly specified,
|
||||
otherwise to an implementation-defined value. Requests cannot exceed Limits.
|
||||
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
type: object
|
||||
type: object
|
||||
version:
|
||||
description: NVIDIA component image tag
|
||||
type: string
|
||||
type: object
|
||||
mig:
|
||||
description: MIG spec
|
||||
properties:
|
||||
|
|
@ -1356,8 +1747,8 @@ spec:
|
|||
- ""
|
||||
type: string
|
||||
name:
|
||||
default: default-mig-parted-config
|
||||
description: ConfigMap name
|
||||
description: ConfigMap name. If not specified, MIG configuration
|
||||
will be dynamically generated from hardware.
|
||||
type: string
|
||||
type: object
|
||||
enabled:
|
||||
|
|
@ -1388,6 +1779,10 @@ spec:
|
|||
description: ConfigMap name
|
||||
type: string
|
||||
type: object
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the MIG Manager pod
|
||||
uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA MIG Manager image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -1464,6 +1859,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Node Status Exporter
|
||||
pod uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: Node Status Exporter image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -1524,16 +1923,12 @@ spec:
|
|||
queryable and should be preserved when modifying objects.
|
||||
type: object
|
||||
defaultRuntime:
|
||||
default: docker
|
||||
description: Runtime defines container runtime type
|
||||
enum:
|
||||
- docker
|
||||
- crio
|
||||
- containerd
|
||||
description: 'Deprecated: DefaultRuntime is no longer used by
|
||||
the gpu-operator. This is instead, detected at runtime.'
|
||||
type: string
|
||||
initContainer:
|
||||
description: InitContainerSpec describes configuration for initContainer
|
||||
image used with all components
|
||||
description: 'Deprecated: InitContainerSpec describes configuration
|
||||
for initContainer image used with all components'
|
||||
properties:
|
||||
image:
|
||||
description: Image represents image name
|
||||
|
|
@ -1570,8 +1965,6 @@ spec:
|
|||
image should be used on OpenShift to build and install driver
|
||||
modules
|
||||
type: boolean
|
||||
required:
|
||||
- defaultRuntime
|
||||
type: object
|
||||
psa:
|
||||
description: PSA defines spec for PodSecurityAdmission configuration
|
||||
|
|
@ -1619,6 +2012,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Sandbox Device
|
||||
Plugin pod uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA Sandbox Device Plugin image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -1686,6 +2083,15 @@ spec:
|
|||
Enabled indicates if the GPU Operator should manage additional operands required
|
||||
for sandbox workloads (i.e. VFIO Manager, vGPU Manager, and additional device plugins)
|
||||
type: boolean
|
||||
mode:
|
||||
default: kubevirt
|
||||
description: |-
|
||||
Mode indicates the sandbox mode. Accepted values are "kubevirt"
|
||||
and "kata". The default value is "kubevirt".
|
||||
enum:
|
||||
- kubevirt
|
||||
- kata
|
||||
type: string
|
||||
type: object
|
||||
toolkit:
|
||||
description: Toolkit component spec
|
||||
|
|
@ -1715,6 +2121,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Container Toolkit
|
||||
pod uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA Container Toolkit image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -1831,6 +2241,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Validator pod uses
|
||||
the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: Validator image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -2049,6 +2463,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the VFIO Manager pod
|
||||
uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: VFIO Manager image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -2137,6 +2555,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the vGPU Device Manager
|
||||
pod uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA vGPU Device Manager image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -2255,6 +2677,10 @@ spec:
|
|||
- name
|
||||
type: object
|
||||
type: array
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the vGPU Manager pod
|
||||
uses the host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
description: NVIDIA vGPU Manager image name
|
||||
pattern: '[a-zA-Z0-9\-]+'
|
||||
|
|
@ -2267,6 +2693,13 @@ spec:
|
|||
items:
|
||||
type: string
|
||||
type: array
|
||||
kernelModuleConfig:
|
||||
description: 'Optional: Kernel module configuration parameters
|
||||
for the vGPU manager'
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
type: object
|
||||
repository:
|
||||
description: NVIDIA vGPU Manager image repository
|
||||
type: string
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
|
|||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
annotations:
|
||||
controller-gen.kubebuilder.io/version: v0.17.2
|
||||
controller-gen.kubebuilder.io/version: v0.20.1
|
||||
name: nvidiadrivers.nvidia.com
|
||||
spec:
|
||||
group: nvidia.com
|
||||
|
|
@ -192,6 +192,10 @@ spec:
|
|||
description: NVIDIA GPUDirect Storage Driver image tag
|
||||
type: string
|
||||
type: object
|
||||
hostNetwork:
|
||||
description: HostNetwork indicates whether the Driver pod uses the
|
||||
host's network namespace.
|
||||
type: boolean
|
||||
image:
|
||||
default: nvcr.io/nvidia/driver
|
||||
description: NVIDIA Driver container image name
|
||||
|
|
@ -234,11 +238,16 @@ spec:
|
|||
description: 'Optional: Licensing configuration for NVIDIA vGPU licensing'
|
||||
properties:
|
||||
name:
|
||||
description: 'Deprecated: ConfigMapName has been deprecated in
|
||||
favour of SecretName. Please use secrets to handle the licensing
|
||||
server configuration more securely'
|
||||
type: string
|
||||
nlsEnabled:
|
||||
description: NLSEnabled indicates if NVIDIA Licensing System is
|
||||
used for licensing.
|
||||
type: boolean
|
||||
secretName:
|
||||
type: string
|
||||
type: object
|
||||
livenessProbe:
|
||||
description: NVIDIA Driver container liveness probe settings
|
||||
|
|
@ -522,6 +531,239 @@ spec:
|
|||
description: NodeSelector specifies a selector for installation of
|
||||
NVIDIA driver
|
||||
type: object
|
||||
podSecurityContext:
|
||||
description: 'Optional: Set pod-level security context for driver
|
||||
pod'
|
||||
properties:
|
||||
appArmorProfile:
|
||||
description: |-
|
||||
appArmorProfile is the AppArmor options to use by the containers in this pod.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
properties:
|
||||
localhostProfile:
|
||||
description: |-
|
||||
localhostProfile indicates a profile loaded on the node that should be used.
|
||||
The profile must be preconfigured on the node to work.
|
||||
Must match the loaded name of the profile.
|
||||
Must be set if and only if type is "Localhost".
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
type indicates which kind of AppArmor profile will be applied.
|
||||
Valid options are:
|
||||
Localhost - a profile pre-loaded on the node.
|
||||
RuntimeDefault - the container runtime's default profile.
|
||||
Unconfined - no AppArmor enforcement.
|
||||
type: string
|
||||
required:
|
||||
- type
|
||||
type: object
|
||||
fsGroup:
|
||||
description: |-
|
||||
A special supplemental group that applies to all containers in a pod.
|
||||
Some volume types allow the Kubelet to change the ownership of that volume
|
||||
to be owned by the pod:
|
||||
|
||||
1. The owning GID will be the FSGroup
|
||||
2. The setgid bit is set (new files created in the volume will be owned by FSGroup)
|
||||
3. The permission bits are OR'd with rw-rw----
|
||||
|
||||
If unset, the Kubelet will not modify the ownership and permissions of any volume.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
format: int64
|
||||
type: integer
|
||||
fsGroupChangePolicy:
|
||||
description: |-
|
||||
fsGroupChangePolicy defines behavior of changing ownership and permission of the volume
|
||||
before being exposed inside Pod. This field will only apply to
|
||||
volume types which support fsGroup based ownership(and permissions).
|
||||
It will have no effect on ephemeral volume types such as: secret, configmaps
|
||||
and emptydir.
|
||||
Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
type: string
|
||||
runAsGroup:
|
||||
description: |-
|
||||
The GID to run the entrypoint of the container process.
|
||||
Uses runtime default if unset.
|
||||
May also be set in SecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence
|
||||
for that container.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
format: int64
|
||||
type: integer
|
||||
runAsNonRoot:
|
||||
description: |-
|
||||
Indicates that the container must run as a non-root user.
|
||||
If true, the Kubelet will validate the image at runtime to ensure that it
|
||||
does not run as UID 0 (root) and fail to start the container if it does.
|
||||
If unset or false, no such validation will be performed.
|
||||
May also be set in SecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence.
|
||||
type: boolean
|
||||
runAsUser:
|
||||
description: |-
|
||||
The UID to run the entrypoint of the container process.
|
||||
Defaults to user specified in image metadata if unspecified.
|
||||
May also be set in SecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence
|
||||
for that container.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
format: int64
|
||||
type: integer
|
||||
seLinuxChangePolicy:
|
||||
description: |-
|
||||
seLinuxChangePolicy defines how the container's SELinux label is applied to all volumes used by the Pod.
|
||||
It has no effect on nodes that do not support SELinux or to volumes does not support SELinux.
|
||||
Valid values are "MountOption" and "Recursive".
|
||||
|
||||
"Recursive" means relabeling of all files on all Pod volumes by the container runtime.
|
||||
This may be slow for large volumes, but allows mixing privileged and unprivileged Pods sharing the same volume on the same node.
|
||||
|
||||
"MountOption" mounts all eligible Pod volumes with `-o context` mount option.
|
||||
This requires all Pods that share the same volume to use the same SELinux label.
|
||||
It is not possible to share the same volume among privileged and unprivileged Pods.
|
||||
Eligible volumes are in-tree FibreChannel and iSCSI volumes, and all CSI volumes
|
||||
whose CSI driver announces SELinux support by setting spec.seLinuxMount: true in their
|
||||
CSIDriver instance. Other volumes are always re-labelled recursively.
|
||||
"MountOption" value is allowed only when SELinuxMount feature gate is enabled.
|
||||
|
||||
If not specified and SELinuxMount feature gate is enabled, "MountOption" is used.
|
||||
If not specified and SELinuxMount feature gate is disabled, "MountOption" is used for ReadWriteOncePod volumes
|
||||
and "Recursive" for all other volumes.
|
||||
|
||||
This field affects only Pods that have SELinux label set, either in PodSecurityContext or in SecurityContext of all containers.
|
||||
|
||||
All Pods that use the same volume should use the same seLinuxChangePolicy, otherwise some pods can get stuck in ContainerCreating state.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
type: string
|
||||
seLinuxOptions:
|
||||
description: |-
|
||||
The SELinux context to be applied to all containers.
|
||||
If unspecified, the container runtime will allocate a random SELinux context for each
|
||||
container. May also be set in SecurityContext. If set in
|
||||
both SecurityContext and PodSecurityContext, the value specified in SecurityContext
|
||||
takes precedence for that container.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
properties:
|
||||
level:
|
||||
description: Level is SELinux level label that applies to
|
||||
the container.
|
||||
type: string
|
||||
role:
|
||||
description: Role is a SELinux role label that applies to
|
||||
the container.
|
||||
type: string
|
||||
type:
|
||||
description: Type is a SELinux type label that applies to
|
||||
the container.
|
||||
type: string
|
||||
user:
|
||||
description: User is a SELinux user label that applies to
|
||||
the container.
|
||||
type: string
|
||||
type: object
|
||||
seccompProfile:
|
||||
description: |-
|
||||
The seccomp options to use by the containers in this pod.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
properties:
|
||||
localhostProfile:
|
||||
description: |-
|
||||
localhostProfile indicates a profile defined in a file on the node should be used.
|
||||
The profile must be preconfigured on the node to work.
|
||||
Must be a descending path, relative to the kubelet's configured seccomp profile location.
|
||||
Must be set if type is "Localhost". Must NOT be set for any other type.
|
||||
type: string
|
||||
type:
|
||||
description: |-
|
||||
type indicates which kind of seccomp profile will be applied.
|
||||
Valid options are:
|
||||
|
||||
Localhost - a profile defined in a file on the node should be used.
|
||||
RuntimeDefault - the container runtime default profile should be used.
|
||||
Unconfined - no profile should be applied.
|
||||
type: string
|
||||
required:
|
||||
- type
|
||||
type: object
|
||||
supplementalGroups:
|
||||
description: |-
|
||||
A list of groups applied to the first process run in each container, in
|
||||
addition to the container's primary GID and fsGroup (if specified). If
|
||||
the SupplementalGroupsPolicy feature is enabled, the
|
||||
supplementalGroupsPolicy field determines whether these are in addition
|
||||
to or instead of any group memberships defined in the container image.
|
||||
If unspecified, no additional groups are added, though group memberships
|
||||
defined in the container image may still be used, depending on the
|
||||
supplementalGroupsPolicy field.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
items:
|
||||
format: int64
|
||||
type: integer
|
||||
type: array
|
||||
x-kubernetes-list-type: atomic
|
||||
supplementalGroupsPolicy:
|
||||
description: |-
|
||||
Defines how supplemental groups of the first container processes are calculated.
|
||||
Valid values are "Merge" and "Strict". If not specified, "Merge" is used.
|
||||
(Alpha) Using the field requires the SupplementalGroupsPolicy feature gate to be enabled
|
||||
and the container runtime must implement support for this feature.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
type: string
|
||||
sysctls:
|
||||
description: |-
|
||||
Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported
|
||||
sysctls (by the container runtime) might fail to launch.
|
||||
Note that this field cannot be set when spec.os.name is windows.
|
||||
items:
|
||||
description: Sysctl defines a kernel parameter to be set
|
||||
properties:
|
||||
name:
|
||||
description: Name of a property to set
|
||||
type: string
|
||||
value:
|
||||
description: Value of a property to set
|
||||
type: string
|
||||
required:
|
||||
- name
|
||||
- value
|
||||
type: object
|
||||
type: array
|
||||
x-kubernetes-list-type: atomic
|
||||
windowsOptions:
|
||||
description: |-
|
||||
The Windows specific settings applied to all containers.
|
||||
If unspecified, the options within a container's SecurityContext will be used.
|
||||
If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.
|
||||
Note that this field cannot be set when spec.os.name is linux.
|
||||
properties:
|
||||
gmsaCredentialSpec:
|
||||
description: |-
|
||||
GMSACredentialSpec is where the GMSA admission webhook
|
||||
(https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the
|
||||
GMSA credential spec named by the GMSACredentialSpecName field.
|
||||
type: string
|
||||
gmsaCredentialSpecName:
|
||||
description: GMSACredentialSpecName is the name of the GMSA
|
||||
credential spec to use.
|
||||
type: string
|
||||
hostProcess:
|
||||
description: |-
|
||||
HostProcess determines if a container should be run as a 'Host Process' container.
|
||||
All of a Pod's containers must have the same effective HostProcess value
|
||||
(it is not allowed to have a mix of HostProcess containers and non-HostProcess containers).
|
||||
In addition, if HostProcess is true then HostNetwork must also be set to true.
|
||||
type: boolean
|
||||
runAsUserName:
|
||||
description: |-
|
||||
The UserName in Windows to run the entrypoint of the container process.
|
||||
Defaults to the user specified in image metadata if unspecified.
|
||||
May also be set in PodSecurityContext. If set in both SecurityContext and
|
||||
PodSecurityContext, the value specified in SecurityContext takes precedence.
|
||||
type: string
|
||||
type: object
|
||||
type: object
|
||||
priorityClassName:
|
||||
description: 'Optional: Set priorityClassName'
|
||||
type: string
|
||||
|
|
@ -616,6 +858,10 @@ spec:
|
|||
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
type: object
|
||||
type: object
|
||||
secretEnv:
|
||||
description: 'Optional: SecretEnv represents the name of the Kubernetes
|
||||
Secret with secret environment variables for the NVIDIA Driver'
|
||||
type: string
|
||||
startupProbe:
|
||||
description: NVIDIA Driver container startup probe settings
|
||||
properties:
|
||||
|
|
@ -675,9 +921,10 @@ spec:
|
|||
operator:
|
||||
description: |-
|
||||
Operator represents a key's relationship to the value.
|
||||
Valid operators are Exists and Equal. Defaults to Equal.
|
||||
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
|
||||
Exists is equivalent to wildcard for value, so that a pod can
|
||||
tolerate all taints of a particular category.
|
||||
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).
|
||||
type: string
|
||||
tolerationSeconds:
|
||||
description: |-
|
||||
|
|
@ -798,6 +1045,7 @@ spec:
|
|||
- ignored
|
||||
- ready
|
||||
- notReady
|
||||
- disabled
|
||||
type: string
|
||||
required:
|
||||
- state
|
||||
|
|
|
|||
|
|
@ -35,16 +35,13 @@ spec:
|
|||
image: {{ include "gpu-operator.fullimage" . }}
|
||||
imagePullPolicy: {{ .Values.operator.imagePullPolicy }}
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- >
|
||||
kubectl delete clusterpolicy cluster-policy;
|
||||
kubectl delete crd clusterpolicies.nvidia.com;
|
||||
kubectl delete crd nvidiadrivers.nvidia.com --ignore-not-found=true;
|
||||
{{- if .Values.nfd.enabled -}}
|
||||
kubectl delete crd nodefeatures.nfd.k8s-sigs.io --ignore-not-found=true;
|
||||
kubectl delete crd nodefeaturegroups.nfd.k8s-sigs.io --ignore-not-found=true;
|
||||
kubectl delete crd nodefeaturerules.nfd.k8s-sigs.io --ignore-not-found=true;
|
||||
{{- end }}
|
||||
- /usr/bin/manage-crds
|
||||
args:
|
||||
- delete
|
||||
- --filepath=/opt/gpu-operator/nvidia.com_clusterpolicies.yaml
|
||||
- --filepath=/opt/gpu-operator/nvidia.com_nvidiadrivers.yaml
|
||||
{{- if .Values.nfd.enabled }}
|
||||
- --filepath=/opt/gpu-operator/nfd-api-crds.yaml
|
||||
{{- end }}
|
||||
restartPolicy: OnFailure
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -85,49 +85,65 @@ spec:
|
|||
{{- if .Values.validator.args }}
|
||||
args: {{ toYaml .Values.validator.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.hostNetwork }}
|
||||
hostNetwork: {{ .Values.validator.hostNetwork }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.plugin }}
|
||||
plugin:
|
||||
{{- if .Values.validator.plugin.env }}
|
||||
env: {{ toYaml .Values.validator.plugin.env | nindent 8 }}
|
||||
{{- else }}
|
||||
env: []
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.cuda }}
|
||||
cuda:
|
||||
{{- if .Values.validator.cuda.env }}
|
||||
env: {{ toYaml .Values.validator.cuda.env | nindent 8 }}
|
||||
{{- else }}
|
||||
env: []
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.driver }}
|
||||
driver:
|
||||
{{- if .Values.validator.driver.env }}
|
||||
env: {{ toYaml .Values.validator.driver.env | nindent 8 }}
|
||||
{{- else }}
|
||||
env: []
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.toolkit }}
|
||||
toolkit:
|
||||
{{- if .Values.validator.toolkit.env }}
|
||||
env: {{ toYaml .Values.validator.toolkit.env | nindent 8 }}
|
||||
{{- else }}
|
||||
env: []
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.vfioPCI }}
|
||||
vfioPCI:
|
||||
{{- if .Values.validator.vfioPCI.env }}
|
||||
env: {{ toYaml .Values.validator.vfioPCI.env | nindent 8 }}
|
||||
{{- else }}
|
||||
env: []
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.vgpuManager }}
|
||||
vgpuManager:
|
||||
{{- if .Values.validator.vgpuManager.env }}
|
||||
env: {{ toYaml .Values.validator.vgpuManager.env | nindent 8 }}
|
||||
{{- else }}
|
||||
env: []
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.validator.vgpuDevices }}
|
||||
vgpuDevices:
|
||||
{{- if .Values.validator.vgpuDevices.env }}
|
||||
env: {{ toYaml .Values.validator.vgpuDevices.env | nindent 8 }}
|
||||
{{- else }}
|
||||
env: []
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
mig:
|
||||
{{- if .Values.mig.strategy }}
|
||||
strategy: {{ .Values.mig.strategy }}
|
||||
|
|
@ -136,7 +152,12 @@ spec:
|
|||
enabled: {{ .Values.psa.enabled }}
|
||||
cdi:
|
||||
enabled: {{ .Values.cdi.enabled }}
|
||||
{{- if .Values.cdi.default }}
|
||||
default: {{ .Values.cdi.default }}
|
||||
{{- end }}
|
||||
{{- if and (.Values.cdi.enabled) (.Values.cdi.nriPluginEnabled) }}
|
||||
nriPluginEnabled: {{ .Values.cdi.nriPluginEnabled }}
|
||||
{{- end }}
|
||||
driver:
|
||||
enabled: {{ .Values.driver.enabled }}
|
||||
useNvidiaDriverCRD: {{ .Values.driver.nvidiaDriverCRD.enabled }}
|
||||
|
|
@ -200,6 +221,9 @@ spec:
|
|||
{{- if .Values.driver.kernelModuleConfig }}
|
||||
kernelModuleConfig: {{ toYaml .Values.driver.kernelModuleConfig | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.driver.secretEnv }}
|
||||
secretEnv: {{ .Values.driver.secretEnv }}
|
||||
{{- end }}
|
||||
{{- if .Values.driver.resources }}
|
||||
resources: {{ toYaml .Values.driver.resources | nindent 6 }}
|
||||
{{- end }}
|
||||
|
|
@ -232,6 +256,9 @@ spec:
|
|||
timeoutSeconds: {{ .Values.driver.upgradePolicy.drain.timeoutSeconds }}
|
||||
deleteEmptyDir: {{ .Values.driver.upgradePolicy.drain.deleteEmptyDir | default false}}
|
||||
{{- end }}
|
||||
{{- if .Values.driver.hostNetwork }}
|
||||
hostNetwork: {{ .Values.driver.hostNetwork }}
|
||||
{{- end }}
|
||||
vgpuManager:
|
||||
enabled: {{ .Values.vgpuManager.enabled }}
|
||||
{{- if .Values.vgpuManager.repository }}
|
||||
|
|
@ -258,6 +285,12 @@ spec:
|
|||
{{- if .Values.vgpuManager.args }}
|
||||
args: {{ toYaml .Values.vgpuManager.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.vgpuManager.kernelModuleConfig }}
|
||||
kernelModuleConfig: {{ toYaml .Values.vgpuManager.kernelModuleConfig | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.vgpuManager.hostNetwork }}
|
||||
hostNetwork: {{ .Values.vgpuManager.hostNetwork }}
|
||||
{{- end }}
|
||||
driverManager:
|
||||
{{- if .Values.vgpuManager.driverManager.repository }}
|
||||
repository: {{ .Values.vgpuManager.driverManager.repository }}
|
||||
|
|
@ -276,7 +309,9 @@ spec:
|
|||
{{- end }}
|
||||
kataManager:
|
||||
enabled: {{ .Values.kataManager.enabled }}
|
||||
{{- if .Values.kataManager.config }}
|
||||
config: {{ toYaml .Values.kataManager.config | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataManager.repository }}
|
||||
repository: {{ .Values.kataManager.repository }}
|
||||
{{- end }}
|
||||
|
|
@ -301,6 +336,9 @@ spec:
|
|||
{{- if .Values.kataManager.args }}
|
||||
args: {{ toYaml .Values.kataManager.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataManager.hostNetwork }}
|
||||
hostNetwork: {{ .Values.kataManager.hostNetwork }}
|
||||
{{- end }}
|
||||
vfioManager:
|
||||
enabled: {{ .Values.vfioManager.enabled }}
|
||||
{{- if .Values.vfioManager.repository }}
|
||||
|
|
@ -343,6 +381,9 @@ spec:
|
|||
{{- if .Values.vfioManager.driverManager.env }}
|
||||
env: {{ toYaml .Values.vfioManager.driverManager.env | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.vfioManager.hostNetwork }}
|
||||
hostNetwork: {{ .Values.vfioManager.hostNetwork }}
|
||||
{{- end }}
|
||||
vgpuDeviceManager:
|
||||
enabled: {{ .Values.vgpuDeviceManager.enabled }}
|
||||
{{- if .Values.vgpuDeviceManager.repository }}
|
||||
|
|
@ -372,6 +413,9 @@ spec:
|
|||
{{- if .Values.vgpuDeviceManager.config }}
|
||||
config: {{ toYaml .Values.vgpuDeviceManager.config | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.vgpuDeviceManager.hostNetwork }}
|
||||
hostNetwork: {{ .Values.vgpuDeviceManager.hostNetwork }}
|
||||
{{- end }}
|
||||
ccManager:
|
||||
enabled: {{ .Values.ccManager.enabled }}
|
||||
defaultMode: {{ .Values.ccManager.defaultMode | quote }}
|
||||
|
|
@ -394,11 +438,14 @@ spec:
|
|||
resources: {{ toYaml .Values.ccManager.resources | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.ccManager.env }}
|
||||
env: {{ toYaml .Values.vfioManager.env | nindent 6 }}
|
||||
env: {{ toYaml .Values.ccManager.env | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.ccManager.args }}
|
||||
args: {{ toYaml .Values.ccManager.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.ccManager.hostNetwork }}
|
||||
hostNetwork: {{ .Values.ccManager.hostNetwork }}
|
||||
{{- end }}
|
||||
toolkit:
|
||||
enabled: {{ .Values.toolkit.enabled }}
|
||||
{{- if .Values.toolkit.repository }}
|
||||
|
|
@ -425,6 +472,9 @@ spec:
|
|||
{{- if .Values.toolkit.installDir }}
|
||||
installDir: {{ .Values.toolkit.installDir }}
|
||||
{{- end }}
|
||||
{{- if .Values.toolkit.hostNetwork }}
|
||||
hostNetwork: {{ .Values.toolkit.hostNetwork }}
|
||||
{{- end }}
|
||||
devicePlugin:
|
||||
enabled: {{ .Values.devicePlugin.enabled }}
|
||||
{{- if .Values.devicePlugin.repository }}
|
||||
|
|
@ -453,8 +503,11 @@ spec:
|
|||
{{- end }}
|
||||
{{- if .Values.devicePlugin.config.name }}
|
||||
config:
|
||||
name: {{ .Values.devicePlugin.config.name }}
|
||||
default: {{ .Values.devicePlugin.config.default }}
|
||||
name: {{ .Values.devicePlugin.config.name | quote }}
|
||||
default: {{ .Values.devicePlugin.config.default | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.devicePlugin.hostNetwork }}
|
||||
hostNetwork: {{ .Values.devicePlugin.hostNetwork }}
|
||||
{{- end }}
|
||||
dcgm:
|
||||
enabled: {{ .Values.dcgm.enabled }}
|
||||
|
|
@ -482,6 +535,9 @@ spec:
|
|||
{{- if .Values.dcgm.args }}
|
||||
args: {{ toYaml .Values.dcgm.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dcgm.hostNetwork }}
|
||||
hostNetwork: {{ .Values.dcgm.hostNetwork }}
|
||||
{{- end }}
|
||||
dcgmExporter:
|
||||
enabled: {{ .Values.dcgmExporter.enabled }}
|
||||
{{- if .Values.dcgmExporter.repository }}
|
||||
|
|
@ -515,6 +571,18 @@ spec:
|
|||
{{- if .Values.dcgmExporter.serviceMonitor }}
|
||||
serviceMonitor: {{ toYaml .Values.dcgmExporter.serviceMonitor | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dcgmExporter.service }}
|
||||
service: {{ toYaml .Values.dcgmExporter.service | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dcgmExporter.hostPID }}
|
||||
hostPID: {{ .Values.dcgmExporter.hostPID }}
|
||||
{{- end }}
|
||||
{{- if .Values.dcgmExporter.hostNetwork }}
|
||||
hostNetwork: {{ .Values.dcgmExporter.hostNetwork }}
|
||||
{{- end }}
|
||||
{{- if .Values.dcgmExporter.hpcJobMapping }}
|
||||
hpcJobMapping: {{ toYaml .Values.dcgmExporter.hpcJobMapping | nindent 6 }}
|
||||
{{- end }}
|
||||
gfd:
|
||||
enabled: {{ .Values.gfd.enabled }}
|
||||
{{- if .Values.gfd.repository }}
|
||||
|
|
@ -541,6 +609,9 @@ spec:
|
|||
{{- if .Values.gfd.args }}
|
||||
args: {{ toYaml .Values.gfd.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.gfd.hostNetwork }}
|
||||
hostNetwork: {{ .Values.gfd.hostNetwork }}
|
||||
{{- end }}
|
||||
migManager:
|
||||
enabled: {{ .Values.migManager.enabled }}
|
||||
{{- if .Values.migManager.repository }}
|
||||
|
|
@ -569,12 +640,17 @@ spec:
|
|||
{{- end }}
|
||||
{{- if .Values.migManager.config }}
|
||||
config:
|
||||
{{- if .Values.migManager.config.name }}
|
||||
name: {{ .Values.migManager.config.name }}
|
||||
default: {{ .Values.migManager.config.default }}
|
||||
{{- end }}
|
||||
default: {{ .Values.migManager.config.default | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.migManager.gpuClientsConfig }}
|
||||
gpuClientsConfig: {{ toYaml .Values.migManager.gpuClientsConfig | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.migManager.hostNetwork }}
|
||||
hostNetwork: {{ .Values.migManager.hostNetwork }}
|
||||
{{- end }}
|
||||
nodeStatusExporter:
|
||||
enabled: {{ .Values.nodeStatusExporter.enabled }}
|
||||
{{- if .Values.nodeStatusExporter.repository }}
|
||||
|
|
@ -599,7 +675,10 @@ spec:
|
|||
{{- if .Values.nodeStatusExporter.args }}
|
||||
args: {{ toYaml .Values.nodeStatusExporter.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.gds.enabled }}
|
||||
{{- if .Values.nodeStatusExporter.hostNetwork }}
|
||||
hostNetwork: {{ .Values.nodeStatusExporter.hostNetwork }}
|
||||
{{- end }}
|
||||
{{- if .Values.gds }}
|
||||
gds:
|
||||
enabled: {{ .Values.gds.enabled }}
|
||||
{{- if .Values.gds.repository }}
|
||||
|
|
@ -650,6 +729,9 @@ spec:
|
|||
{{- if .Values.sandboxWorkloads.defaultWorkload }}
|
||||
defaultWorkload: {{ .Values.sandboxWorkloads.defaultWorkload }}
|
||||
{{- end }}
|
||||
{{- if .Values.sandboxWorkloads.mode }}
|
||||
mode: {{ .Values.sandboxWorkloads.mode | quote }}
|
||||
{{- end }}
|
||||
sandboxDevicePlugin:
|
||||
{{- if .Values.sandboxDevicePlugin.enabled }}
|
||||
enabled: {{ .Values.sandboxDevicePlugin.enabled }}
|
||||
|
|
@ -678,3 +760,37 @@ spec:
|
|||
{{- if .Values.sandboxDevicePlugin.args }}
|
||||
args: {{ toYaml .Values.sandboxDevicePlugin.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.sandboxDevicePlugin.hostNetwork }}
|
||||
hostNetwork: {{ .Values.sandboxDevicePlugin.hostNetwork }}
|
||||
{{- end }}
|
||||
kataSandboxDevicePlugin:
|
||||
{{- if ne .Values.kataSandboxDevicePlugin.enabled nil }}
|
||||
enabled: {{ .Values.kataSandboxDevicePlugin.enabled }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.repository }}
|
||||
repository: {{ .Values.kataSandboxDevicePlugin.repository }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.image }}
|
||||
image: {{ .Values.kataSandboxDevicePlugin.image }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.version }}
|
||||
version: {{ .Values.kataSandboxDevicePlugin.version | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.imagePullPolicy }}
|
||||
imagePullPolicy: {{ .Values.kataSandboxDevicePlugin.imagePullPolicy }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.imagePullSecrets }}
|
||||
imagePullSecrets: {{ toYaml .Values.kataSandboxDevicePlugin.imagePullSecrets | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.resources }}
|
||||
resources: {{ toYaml .Values.kataSandboxDevicePlugin.resources | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.env }}
|
||||
env: {{ toYaml .Values.kataSandboxDevicePlugin.env | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.args }}
|
||||
args: {{ toYaml .Values.kataSandboxDevicePlugin.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.kataSandboxDevicePlugin.hostNetwork }}
|
||||
hostNetwork: {{ .Values.kataSandboxDevicePlugin.hostNetwork }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,8 @@
|
|||
{{ range .Values.extraObjects }}
|
||||
---
|
||||
{{ if typeIs "string" . }}
|
||||
{{- tpl . $ }}
|
||||
{{- else }}
|
||||
{{- tpl (toYaml .) $ }}
|
||||
{{- end }}
|
||||
{{ end }}
|
||||
|
|
@ -2,106 +2,32 @@
|
|||
apiVersion: nfd.k8s-sigs.io/v1alpha1
|
||||
kind: NodeFeatureRule
|
||||
metadata:
|
||||
name: nvidia-nfd-nodefeaturerules
|
||||
name: nvidia-kernel-modules
|
||||
spec:
|
||||
rules:
|
||||
- name: "TDX rule"
|
||||
- name: kernel-module-gdrdrv
|
||||
labels:
|
||||
tdx.enabled: "true"
|
||||
nvidia.com/gdrcopy.capable: "true"
|
||||
matchFeatures:
|
||||
- feature: cpu.security
|
||||
- feature: kernel.loadedmodule
|
||||
matchExpressions:
|
||||
tdx.enabled: {op: IsTrue}
|
||||
- name: "TDX total keys rule"
|
||||
extendedResources:
|
||||
tdx.total_keys: "@cpu.security.tdx.total_keys"
|
||||
gdrdrv:
|
||||
op: Exists
|
||||
- name: kernel-module-nvidia_fs
|
||||
labels:
|
||||
nvidia.com/gds.capable: "true"
|
||||
matchFeatures:
|
||||
- feature: cpu.security
|
||||
- feature: kernel.loadedmodule
|
||||
matchExpressions:
|
||||
tdx.enabled: {op: IsTrue}
|
||||
- name: "SEV-SNP rule"
|
||||
nvidia_fs:
|
||||
op: Exists
|
||||
- name: kernel-module-nvidia_peermem
|
||||
labels:
|
||||
sev.snp.enabled: "true"
|
||||
nvidia.com/peermem.capable: "true"
|
||||
matchFeatures:
|
||||
- feature: cpu.security
|
||||
matchExpressions:
|
||||
sev.snp.enabled:
|
||||
op: IsTrue
|
||||
- name: "SEV-ES rule"
|
||||
labels:
|
||||
sev.es.enabled: "true"
|
||||
matchFeatures:
|
||||
- feature: cpu.security
|
||||
matchExpressions:
|
||||
sev.es.enabled:
|
||||
op: IsTrue
|
||||
- name: SEV system capacities
|
||||
extendedResources:
|
||||
sev_asids: '@cpu.security.sev.asids'
|
||||
sev_es: '@cpu.security.sev.encrypted_state_ids'
|
||||
matchFeatures:
|
||||
- feature: cpu.security
|
||||
matchExpressions:
|
||||
sev.enabled:
|
||||
op: Exists
|
||||
- name: "NVIDIA H100"
|
||||
labels:
|
||||
"nvidia.com/gpu.H100": "true"
|
||||
"nvidia.com/gpu.family": "hopper"
|
||||
matchFeatures:
|
||||
- feature: pci.device
|
||||
- feature: kernel.loadedmodule
|
||||
matchExpressions:
|
||||
vendor: {op: In, value: ["10de"]}
|
||||
device: {op: In, value: ["2339"]}
|
||||
- name: "NVIDIA H100 PCIe"
|
||||
labels:
|
||||
"nvidia.com/gpu.H100.pcie": "true"
|
||||
"nvidia.com/gpu.family": "hopper"
|
||||
matchFeatures:
|
||||
- feature: pci.device
|
||||
matchExpressions:
|
||||
vendor: {op: In, value: ["10de"]}
|
||||
device: {op: In, value: ["2331"]}
|
||||
- name: "NVIDIA H100 80GB HBM3"
|
||||
labels:
|
||||
"nvidia.com/gpu.H100.HBM3": "true"
|
||||
"nvidia.com/gpu.family": "hopper"
|
||||
matchFeatures:
|
||||
- feature: pci.device
|
||||
matchExpressions:
|
||||
vendor: {op: In, value: ["10de"]}
|
||||
device: {op: In, value: ["2330"]}
|
||||
- name: "NVIDIA H800"
|
||||
labels:
|
||||
"nvidia.com/gpu.H800": "true"
|
||||
"nvidia.com/gpu.family": "hopper"
|
||||
matchFeatures:
|
||||
- feature: pci.device
|
||||
matchExpressions:
|
||||
vendor: {op: In, value: ["10de"]}
|
||||
device: {op: In, value: ["2324"]}
|
||||
- name: "NVIDIA H800 PCIE"
|
||||
labels:
|
||||
"nvidia.com/gpu.H800.pcie": "true"
|
||||
"nvidia.com/gpu.family": "hopper"
|
||||
matchFeatures:
|
||||
- feature: pci.device
|
||||
matchExpressions:
|
||||
vendor: {op: In, value: ["10de"]}
|
||||
device: {op: In, value: ["2322"]}
|
||||
- name: "NVIDIA CC Enabled"
|
||||
labels:
|
||||
"nvidia.com/cc.capable": "true"
|
||||
matchAny: # TDX/SEV + Hopper GPU
|
||||
- matchFeatures:
|
||||
- feature: rule.matched
|
||||
matchExpressions:
|
||||
nvidia.com/gpu.family: {op: In, value: ["hopper"]}
|
||||
sev.snp.enabled: {op: IsTrue}
|
||||
- matchFeatures:
|
||||
- feature: rule.matched
|
||||
matchExpressions:
|
||||
nvidia.com/gpu.family: {op: In, value: ["hopper"]}
|
||||
tdx.enabled: {op: IsTrue}
|
||||
nvidia_peermem:
|
||||
op: Exists
|
||||
{{- end }}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
{{- if .Values.driver.enabled }}
|
||||
{{- if and .Values.driver.nvidiaDriverCRD.enabled .Values.driver.nvidiaDriverCRD.deployDefaultCR }}
|
||||
apiVersion: nvidia.com/v1alpha1
|
||||
kind: NVIDIADriver
|
||||
|
|
@ -48,7 +49,11 @@ spec:
|
|||
certConfig:
|
||||
name: {{ .Values.driver.certConfig.name }}
|
||||
{{- end }}
|
||||
{{- if .Values.driver.licensingConfig.configMapName }}
|
||||
{{- if .Values.driver.licensingConfig.secretName }}
|
||||
licensingConfig:
|
||||
secretName: {{ .Values.driver.licensingConfig.secretName }}
|
||||
nlsEnabled: {{ .Values.driver.licensingConfig.nlsEnabled | default true }}
|
||||
{{- else if .Values.driver.licensingConfig.configMapName }}
|
||||
licensingConfig:
|
||||
name: {{ .Values.driver.licensingConfig.configMapName }}
|
||||
nlsEnabled: {{ .Values.driver.licensingConfig.nlsEnabled | default true }}
|
||||
|
|
@ -61,6 +66,9 @@ spec:
|
|||
kernelModuleConfig:
|
||||
name: {{ .Values.driver.kernelModuleConfig.name }}
|
||||
{{- end }}
|
||||
{{- if .Values.driver.secretEnv }}
|
||||
secretEnv: {{ .Values.driver.secretEnv }}
|
||||
{{- end }}
|
||||
{{- if .Values.driver.resources }}
|
||||
resources: {{ toYaml .Values.driver.resources | nindent 6 }}
|
||||
{{- end }}
|
||||
|
|
@ -70,6 +78,9 @@ spec:
|
|||
{{- if .Values.driver.args }}
|
||||
args: {{ toYaml .Values.driver.args | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.driver.hostNetwork }}
|
||||
hostNetwork: {{ .Values.driver.hostNetwork }}
|
||||
{{- end }}
|
||||
{{- if .Values.gds.enabled }}
|
||||
gds:
|
||||
enabled: {{ .Values.gds.enabled }}
|
||||
|
|
@ -117,3 +128,4 @@ spec:
|
|||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ apiVersion: apps/v1
|
|||
kind: Deployment
|
||||
metadata:
|
||||
name: gpu-operator
|
||||
namespace: {{ .Release.Namespace }}
|
||||
labels:
|
||||
{{- include "gpu-operator.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: "gpu-operator"
|
||||
|
|
@ -58,10 +59,6 @@ spec:
|
|||
fieldPath: metadata.namespace
|
||||
- name: "DRIVER_MANAGER_IMAGE"
|
||||
value: "{{ include "driver-manager.fullimage" . }}"
|
||||
volumeMounts:
|
||||
- name: host-os-release
|
||||
mountPath: "/host-etc/os-release"
|
||||
readOnly: true
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
|
|
@ -81,10 +78,6 @@ spec:
|
|||
ports:
|
||||
- name: metrics
|
||||
containerPort: 8080
|
||||
volumes:
|
||||
- name: host-os-release
|
||||
hostPath:
|
||||
path: "/etc/os-release"
|
||||
{{- with .Values.operator.nodeSelector }}
|
||||
nodeSelector:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ apiVersion: rbac.authorization.k8s.io/v1
|
|||
kind: Role
|
||||
metadata:
|
||||
name: gpu-operator
|
||||
namespace: {{ .Release.Namespace }}
|
||||
labels:
|
||||
{{- include "gpu-operator.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: "gpu-operator"
|
||||
|
|
@ -82,3 +83,13 @@ rules:
|
|||
- watch
|
||||
- update
|
||||
- delete
|
||||
- apiGroups:
|
||||
- "nfd.k8s-sigs.io"
|
||||
resources:
|
||||
- "nodefeatures"
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- create
|
||||
- update
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ kind: RoleBinding
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
metadata:
|
||||
name: gpu-operator
|
||||
namespace: {{ .Release.Namespace }}
|
||||
labels:
|
||||
{{- include "gpu-operator.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: "gpu-operator"
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ apiVersion: v1
|
|||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: gpu-operator
|
||||
namespace: {{ .Release.Namespace }}
|
||||
labels:
|
||||
{{- include "gpu-operator.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: "gpu-operator"
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ apiVersion: v1
|
|||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: gpu-operator-upgrade-crd-hook-sa
|
||||
namespace: {{ .Release.Namespace }}
|
||||
annotations:
|
||||
helm.sh/hook: pre-upgrade
|
||||
helm.sh/hook-delete-policy: hook-succeeded,before-hook-creation
|
||||
|
|
@ -83,13 +84,13 @@ spec:
|
|||
image: {{ include "gpu-operator.fullimage" . }}
|
||||
imagePullPolicy: {{ .Values.operator.imagePullPolicy }}
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- >
|
||||
kubectl apply -f /opt/gpu-operator/nvidia.com_clusterpolicies.yaml;
|
||||
kubectl apply -f /opt/gpu-operator/nvidia.com_nvidiadrivers.yaml;
|
||||
- /usr/bin/manage-crds
|
||||
args:
|
||||
- apply
|
||||
- --filepath=/opt/gpu-operator/nvidia.com_clusterpolicies.yaml
|
||||
- --filepath=/opt/gpu-operator/nvidia.com_nvidiadrivers.yaml
|
||||
{{- if .Values.nfd.enabled }}
|
||||
kubectl apply -f /opt/gpu-operator/nfd-api-crds.yaml;
|
||||
- --filepath=/opt/gpu-operator/nfd-api-crds.yaml
|
||||
{{- end }}
|
||||
restartPolicy: OnFailure
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
{{- if and (eq .Values.cdi.enabled false) (eq .Values.cdi.nriPluginEnabled true) }}
|
||||
{{ fail "the NRI Plugin cannot be enabled when CDI is disabled" }}
|
||||
{{- end}}
|
||||
|
|
@ -13,12 +13,14 @@ psa:
|
|||
enabled: false
|
||||
|
||||
cdi:
|
||||
enabled: false
|
||||
default: false
|
||||
enabled: true
|
||||
nriPluginEnabled: false
|
||||
|
||||
sandboxWorkloads:
|
||||
enabled: false
|
||||
defaultWorkload: "container"
|
||||
# Sandbox mode: "kubevirt" (default) or "kata". When "kata", the Kata device plugin is deployed on vm-passthrough nodes.
|
||||
mode: "kubevirt"
|
||||
|
||||
hostPaths:
|
||||
# rootFS represents the path to the root filesystem of the host.
|
||||
|
|
@ -50,8 +52,8 @@ daemonsets:
|
|||
maxUnavailable: "1"
|
||||
|
||||
validator:
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: gpu-operator-validator
|
||||
repository: nvcr.io/nvidia
|
||||
image: gpu-operator
|
||||
# If version is not specified, then default is to use chart.AppVersion
|
||||
#version: ""
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
|
@ -59,10 +61,9 @@ validator:
|
|||
env: []
|
||||
args: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
plugin:
|
||||
env:
|
||||
- name: WITH_WORKLOAD
|
||||
value: "false"
|
||||
env: []
|
||||
|
||||
operator:
|
||||
repository: nvcr.io/nvidia
|
||||
|
|
@ -79,16 +80,7 @@ operator:
|
|||
# upgrade CRD on chart upgrade, requires --disable-openapi-validation flag
|
||||
# to be passed during helm upgrade.
|
||||
upgradeCRD: true
|
||||
initContainer:
|
||||
image: cuda
|
||||
repository: nvcr.io/nvidia
|
||||
version: 12.8.1-base-ubi9
|
||||
imagePullPolicy: IfNotPresent
|
||||
tolerations:
|
||||
- key: "node-role.kubernetes.io/master"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
effect: "NoSchedule"
|
||||
- key: "node-role.kubernetes.io/control-plane"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
|
|
@ -98,12 +90,6 @@ operator:
|
|||
affinity:
|
||||
nodeAffinity:
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- weight: 1
|
||||
preference:
|
||||
matchExpressions:
|
||||
- key: "node-role.kubernetes.io/master"
|
||||
operator: In
|
||||
values: [""]
|
||||
- weight: 1
|
||||
preference:
|
||||
matchExpressions:
|
||||
|
|
@ -146,7 +132,7 @@ driver:
|
|||
usePrecompiled: false
|
||||
repository: nvcr.io/nvidia
|
||||
image: driver
|
||||
version: "570.124.06"
|
||||
version: "580.126.20"
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
startupProbe:
|
||||
|
|
@ -192,25 +178,13 @@ driver:
|
|||
timeoutSeconds: 300
|
||||
deleteEmptyDir: false
|
||||
manager:
|
||||
image: k8s-driver-manager
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-driver-manager
|
||||
# When choosing a different version of k8s-driver-manager, DO NOT downgrade to a version lower than v0.6.4
|
||||
# to ensure k8s-driver-manager stays compatible with gpu-operator starting from v24.3.0
|
||||
version: v0.8.0
|
||||
version: v0.10.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
env:
|
||||
- name: ENABLE_GPU_POD_EVICTION
|
||||
value: "true"
|
||||
- name: ENABLE_AUTO_DRAIN
|
||||
value: "false"
|
||||
- name: DRAIN_USE_FORCE
|
||||
value: "false"
|
||||
- name: DRAIN_POD_SELECTOR_LABEL
|
||||
value: ""
|
||||
- name: DRAIN_TIMEOUT_SECONDS
|
||||
value: "0s"
|
||||
- name: DRAIN_DELETE_EMPTYDIR_DATA
|
||||
value: "false"
|
||||
env: []
|
||||
env: []
|
||||
resources: {}
|
||||
# Private mirror repository configuration
|
||||
|
|
@ -221,7 +195,7 @@ driver:
|
|||
name: ""
|
||||
# vGPU licensing configuration
|
||||
licensingConfig:
|
||||
configMapName: ""
|
||||
secretName: ""
|
||||
nlsEnabled: true
|
||||
# vGPU topology daemon configuration
|
||||
virtualTopology:
|
||||
|
|
@ -229,39 +203,31 @@ driver:
|
|||
# kernel module configuration for NVIDIA driver
|
||||
kernelModuleConfig:
|
||||
name: ""
|
||||
# Name of Kubernetes Secret which contains secrets to be passed in as environment variables
|
||||
secretEnv: ""
|
||||
hostNetwork: false
|
||||
|
||||
toolkit:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia/k8s
|
||||
image: container-toolkit
|
||||
version: v1.17.5-ubuntu20.04
|
||||
version: v1.19.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env: []
|
||||
resources: {}
|
||||
installDir: "/usr/local/nvidia"
|
||||
hostNetwork: false
|
||||
|
||||
devicePlugin:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia
|
||||
image: k8s-device-plugin
|
||||
version: v0.17.1
|
||||
version: v0.19.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
args: []
|
||||
env:
|
||||
- name: PASS_DEVICE_SPECS
|
||||
value: "true"
|
||||
- name: FAIL_ON_INIT_ERROR
|
||||
value: "true"
|
||||
- name: DEVICE_LIST_STRATEGY
|
||||
value: envvar
|
||||
- name: DEVICE_ID_STRATEGY
|
||||
value: uuid
|
||||
- name: NVIDIA_VISIBLE_DEVICES
|
||||
value: all
|
||||
- name: NVIDIA_DRIVER_CAPABILITIES
|
||||
value: all
|
||||
env: []
|
||||
resources: {}
|
||||
# Plugin configuration
|
||||
# Use "name" to either point to an existing ConfigMap or to create a new one with a list of configurations(i.e with create=true).
|
||||
|
|
@ -296,6 +262,7 @@ devicePlugin:
|
|||
mps:
|
||||
# MPS root path on the host
|
||||
root: "/run/nvidia/mps"
|
||||
hostNetwork: false
|
||||
|
||||
# standalone dcgm hostengine
|
||||
dcgm:
|
||||
|
|
@ -303,26 +270,30 @@ dcgm:
|
|||
enabled: false
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: dcgm
|
||||
version: 4.1.1-2-ubuntu22.04
|
||||
version: 4.5.2-1-ubuntu22.04
|
||||
imagePullPolicy: IfNotPresent
|
||||
args: []
|
||||
env: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
|
||||
dcgmExporter:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia/k8s
|
||||
image: dcgm-exporter
|
||||
version: 4.1.1-4.0.4-ubuntu22.04
|
||||
version: 4.5.1-4.8.0-distroless
|
||||
imagePullPolicy: IfNotPresent
|
||||
env:
|
||||
- name: DCGM_EXPORTER_LISTEN
|
||||
value: ":9400"
|
||||
- name: DCGM_EXPORTER_KUBERNETES
|
||||
value: "true"
|
||||
- name: DCGM_EXPORTER_COLLECTORS
|
||||
value: "/etc/dcgm-exporter/dcp-metrics-included.csv"
|
||||
env: []
|
||||
resources: {}
|
||||
hostPID: false
|
||||
hostNetwork: false
|
||||
# HPC job mapping configuration for correlating GPU metrics with HPC workload manager jobs
|
||||
# This is used by HPC workload managers like Slurm to label GPU metrics with job IDs
|
||||
# hpcJobMapping:
|
||||
# enabled: true
|
||||
# directory: /var/lib/dcgm-exporter/job-mapping
|
||||
service:
|
||||
internalTrafficPolicy: Cluster
|
||||
serviceMonitor:
|
||||
enabled: false
|
||||
interval: 15s
|
||||
|
|
@ -359,31 +330,35 @@ gfd:
|
|||
enabled: true
|
||||
repository: nvcr.io/nvidia
|
||||
image: k8s-device-plugin
|
||||
version: v0.17.1
|
||||
version: v0.19.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env:
|
||||
- name: GFD_SLEEP_INTERVAL
|
||||
value: 60s
|
||||
- name: GFD_FAIL_ON_INIT_ERROR
|
||||
value: "true"
|
||||
env: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
|
||||
migManager:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-mig-manager
|
||||
version: v0.12.1-ubuntu20.04
|
||||
version: v0.14.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env:
|
||||
- name: WITH_REBOOT
|
||||
value: "false"
|
||||
env: []
|
||||
resources: {}
|
||||
# MIG configuration
|
||||
# Use "name" to either point to an existing ConfigMap or to create a new one with a list of configurations(i.e with create=true).
|
||||
# Use "data" to build an integrated ConfigMap from a set of configurations as
|
||||
# part of this helm chart. An example of setting "data" might be:
|
||||
# NOTE: MIG manager automatically generates configuration from hardware on each node.
|
||||
# Only provide a custom config if you need settings that differ from hardware discovery.
|
||||
#
|
||||
# To use an existing ConfigMap:
|
||||
# - Set name="your-configmap-name" with create=false
|
||||
# - ConfigMap MUST have a key named "config.yaml"
|
||||
#
|
||||
# To create a new ConfigMap via Helm:
|
||||
# - Set create=true, name="your-configmap-name", and provide data below
|
||||
# - If create=true but data is empty, ConfigMap creation is skipped
|
||||
#
|
||||
# Example of creating a custom ConfigMap:
|
||||
# config:
|
||||
# name: custom-mig-parted-configs
|
||||
# create: true
|
||||
|
|
@ -415,28 +390,32 @@ migManager:
|
|||
default: "all-disabled"
|
||||
# Create a ConfigMap (default: false)
|
||||
create: false
|
||||
# ConfigMap name (either existing or to create a new one with create=true above)
|
||||
# ConfigMap name (either existing or to create with create=true)
|
||||
# If name is provided, mig-manager will use this config instead of auto-generated one.
|
||||
# REQUIREMENT: Custom ConfigMaps must contain a key named "config.yaml"
|
||||
name: ""
|
||||
# Data section for the ConfigMap to create (i.e only applies when create=true)
|
||||
# Data section for the ConfigMap (required only if create=true)
|
||||
data: {}
|
||||
gpuClientsConfig:
|
||||
name: ""
|
||||
hostNetwork: false
|
||||
|
||||
nodeStatusExporter:
|
||||
enabled: false
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: gpu-operator-validator
|
||||
repository: nvcr.io/nvidia
|
||||
image: gpu-operator
|
||||
# If version is not specified, then default is to use chart.AppVersion
|
||||
#version: ""
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
|
||||
gds:
|
||||
enabled: false
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: nvidia-fs
|
||||
version: "2.20.5"
|
||||
version: "2.27.3"
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env: []
|
||||
|
|
@ -446,7 +425,7 @@ gdrcopy:
|
|||
enabled: false
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: gdrdrv
|
||||
version: "v2.4.4"
|
||||
version: "v2.5.2"
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env: []
|
||||
|
|
@ -462,102 +441,100 @@ vgpuManager:
|
|||
env: []
|
||||
resources: {}
|
||||
driverManager:
|
||||
image: k8s-driver-manager
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-driver-manager
|
||||
# When choosing a different version of k8s-driver-manager, DO NOT downgrade to a version lower than v0.6.4
|
||||
# to ensure k8s-driver-manager stays compatible with gpu-operator starting from v24.3.0
|
||||
version: v0.8.0
|
||||
version: v0.10.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
env:
|
||||
- name: ENABLE_GPU_POD_EVICTION
|
||||
value: "false"
|
||||
- name: ENABLE_AUTO_DRAIN
|
||||
value: "false"
|
||||
env: []
|
||||
# kernel module configuration for vGPU manager
|
||||
kernelModuleConfig:
|
||||
name: ""
|
||||
hostNetwork: false
|
||||
|
||||
vgpuDeviceManager:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: vgpu-device-manager
|
||||
version: v0.3.0
|
||||
version: v0.4.2
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env: []
|
||||
config:
|
||||
name: ""
|
||||
default: "default"
|
||||
hostNetwork: false
|
||||
|
||||
vfioManager:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia
|
||||
image: cuda
|
||||
version: 12.8.1-base-ubi9
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-driver-manager
|
||||
version: v0.10.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env: []
|
||||
resources: {}
|
||||
driverManager:
|
||||
image: k8s-driver-manager
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-driver-manager
|
||||
# When choosing a different version of k8s-driver-manager, DO NOT downgrade to a version lower than v0.6.4
|
||||
# to ensure k8s-driver-manager stays compatible with gpu-operator starting from v24.3.0
|
||||
version: v0.8.0
|
||||
version: v0.10.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
env:
|
||||
- name: ENABLE_GPU_POD_EVICTION
|
||||
value: "false"
|
||||
- name: ENABLE_AUTO_DRAIN
|
||||
value: "false"
|
||||
env: []
|
||||
hostNetwork: false
|
||||
|
||||
kataManager:
|
||||
enabled: false
|
||||
config:
|
||||
artifactsDir: "/opt/nvidia-gpu-operator/artifacts/runtimeclasses"
|
||||
runtimeClasses:
|
||||
- name: kata-nvidia-gpu
|
||||
nodeSelector: {}
|
||||
artifacts:
|
||||
url: nvcr.io/nvidia/cloud-native/kata-gpu-artifacts:ubuntu22.04-535.54.03
|
||||
pullSecret: ""
|
||||
- name: kata-nvidia-gpu-snp
|
||||
nodeSelector:
|
||||
"nvidia.com/cc.capable": "true"
|
||||
artifacts:
|
||||
url: nvcr.io/nvidia/cloud-native/kata-gpu-artifacts:ubuntu22.04-535.86.10-snp
|
||||
pullSecret: ""
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-kata-manager
|
||||
version: v0.2.3
|
||||
config: {}
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
|
||||
sandboxDevicePlugin:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia
|
||||
image: kubevirt-gpu-device-plugin
|
||||
version: v1.3.1
|
||||
version: v1.5.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
args: []
|
||||
env: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
|
||||
ccManager:
|
||||
enabled: false
|
||||
defaultMode: "off"
|
||||
# Kata sandbox device plugin (used when sandboxWorkloads.mode is "kata").
|
||||
kataSandboxDevicePlugin:
|
||||
enabled: true
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-cc-manager
|
||||
version: v0.1.1
|
||||
image: nvidia-sandbox-device-plugin
|
||||
version: "v0.0.3"
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
env:
|
||||
- name: CC_CAPABLE_DEVICE_IDS
|
||||
value: "0x2339,0x2331,0x2330,0x2324,0x2322,0x233d"
|
||||
args: []
|
||||
env: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
|
||||
ccManager:
|
||||
enabled: true
|
||||
defaultMode: "on"
|
||||
repository: nvcr.io/nvidia/cloud-native
|
||||
image: k8s-cc-manager
|
||||
version: v0.4.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
imagePullSecrets: []
|
||||
resources: {}
|
||||
hostNetwork: false
|
||||
|
||||
# Array of extra K8s manifests to deploy
|
||||
# Supports use of custom Helm templates
|
||||
extraObjects: []
|
||||
|
||||
node-feature-discovery:
|
||||
enableNodeFeatureApi: true
|
||||
priorityClassName: system-node-critical
|
||||
gc:
|
||||
enable: true
|
||||
|
|
@ -571,10 +548,6 @@ node-feature-discovery:
|
|||
# disable creation to avoid duplicate serviceaccount creation by master spec below
|
||||
create: false
|
||||
tolerations:
|
||||
- key: "node-role.kubernetes.io/master"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
effect: "NoSchedule"
|
||||
- key: "node-role.kubernetes.io/control-plane"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
|
|
|
|||
16
packages/system/gpu-operator/values-talos-vgpu.yaml
Normal file
16
packages/system/gpu-operator/values-talos-vgpu.yaml
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
gpu-operator:
|
||||
sandboxWorkloads:
|
||||
enabled: true
|
||||
driver:
|
||||
enabled: false
|
||||
devicePlugin:
|
||||
enabled: false
|
||||
vgpuManager:
|
||||
enabled: true
|
||||
repository: ""
|
||||
image: vgpu-manager
|
||||
version: ""
|
||||
vgpuDeviceManager:
|
||||
enabled: true
|
||||
config:
|
||||
default: default
|
||||
Loading…
Add table
Add a link
Reference in a new issue