HAMi Enterprise 2.9.0-r3

Chart version	Declared appVersion	Enterprise release date	Chart package date
2.9.0-r3	2.9.0-r3	2026-06-26	2026-06-29

This is a patch release. Changes below are compared with HAMi Enterprise v2.9.0-rc2. The package is hami-enterprise-2.9.0-r3.tgz.

Software changes

• Fix NVIDIA runtime-library mount order. Mount the container cache directory before libvgpu.so to prevent the directory mount from hiding the library file and causing preload failure.
• Set the shared cache environment variable for the Downward API path. When device configuration sets nvidia.useDownward: true, inject CUDA_DEVICE_MEMORY_SHARED_CACHE into GPU workload containers. Its value identifies the cache file by Pod namespace, Pod name, and container name. The Chart's device configuration enables the Downward API path by default.
• Add NVIDIA device-allocation annotations to init containers. The annotation phase no longer skips init containers; regular containers continue to be matched after accounting for the number of init containers. This describes an annotation fix only. The webhook still iterates over regular containers, so this does not mean that the complete GPU workflow for init containers is supported.
• Fix the scheduler using stale license state after an update. License validity and expiration changes are now reflected in scheduler state even when node device registration information has not changed.
• Add Kubernetes events related to licensing. Events associated with the license Secret can show license loading failures, expiration, or invalidity, helping diagnose why devices are unavailable for allocation.

Chart changes

Scheduler name and component labels

The default schedulerName is restored from hami-enterprise-scheduler to hami-scheduler. The Chart is still named hami-enterprise; this does not mean Kubernetes resource object names also revert to their previous values.

Label	2.9.0-rc2	2.9.0-r3
Scheduler app.kubernetes.io/component	hami-enterprise-scheduler	hami-scheduler
Device plugin app.kubernetes.io/component	hami-enterprise-device-plugin	hami-device-plugin
app.kubernetes.io/name for both components	Defaults to hami-enterprise	Fixed to hami
app.kubernetes.io/instance for both components	Helm release name	Fixed to hami

Component selectors use these fixed labels. Scripts that select components by Helm release name must be updated; in particular, do not assume that app.kubernetes.io/instance matches the actual release name.

Custom values validation

Fixes the 2.9.0-rc2 issue that restricted the following settings to null; they now accept their respective types and defaults:

Setting	Type	Default
devicePlugin.nvidiaDriverRoot, devicePlugin.nvidiaHookPath	String	""
devicePlugin.gdrcopyEnabled, devicePlugin.gdsEnabled, devicePlugin.mofedEnabled	Boolean	false
scheduler.admissionWebhook.whitelistNamespaces	List	[]
scheduler.admissionWebhook.namespaceSelector.matchLabels	Map	{}

Driver paths, GDRCopy, GDS, OFED, and namespace filtering were already supported; this release fixes their type validation and template conditions. A boolean set to false does not inject the corresponding enablement environment variable, and an empty driver path does not render a mount for that driver directory.

The default global.imageTag is updated to v2.9.0-r3. The default repositories for the scheduler extender, NVIDIA device plugin, and monitor container remain ghcr.io/dynamia-ai/hami-enterprise.

Upgrade notes

When upgrading from 2.9.0-rc2, account for selector changes on the scheduler Deployment and device-plugin DaemonSet. Kubernetes does not allow in-place changes to .spec.selector on these resources. With default values, resource names are unchanged, but their selectors have changed; a direct in-place upgrade will fail because the field is immutable.

Before upgrading, save the current values and resource manifests. Render the target Chart with the actual release name, namespace, and custom values, then identify the affected resource names. Schedule recreation of both resources during a maintenance window and apply the new Chart. Then check scheduler readiness, device-plugin registration, and allocation for newly created GPU workloads. Do not recreate only the Deployment and overlook the DaemonSet, or apply deletion commands with fixed names before verifying the actual names.

Also check these integrations:

• Scheduler name: Update workload templates that explicitly use hami-enterprise-scheduler to use hami-scheduler, or explicitly retain the old name through the Chart's schedulerName so both sides match.
• Label selectors: Update custom monitoring, inspection, and Service selectors to use the target labels in the table. For multiple installations, distinguish instances by their actual namespaces and resource names as well.
• Custom values: Migrate old null values to the types in the table, and ensure that boolean switches are set to true where required. Validate merged values before installation so reusing old values does not cause validation or rendering surprises.
• Device-bound licenses: This release changes the requirements for device-bound licenses. Before upgrading, deployments using this license type should confirm with the delivery provider and prepare a license file compatible with the target version to avoid devices becoming unavailable for new workload allocation.
• NVIDIA workloads: The shared-cache environment variable and mount fixes apply to newly created containers. Existing containers are not reinjected automatically when management components are upgraded.

Obtain the package and license files through the agreed delivery channel. See the Chart and appVersion mapping or the software center.

Values reference

The following parameter descriptions are taken from the README bundled with the Chart 2.9.0-r3 release package and are preserved as published. Use this version's values.yaml as the source of default configuration.

Expand values table

Key	Type	Default	Description
dcuResourceCores	string	"hygon.com/dcucores"	Hygon DCU cores resource name
dcuResourceMem	string	"hygon.com/dcumem"	Hygon DCU memory resource name
dcuResourceName	string	"hygon.com/dcunum"	Hygon DCU resource name
devicePlugin.createRuntimeClass	bool	false	Whether to create runtime class
devicePlugin.deviceCoreScaling	int	1	Device core scaling factor
devicePlugin.deviceListStrategy	string	"envvar"	Device list strategy. Supported values: envvar, volume-mounts
devicePlugin.deviceMemoryScaling	int	1	Device memory scaling factor
devicePlugin.deviceSplitCount	int	10	Number of slices to split each GPU into
devicePlugin.disablecorelimit	string	"false"	Disable core limit
devicePlugin.enabled	bool	true	Enable device plugin
devicePlugin.extraArgs	list	["-v=4"]	Extra arguments for device plugin
devicePlugin.extraEnvs	object	{}	Extra environment variables for device plugin
devicePlugin.gdrcopyEnabled	bool	false	Enable GDRCopy
devicePlugin.gdsEnabled	bool	false	Enable GDS
devicePlugin.gpuOperatorToolkitReady.enabled	bool	false	Enable GPU operator toolkit ready check
devicePlugin.gpuOperatorToolkitReady.hostPath	string	"/run/nvidia/validations"	Host path for GPU operator toolkit ready check
devicePlugin.image.pullPolicy	string	"IfNotPresent"	Device plugin image pull policy
devicePlugin.image.pullSecrets	list	[]	Specify docker-registry secret names as an array
devicePlugin.image.registry	string	"ghcr.io"	Device plugin image registry
devicePlugin.image.repository	string	"dynamia-ai/hami-enterprise"	Device plugin image repository
devicePlugin.image.tag	string	""	Device plugin image tag (immutable tags are recommended)
devicePlugin.libPath	string	"/usr/local/vgpu"	Library path for vGPU
devicePlugin.migStrategy	string	"none"	MIG strategy. Supported values: none, single, mixed
devicePlugin.mofedEnabled	bool	false	Enable Mellanox OFED
devicePlugin.monitor.ctrPath	string	"/usr/local/vgpu/containers"	Container path for monitor
devicePlugin.monitor.extraArgs	list	["-v=4"]	Extra arguments for monitor
devicePlugin.monitor.extraEnvs	object	{}	Extra environment variables for monitor
devicePlugin.monitor.image.pullPolicy	string	"IfNotPresent"	Monitor image pull policy
devicePlugin.monitor.image.pullSecrets	list	[]	Specify docker-registry secret names as an array
devicePlugin.monitor.image.registry	string	"ghcr.io"	Monitor image registry
devicePlugin.monitor.image.repository	string	"dynamia-ai/hami-enterprise"	Monitor image repository
devicePlugin.monitor.image.tag	string	""	Monitor image tag (immutable tags are recommended)
devicePlugin.monitor.resources	object	{}	Resources for monitor container
devicePlugin.monitor.resyncInterval	string	"5m"	Resync interval for monitor
devicePlugin.nodeConfiguration.config	string	"{\n \"nodeconfig\": [\n {\n \"name\": \"your-node-name\",\n \"operatingmode\": \"hami-core\",\n \"devicememoryscaling\": 1,\n \"devicesplitcount\": 10,\n \"preconfigureddevicememory\": 0,\n \"migstrategy\": \"none\",\n \"filterdevices\": {\n \"uuid\": [],\n \"index\": []\n },\n \"enablegetpreferredallocation\": false\n }\n ]\n}\n"	Node configuration for device plugin. If set, overrides the default config.json
devicePlugin.nodeConfiguration.externalConfigName	string	""	Name of an existing ConfigMap to use for node configuration
devicePlugin.nvidiaDriverRoot	string	""	Root path for NVIDIA driver
devicePlugin.nvidiaHookPath	string	""	Path for NVIDIA hook
devicePlugin.nvidiaNodeSelector	object	{"gpu":"on"}	Node selector for NVIDIA device plugin
devicePlugin.passDeviceSpecsEnabled	bool	false	Enable passing device specs
devicePlugin.pluginPath	string	"/var/lib/kubelet/device-plugins"	Path for device plugin socket
devicePlugin.podAnnotations	object	{}	Annotations for device plugin pod
devicePlugin.preConfiguredDeviceMemory	int	0	Pre-configured device memory in MB for GPUs that don't support memory query. Set to 0 for auto-detection
devicePlugin.resources	object	{}	Resources for device plugin container
devicePlugin.runtimeClassName	string	""	Runtime class name to be used by the device plugin
devicePlugin.service.annotations	object	{}	Annotations for device plugin service
devicePlugin.service.httpPort	int	31992	HTTP port for device plugin service
devicePlugin.service.labels	object	{}	Labels for device plugin service
devicePlugin.service.type	string	"NodePort"	Service type for device plugin
devicePlugin.tolerations	list	[]	Tolerations for device plugin pod
devicePlugin.updateStrategy.rollingUpdate.maxUnavailable	int	1	Maximum unavailable pods during update
devicePlugin.updateStrategy.type	string	"RollingUpdate"	Update strategy for device plugin DaemonSet
devices.alibaba.customresources	list	["alibaba.com/ppu","alibaba.com/ppu-memory","alibaba.com/gpu-core"]	Custom resource definitions for Alibaba devices
devices.alibaba.enabled	bool	true	Enable Alibaba PPU device
devices.alibaba.ppuCorePolicy	string	"default"	PPU core policy for Alibaba devices
devices.amd.customresources	list	["amd.com/gpu","amd.com/gpumem"]	Custom resource definitions for AMD devices
devices.ascend.customresources	list	["huawei.com/Ascend910A","huawei.com/Ascend910A-memory","huawei.com/Ascend910A-core","huawei.com/Ascend910B2","huawei.com/Ascend910B2-memory","huawei.com/Ascend910B2-core","huawei.com/Ascend910B3","huawei.com/Ascend910B3-memory","huawei.com/Ascend910B3-core","huawei.com/Ascend910B4","huawei.com/Ascend910B4-memory","huawei.com/Ascend910B4-core","huawei.com/Ascend910B4-1","huawei.com/Ascend910B4-1-memory","huawei.com/Ascend910B4-1-core","huawei.com/Ascend310P","huawei.com/Ascend310P-memory","huawei.com/Ascend310P-core","huawei.com/Ascend910C","huawei.com/Ascend910C-memory","huawei.com/Ascend910C-core"]	Custom resource definitions for Ascend devices
devices.ascend.enabled	bool	false	Enable Ascend device plugin
devices.ascend.extraArgs	list	[]	Extra arguments for Ascend device plugin
devices.ascend.hamiVnpuCore	bool	false	Enable soft-partitioning based on hami-vnpu-core (node-level annotation takes higher priority)
devices.ascend.image	string	""	Ascend device plugin image
devices.ascend.imagePullPolicy	string	"IfNotPresent"	Image pull policy for Ascend device plugin
devices.ascend.nodeSelector	object	{"ascend":"on"}	Node selector for Ascend device plugin
devices.ascend.runtimeClassName	string	""	Runtime class name for Ascend NPU pods
devices.ascend.tolerations	list	[]	Tolerations for Ascend device plugin
devices.awsneuron.customresources	list	["aws.amazon.com/neuron","aws.amazon.com/neuroncore"]	Custom resource definitions for AWS Neuron devices
devices.enflame.customresources	list	["enflame.com/vgcu","enflame.com/vgcu-percentage","enflame.com/gcu"]	Custom resource definitions for Enflame devices
devices.enflame.enabled	bool	true	Enable Enflame device plugin
devices.iluvatar.customresources	list	["iluvatar.ai/BI-V100-vgpu","iluvatar.ai/BI-V100.vCore","iluvatar.ai/BI-V100.vMem","iluvatar.ai/BI-V150-vgpu","iluvatar.ai/BI-V150.vCore","iluvatar.ai/BI-V150.vMem","iluvatar.ai/MR-V100-vgpu","iluvatar.ai/MR-V100.vCore","iluvatar.ai/MR-V100.vMem","iluvatar.ai/MR-V50-vgpu","iluvatar.ai/MR-V50.vCore","iluvatar.ai/MR-V50.vMem"]	Custom resource definitions for Iluvatar devices
devices.iluvatar.enabled	bool	false	Enable Iluvatar device plugin
devices.kunlun.customresources	list	["kunlunxin.com/xpu","kunlunxin.com/vxpu","kunlunxin.com/vxpu-memory"]	Custom resource definitions for Kunlun devices
devices.kunlun.enabled	bool	true	Enable Kunlun device plugin
devices.mthreads.customresources	list	["mthreads.com/vgpu"]	Custom resource definitions for MThreads devices
devices.mthreads.enabled	bool	true	Enable MThreads device plugin
devices.nvidia.gpuCorePolicy	string	"default"	GPU core policy for NVIDIA devices
devices.nvidia.libCudaLogLevel	int	2	libCUDA log level
devices.vastai.customresources	list	["vastaitech.com/va"]	Custom resource definitions for Vastai devices
devices.vastai.enabled	bool	true	Enable Vastai device plugin
dra.enabled	bool	false	Enable DRA (Dynamic Resource Allocation)
enflameResourceNameVGCU	string	"enflame.com/vgcu"	Enflame VGCU resource name
enflameResourceNameVGCUPercentage	string	"enflame.com/vgcu-percentage"	Enflame VGCU percentage resource name
fullnameOverride	string	""	Override the fullname of the chart
global.annotations	object	{}	Global annotations to add to all resources
global.gpuHookPath	string	"/usr/local"	Path for GPU hook installation on nodes
global.imagePullSecrets	list	[]	Global Docker image pull secrets
global.imageRegistry	string	""	Global Docker image registry
global.imageTag	string	"v2.9.0-r3"	Global image tag for HAMi Enterprise components
global.labels	object	{}	Global labels to add to all resources
global.managedNodeSelector	object	{"usage":"gpu"}	Node selector configuration for managed GPU nodes
global.managedNodeSelectorEnable	bool	false	Enable managed node selector for GPU nodes
hami-dra.drivers.nvidia.containerDriver	bool	true	Enable container driver for NVIDIA DRA
hami-dra.drivers.nvidia.enabled	bool	true	Enable NVIDIA DRA driver
hami-dra.drivers.nvidia.image.repository	string	"ghcr.io/project-hami/k8s-dra-driver"	NVIDIA DRA driver image repository
hami-dra.drivers.nvidia.image.tag	string	"main"	NVIDIA DRA driver image tag
hami-dra.monitor.enabled	bool	true	Enable monitor for hami-dra
kunlunResourceName	string	"kunlunxin.com/xpu"	Kunlun XPU resource name
kunlunResourceVCountName	string	"kunlunxin.com/vxpu"	Kunlun virtual XPU count resource name
kunlunResourceVMemoryName	string	"kunlunxin.com/vxpu-memory"	Kunlun virtual XPU memory resource name
legacyMetrics	bool	false	Enable legacy metrics
metaxResourceCore	string	"metax-tech.com/vcore"	Metax sGPU core resource name
metaxResourceMem	string	"metax-tech.com/vmemory"	Metax sGPU memory resource name
metaxResourceName	string	"metax-tech.com/sgpu"	Metax sGPU resource name
metaxsGPUTopologyAware	string	"false"	Enable Metax sGPU topology awareness
mluResourceCores	string	"cambricon.com/mlu.smlu.vcore"	MLU cores resource name
mluResourceMem	string	"cambricon.com/mlu.smlu.vmemory"	MLU memory resource name
mluResourceName	string	"cambricon.com/vmlu"	MLU resource name
mockDevicePlugin.enabled	bool	false	Enable mock device plugin
mockDevicePlugin.image.pullPolicy	string	"IfNotPresent"	Mock device plugin image pull policy
mockDevicePlugin.image.pullSecrets	list	[]	Specify docker-registry secret names as an array
mockDevicePlugin.image.registry	string	"docker.io"	Mock device plugin image registry
mockDevicePlugin.image.repository	string	"projecthami/mock-device-plugin"	Mock device plugin image repository
mockDevicePlugin.image.tag	string	"1.0.1"	Mock device plugin image tag
nameOverride	string	""	Override the chart name
namespaceOverride	string	""	Override the namespace
podSecurityPolicy.enabled	bool	false	Enable PodSecurityPolicy
prometheus.enabled	bool	false	Enable Prometheus metrics
resourceCores	string	"nvidia.com/gpucores"	NVIDIA GPU cores resource name
resourceMem	string	"nvidia.com/gpumem"	NVIDIA GPU memory resource name
resourceMemPercentage	string	"nvidia.com/gpumem-percentage"	NVIDIA GPU memory percentage resource name
resourceName	string	"nvidia.com/gpu"	NVIDIA GPU resource name
resourcePriority	string	"nvidia.com/priority"	NVIDIA GPU priority resource name
scheduler.admissionWebhook.customURL.enabled	bool	false	Enable custom webhook URL
scheduler.admissionWebhook.customURL.host	string	"127.0.0.1"	Webhook host (must be an endpoint using https)
scheduler.admissionWebhook.customURL.path	string	"/webhook"	Webhook path
scheduler.admissionWebhook.customURL.port	int	31998	Webhook port
scheduler.admissionWebhook.enabled	bool	true	Enable admission webhook. If disabled, pods must be configured with correct schedulerName manually
scheduler.admissionWebhook.failurePolicy	string	"Ignore"	Failure policy for webhook
scheduler.admissionWebhook.namespaceSelector	object	{"matchExpressions":[],"matchLabels":{}}	Namespace selector for webhook
scheduler.admissionWebhook.objectSelector	object	{"matchExpressions":[]}	Object selector for webhook
scheduler.admissionWebhook.reinvocationPolicy	string	"Never"	Reinvocation policy for webhook
scheduler.admissionWebhook.whitelistNamespaces	list	[]	Namespaces that the webhook will not be applied to
scheduler.certManager.enabled	bool	false	Enable cert-manager for TLS certificate generation
scheduler.defaultSchedulerPolicy.gpuSchedulerPolicy	string	"spread"	GPU-level scheduling policy. Supported values: binpack, spread
scheduler.defaultSchedulerPolicy.nodeSchedulerPolicy	string	"binpack"	Node-level scheduling policy. Supported values: binpack, spread
scheduler.extender.extraArgs	list	["--debug","-v=4"]	Extra arguments for scheduler extender
scheduler.extender.image.pullPolicy	string	"IfNotPresent"	Scheduler extender image pull policy
scheduler.extender.image.pullSecrets	list	[]	Specify docker-registry secret names as an array
scheduler.extender.image.registry	string	"ghcr.io"	Scheduler extender image registry
scheduler.extender.image.repository	string	"dynamia-ai/hami-enterprise"	Scheduler extender image repository
scheduler.extender.image.tag	string	""	Scheduler extender image tag (immutable tags are recommended)
scheduler.extender.resources	object	{}	Resources for scheduler extender container
scheduler.forceOverwriteDefaultScheduler	bool	true	Force overwrite the default scheduler name in pods when it equals the K8s default scheduler name
scheduler.kubeScheduler.enabled	bool	true	Whether to run kube-scheduler container in the scheduler pod
scheduler.kubeScheduler.extraArgs	list	["--policy-config-file=/config/config.json","-v=4"]	Extra arguments for kube-scheduler (legacy config format)
scheduler.kubeScheduler.extraNewArgs	list	["--config=/config/config.yaml","-v=4"]	Extra arguments for kube-scheduler (new config format)
scheduler.kubeScheduler.image.pullPolicy	string	"IfNotPresent"	kube-scheduler image pull policy
scheduler.kubeScheduler.image.pullSecrets	list	[]	Specify docker-registry secret names as an array
scheduler.kubeScheduler.image.registry	string	"registry.cn-hangzhou.aliyuncs.com"	kube-scheduler image registry
scheduler.kubeScheduler.image.repository	string	"google_containers/kube-scheduler"	kube-scheduler image repository
scheduler.kubeScheduler.image.tag	string	""	kube-scheduler image tag (immutable tags are recommended)
scheduler.kubeScheduler.resources	object	{}	Resources for kube-scheduler container
scheduler.leaderElect	bool	true	Enable leader election for scheduler
scheduler.livenessProbe	bool	false	Enable liveness probe for scheduler
scheduler.metricsBindAddress	string	":9395"	Address to bind metrics server
scheduler.nodeLockExpire	string	"5m"	Node lock expiration time
scheduler.nodeName	string	""	Node name for scheduler pod. If installing as default scheduler, set this to skip the schedule workflow
scheduler.overwriteEnv	string	"false"	Overwrite environment variables
scheduler.patch.enabled	bool	true	Enable kube-webhook-certgen patch job
scheduler.patch.image.pullPolicy	string	"IfNotPresent"	kube-webhook-certgen image pull policy
scheduler.patch.image.pullSecrets	list	[]	Specify docker-registry secret names as an array
scheduler.patch.image.registry	string	"docker.io"	kube-webhook-certgen image registry
scheduler.patch.image.repository	string	"jettech/kube-webhook-certgen"	kube-webhook-certgen image repository
scheduler.patch.image.tag	string	"v1.5.2"	kube-webhook-certgen image tag
scheduler.patch.imageNew.pullPolicy	string	"IfNotPresent"	New kube-webhook-certgen image pull policy
scheduler.patch.imageNew.pullSecrets	list	[]	Specify docker-registry secret names as an array
scheduler.patch.imageNew.registry	string	"docker.io"	New kube-webhook-certgen image registry
scheduler.patch.imageNew.repository	string	"liangjw/kube-webhook-certgen"	New kube-webhook-certgen image repository
scheduler.patch.imageNew.tag	string	"v1.1.1"	New kube-webhook-certgen image tag
scheduler.patch.nodeSelector	object	{}	Node selector for patch job
scheduler.patch.podAnnotations	object	{}	Annotations for patch job pod
scheduler.patch.priorityClassName	string	""	Priority class name for patch job
scheduler.patch.runAsUser	int	2000	User ID to run patch job as
scheduler.patch.tolerations	list	[]	Tolerations for patch job
scheduler.podAnnotations	object	{}	Annotations for scheduler pod
scheduler.replicas	int	1	Number of scheduler replicas (only effective when leaderElect is true)
scheduler.service.annotations	object	{}	Annotations for scheduler service
scheduler.service.httpPort	int	443	HTTP port for scheduler service
scheduler.service.httpTargetPort	int	443	Target port for HTTP
scheduler.service.labels	object	{}	Labels for scheduler service
scheduler.service.monitorPort	int	31993	Monitoring port for scheduler service
scheduler.service.monitorTargetPort	int	9395	Target port for monitoring
scheduler.service.schedulerPort	int	31998	NodePort for scheduler HTTP
scheduler.service.type	string	"NodePort"	Service type for scheduler
scheduler.tolerations	list	[]	Tolerations for scheduler pod
schedulerName	string	"hami-scheduler"	Name of the scheduler
vastaiResourceName	string	"vastaitech.com/va"	Vastai resource name