HAMi Enterprise 2.7.1-rc0
| Chart version | Declared appVersion | Release date |
|---|---|---|
2.7.1-rc0 | 2.7.1-rc0 | 2026-01-16 |
This is Dynamia's first HAMi Enterprise release and is a pre-release. This page uses open-source HAMi v2.7.0 as its baseline.
Software changes
Enterprise extensions compared with open-source HAMi v2.7.0
Pass NVIDIA device-allocation results through the Downward API. The scheduler writes GPU UUIDs, memory limits, and compute limits to Pod annotations and then passes them to the corresponding containers through the Kubernetes Downward API. The regular NVIDIA vGPU allocation path can therefore skip the step that relies on a node lock to find the Pod awaiting allocation. nvidia.useDownward is true in the default device configuration. The companion device plugin, Enterprise runtime library, and NVIDIA container runtime environment are still required.
Adjust the memory limit of a running container. The Enterprise monitor receives memory-adjustment requests reported by the runtime and submits them to the scheduler, which checks available device capacity. After the scheduler updates allocation records and quota usage, the monitor synchronizes the approved limit back to the runtime. This capability requires the companion Enterprise runtime, monitor, and scheduler. It does not modify the Pod's resources.limits, and it does not mean all workloads will expand automatically. The first release does not additionally check the namespace ResourceQuota limit when deciding whether to expand memory; do not rely on it as a hard quota boundary for runtime expansion.
Manage NVIDIA device licenses. The scheduler includes device-license validity in device selection, and the monitor synchronizes license updates. The scheduler metric vGPULicenseRemains reports the remaining license time in seconds and can be used to configure expiration alerts. Obtain license materials for installation and renewal through the agreed delivery channel.
Expose container memory details. In addition to total memory usage and limits, Enterprise monitoring provides the following metrics in bytes. These are current values and should be treated as Gauges:
| Metric | Meaning |
|---|---|
hami_container_device_memory_context_bytes | Context memory usage |
hami_container_device_memory_data_bytes | Data memory usage |
hami_container_device_memory_module_bytes | Module memory usage |
hami_container_device_memory_offset_bytes | Offset memory usage |
These metrics associate containers and devices using the podnamespace, podname, ctrname, vdeviceid, and deviceuuid labels. Use these exact label names in collection rules.
Upstream changes included in this release
This release also includes the following community changes made after open-source HAMi v2.7.0.
| Change | Impact on usage and integration | Upstream PR |
|---|---|---|
| Multi-model configuration for Iluvatar devices | Configure device count, memory, and compute resource names separately for BI-V100, BI-V150, MR-V100, and MR-V50. Requires the matching GPU Manager and devices.iluvatar.enabled: true (default false). For example, BI-V150 uses iluvatar.ai/BI-V150-vgpu, iluvatar.ai/BI-V150.vMem, and iluvatar.ai/BI-V150.vCore. | HAMi #1399 |
| Online/offline task classification for Metax sGPU | The Pod annotation metax-tech.com/sgpu-app-class accepts online or offline to distinguish device selection for these task types. When this annotation is set, each applicable container can request only one sGPU. Task execution priority still depends on the companion Metax components. | HAMi #1436 |
| AMD MI300X full-card resources | Requires a separate compatible AMD device plugin. HAMi uses the reported amd.com/gpu capacity to include full-card allocations in scheduler records and metrics. Memory accounting is fixed at 192000 MiB per card. These metrics represent scheduler allocation amounts, not real-time memory usage or utilization telemetry. This release does not provide AMD memory or compute partitioning. | HAMi #1472 |
| Mixed-size selection for dynamic MIG | When a partition profile includes multiple instance sizes, matches each request to a suitable instance, fixing cases where a small-memory request incorrectly occupied a larger instance. | HAMi #1518 |
| Quota usage cleanup | Reclaims quota usage when a Pod enters a terminating state. When a ResourceQuota is updated, clears the old configuration first so removed quota entries do not continue to impose stale limits. | HAMi #1400 |
| Track GPU counts per node | Tracks NVIDIA GPU counts reported by each node separately, preventing nodes from overwriting each other's device-count records. | HAMi #1555 |
| Node-lock annotation parsing | Returns a clear error when a node-lock annotation contains a delimiter but has an invalid number of fields; the legacy timestamp-only format remains parseable. The Enterprise default Downward API path does not depend on this node-lock allocation flow. | HAMi #1560 |
| Node configuration and selector fixes | Writes node configuration to the ConfigMap as JSON text and consistently uses devicePlugin.nvidiaNodeSelector as the NVIDIA node selector. Check key names when migrating old values. | HAMi #1558, HAMi #1546 |
| Optional mock device plugin | Installs a mock device plugin for scheduling validation without a GPU. It is disabled by default; mock devices do not perform real GPU computation. | HAMi #1534 |
| Optional DRA installation path | The main Chart includes the hami-dra subchart 0.0.1, which can switch deployment to the Kubernetes Dynamic Resource Allocation path. It is disabled by default. See below for the behavior when switching paths. | HAMi #1542 |
Chart changes
The first main Chart is named hami; the default application image is ghcr.io/dynamia-ai/hami:v2.7.1-rc0. This tag is used for the scheduler extender, NVIDIA device plugin, and vGPU monitor. The bundled kube-scheduler, certificate helper, and optional subcharts use their own image settings.
| Setting | Default | Description |
|---|---|---|
schedulerName | hami-scheduler | Scheduler name used by workloads |
scheduler.replicas | 1 | Deploy one scheduler Pod by default |
devicePlugin.nvidiaNodeSelector | gpu: "on" | Deploy the NVIDIA device plugin to nodes with this label |
devicePlugin.deviceSplitCount | 10 | Default number of device-sharing slices |
devicePlugin.deviceMemoryScaling | 1 | Do not scale up the default allocatable memory capacity |
devicePlugin.deviceListStrategy | envvar | Pass the device list through environment variables by default |
mockDevicePlugin.enabled | false | When enabled, use mock plugin image tag 0.1.0 by default |
dra.enabled | false | When enabled, use the DRA subchart instead of the regular scheduler and NVIDIA device plugin from the main Chart |
The DRA path must meet the requirements of the companion driver and Kubernetes DRA. Enterprise extension notes for the regular device-plugin path do not automatically apply to it.
Upgrade notes
Install the first release or migrate from open source
Use the delivered hami-2.7.1-rc0.tgz, the companion Enterprise runtime, and license materials. If you use a custom image registry or explicit image tags, check the versions of the scheduler extender, device plugin, and monitor together; do not replace only one component.
When migrating from an open-source deployment, check node selection, scheduler integration, and device resource requests against this release's configuration. If old values use devicePlugin.nvidianodeSelector, change the key to devicePlugin.nvidiaNodeSelector. The default selector is gpu: "on"; omitting a custom selector may deploy the plugin to the wrong set of nodes.
The old Iluvatar settings iluvatarResourceName, iluvatarResourceMem, and iluvatarResourceCore are no longer used by the Chart, and old values are not migrated automatically. The former default resources iluvatar.ai/vgpu, iluvatar.ai/vcuda-memory, and iluvatar.ai/vcuda-core are replaced with model-specific resource names. Set devices.iluvatar.enabled: true, update Pod device-count, memory, and compute requests for the actual model, and deploy the companion GPU Manager with the same resource names. Replacing only the Chart while keeping old Pod resource names does not complete the migration.
devicePlugin.nodeConfiguration.config accepts JSON text. Set name to the actual node name; the default example your-node-name does not automatically match a node in your cluster. When devicePlugin.nodeConfiguration.externalConfigName is set, the Chart uses the specified ConfigMap and does not create this node configuration.
Check the Downward API and startup hook
The default device configuration sets both nvidia.useDownward: true and nvidia.postNvidiaSmi: true. With the Downward API enabled, the latter adds a postStart hook that runs nvidia-smi in containers requesting NVIDIA devices. The application image must provide this command. An existing postStart hook is replaced, not merged automatically.
To disable the startup hook, keep nvidia.useDownward: true in the device configuration and set nvidia.postNvidiaSmi to false. These two fields are not top-level Helm parameters in this release. If you use a custom files/device-config.yaml, it replaces the entire default device configuration; preserve the other device fields your deployment uses rather than supplying only these two. Update the scheduler and device plugin with the modified configuration, then recreate workload Pods whose injection behavior must change.
The first release handles NVIDIA allocation annotations using the regular-container list and does not yet include the later protections for empty allocation lists and out-of-range container indexes. Combinations involving non-GPU containers, init containers, or native sidecars should not be considered fully supported on this basis.
Check quotas and monitoring integrations
The first release does not include the NVIDIA multi-container quota-aggregation fix. Multiple GPU containers in one Pod may each pass their individual checks even when their combined request exceeds the namespace's remaining quota. Deployments that need this fix can refer to Chart 2.8.0-rc0. The quota boundary for runtime memory adjustment is described above.
Memory-detail metrics come from the vGPU monitor; vGPULicenseRemains comes from the scheduler. Configure collection for both components and use current-value semantics in dashboards and alerts. This release still retains older metrics such as HostGPUMemoryUsage and HostCoreUtilization; do not remove their collection rules based on a removal list from a later release.
See the Chart and appVersion mapping or the software center.
Values reference
The following parameter descriptions are taken from the README bundled with the Chart 2.7.1-rc0 release package and are preserved as published. Use this version's values.yaml as the source of default configuration.
Expand values table
Global Configuration
| Parameter | Description | Default Value |
|---|---|---|
global.imageRegistry | Global Docker image registry | "" |
global.imagePullSecrets | Global Docker image pull secrets | [] |
global.imageTag | Image tag | "v2.7.1" |
global.gpuHookPath | GPU Hook path | /usr/local |
global.labels | Global labels | {} |
global.annotations | Global annotations | {} |
global.managedNodeSelectorEnable | Whether to enable managed node selector | false |
global.managedNodeSelector.usage | Managed node selector usage | "gpu" |
nameOverride | Name override | "" |
fullnameOverride | Full name override | "" |
namespaceOverride | Namespace override | "" |
Resource Name Configuration
NVIDIA GPU Resources
| Parameter | Description | Default Value |
|---|---|---|
resourceName | GPU resource name | "nvidia.com/gpu" |
resourceMem | GPU memory resource name | "nvidia.com/gpumem" |
resourceMemPercentage | GPU memory percentage resource name | "nvidia.com/gpumem-percentage" |
resourceCores | GPU core resource name | "nvidia.com/gpucores" |
resourcePriority | GPU priority resource name | "nvidia.com/priority" |
Cambricon MLU Resources
| Parameter | Description | Default Value |
|---|---|---|
mluResourceName | MLU resource name | "cambricon.com/vmlu" |
mluResourceMem | MLU memory resource name | "cambricon.com/mlu.smlu.vmemory" |
mluResourceCores | MLU core resource name | "cambricon.com/mlu.smlu.vcore" |
Hygon DCU Resources
| Parameter | Description | Default Value |
|---|---|---|
dcuResourceName | DCU resource name | "hygon.com/dcunum" |
dcuResourceMem | DCU memory resource name | "hygon.com/dcumem" |
dcuResourceCores | DCU core resource name | "hygon.com/dcucores" |
Metax GPU Resources
| Parameter | Description | Default Value |
|---|---|---|
metaxResourceName | GPU resource name | "metax-tech.com/sgpu" |
metaxResourceCore | GPU core resource name | "metax-tech.com/vcore" |
metaxResourceMem | GPU memory resource name | "metax-tech.com/vmemory" |
metaxsGPUTopologyAware | GPU topology awareness | "false" |
Enflame GCU Resources
| Parameter | Description | Default Value |
|---|---|---|
enflameResourceNameVGCU | vGCU resource name | "enflame.com/vgcu" |
enflameResourceNameVGCUPercentage | vGCU percentage resource name | "enflame.com/vgcu-percentage" |
Kunlunxin XPU Resources
| Parameter | Description | Default Value |
|---|---|---|
kunlunResourceName | XPU resource name | "kunlunxin.com/xpu" |
Scheduler Configuration
| Parameter | Description | Default Value |
|---|---|---|
schedulerName | Scheduler name | "hami-scheduler" |
scheduler.nodeName | Define node name, scheduler will schedule to this node | "" |
scheduler.overwriteEnv | Whether to overwrite environment variables | "false" |
scheduler.defaultSchedulerPolicy.nodeSchedulerPolicy | Node scheduler policy | binpack |
scheduler.defaultSchedulerPolicy.gpuSchedulerPolicy | GPU scheduler policy | spread |
scheduler.metricsBindAddress | Metrics bind address | ":9395" |
scheduler.forceOverwriteDefaultScheduler | Whether to force overwrite default scheduler | true |
scheduler.livenessProbe | Whether to enable liveness probe | false |
scheduler.leaderElect | Whether to enable leader election | true |
scheduler.replicas | Number of replicas | 1 |
Kube Scheduler Configuration
| Parameter | Description | Default Value |
|---|---|---|
scheduler.kubeScheduler.enabled | Whether to run kube-scheduler container in scheduler pod | true |
scheduler.kubeScheduler.image.registry | Kube scheduler image registry | "registry.cn-hangzhou.aliyuncs.com" |
scheduler.kubeScheduler.image.repository | Kube scheduler image repository | "google_containers/kube-scheduler" |
scheduler.kubeScheduler.image.tag | Kube scheduler image tag | "" |
scheduler.kubeScheduler.image.pullPolicy | Kube scheduler image pull policy | IfNotPresent |
scheduler.kubeScheduler.image.pullSecrets | Kube scheduler image pull secrets | [] |
scheduler.kubeScheduler.extraNewArgs | Extra new arguments | ["--config=/config/config.yaml", "-v=4"] |
scheduler.kubeScheduler.extraArgs | Extra arguments | ["--policy-config-file=/config/config.json", "-v=4"] |
Extender Configuration
| Parameter | Description | Default Value |
|---|---|---|
scheduler.extender.image.registry | Scheduler extender image registry | "docker.io" |
scheduler.extender.image.repository | Scheduler extender image repository | "projecthami/hami" |
scheduler.extender.image.tag | Scheduler extender image tag | "" |
scheduler.extender.image.pullPolicy | Scheduler extender image pull policy | IfNotPresent |
scheduler.extender.image.pullSecrets | Scheduler extender image pull secrets | [] |
scheduler.extender.extraArgs | Scheduler extender extra arguments | ["--debug", "-v=4"] |
Admission Webhook Configuration
| Parameter | Description | Default Value |
|---|---|---|
scheduler.admissionWebhook.enabled | Whether to enable admission webhook | true |
scheduler.admissionWebhook.customURL.enabled | Whether to enable custom URL | false |
scheduler.admissionWebhook.customURL.host | Custom URL host | 127.0.0.1 |
scheduler.admissionWebhook.customURL.port | Custom URL port | 31998 |
scheduler.admissionWebhook.customURL.path | Custom URL path | /webhook |
scheduler.admissionWebhook.reinvocationPolicy | Reinvocation policy | Never |
scheduler.admissionWebhook.failurePolicy | Failure policy | Ignore |
TLS Certificate Configuration
| Parameter | Description | Default Value |
|---|---|---|
scheduler.certManager.enabled | Whether to use cert-manager to generate self-signed certificates | false |
scheduler.patch.enabled | Whether to use kube-webhook-certgen to generate self-signed certificates | true |
scheduler.patch.image.registry | Certgen image registry | "docker.io" |
scheduler.patch.image.repository | Certgen image repository | "jettech/kube-webhook-certgen" |
scheduler.patch.image.tag | Certgen image tag | "v1.5.2" |
scheduler.patch.image.pullPolicy | Certgen image pull policy | IfNotPresent |
scheduler.patch.imageNew.registry | New certgen image registry | "docker.io" |
scheduler.patch.imageNew.repository | New certgen image repository | "liangjw/kube-webhook-certgen" |
scheduler.patch.imageNew.tag | New certgen image tag | "v1.1.1" |
Scheduler Service Configuration
| Parameter | Description | Default Value |
|---|---|---|
scheduler.service.type | Service type | NodePort |
scheduler.service.httpPort | HTTP port | 443 |
scheduler.service.schedulerPort | Scheduler NodePort | 31998 |
scheduler.service.monitorPort | Monitor port | 31993 |
scheduler.service.monitorTargetPort | Monitor target port | 9395 |
Device Plugin Configuration
| Parameter | Description | Default Value |
|---|---|---|
devicePlugin.image.registry | Device plugin image registry | "docker.io" |
devicePlugin.image.repository | Device plugin image repository | "projecthami/hami" |
devicePlugin.image.tag | Device plugin image tag | "" |
devicePlugin.image.pullPolicy | Device plugin image pull policy | IfNotPresent |
devicePlugin.image.pullSecrets | Device plugin image pull secrets | [] |
Monitor Configuration
| Parameter | Description | Default Value |
|---|---|---|
devicePlugin.monitor.image.registry | Monitor image registry | "docker.io" |
devicePlugin.monitor.image.repository | Monitor image repository | "projecthami/hami" |
devicePlugin.monitor.image.tag | Monitor image tag | "" |
devicePlugin.monitor.image.pullPolicy | Monitor image pull policy | IfNotPresent |
devicePlugin.monitor.image.pullSecrets | Monitor image pull secrets | [] |
devicePlugin.monitor.ctrPath | Container path | /usr/local/vgpu/containers |
devicePlugin.monitor.extraArgs | Monitor extra arguments | ["-v=4"] |
devicePlugin.monitor.extraEnvs | Monitor extra environments | {} |
Device Plugin Other Configuration
| Parameter | Description | Default Value |
|---|---|---|
devicePlugin.deviceSplitCount | Integer type, default value: 10. Maximum number of tasks assigned to a single GPU device | 10 |
devicePlugin.deviceMemoryScaling | Device memory scaling ratio | 1 |
devicePlugin.deviceCoreScaling | Device core scaling ratio | 1 |
devicePlugin.runtimeClassName | Runtime class name | "" |
devicePlugin.createRuntimeClass | Whether to create runtime class | false |
devicePlugin.migStrategy | String type, "none" means ignore MIG functionality, "mixed" means allocate MIG devices through independent resources | "none" |
devicePlugin.disablecorelimit | String type, "true" means disable core limit, "false" means enable core limit | "false" |
devicePlugin.passDeviceSpecsEnabled | Whether to enable passing device specs | false |
devicePlugin.extraArgs | Device plugin extra arguments | ["-v=4"] |
devicePlugin.nodeConfiguration.config | Node configuration for device plugin by json | An example of default configuration. |
devicePlugin.nodeConfiguration.externalConfigName | Node configuration for device plugin by external congimap | "" |
devicePlugin.extraEnvs | Device plugin extra environments | {} |
Device Plugin Service Configuration
| Parameter | Description | Default Value |
|---|---|---|
devicePlugin.service.type | Service type | NodePort |
devicePlugin.service.httpPort | HTTP port | 31992 |
Device Plugin Deployment Configuration
| Parameter | Description | Default Value |
|---|---|---|
devicePlugin.pluginPath | Plugin path | /var/lib/kubelet/device-plugins |
devicePlugin.libPath | Library path | /usr/local/vgpu |
devicePlugin.nvidiaNodeSelector | NVIDIA node selector | {"gpu": "on"} |
devicePlugin.updateStrategy.type | Update strategy type | RollingUpdate |
devicePlugin.updateStrategy.rollingUpdate.maxUnavailable | Maximum unavailable count | 1 |
Device Configuration
AWS Neuron
| Parameter | Description | Default Value |
|---|---|---|
devices.awsneuron.customresources | Custom resources | ["aws.amazon.com/neuron", "aws.amazon.com/neuroncore"] |
Kunlunxin
| Parameter | Description | Default Value |
|---|---|---|
devices.kunlun.enabled | Whether to enable | true |
devices.kunlun.customresources | Custom resources | ["kunlunxin.com/xpu"] |
Mthreads
| Parameter | Description | Default Value |
|---|---|---|
devices.mthreads.enabled | Whether to enable | true |
devices.mthreads.customresources | Custom resources | ["mthreads.com/vgpu"] |
NVIDIA
| Parameter | Description | Default Value |
|---|---|---|
devices.nvidia.gpuCorePolicy | GPU core policy | default |
devices.nvidia.libCudaLogLevel | CUDA library log level | 1 |
Huawei Ascend
| Parameter | Description | Default Value |
|---|---|---|
devices.ascend.enabled | Whether to enable | false |
devices.ascend.image | Image | "" |
devices.ascend.imagePullPolicy | Image pull policy | IfNotPresent |
devices.ascend.extraArgs | Extra arguments | [] |
devices.ascend.nodeSelector | Node selector | {"ascend": "on"} |
devices.ascend.tolerations | Tolerations | [] |
devices.ascend.customresources | Custom resources | ["huawei.com/Ascend910A", "huawei.com/Ascend910A-memory", ...] |
Iluvatar
| Parameter | Description | Default Value |
|---|---|---|
devices.iluvatar.enabled | Whether to enable | false |
devices.iluvatar.customresources | Custom resources | ["iluvatar.ai/BI-V150-vgpu", "iluvatar.ai/BI-V150.vMem","iluvatar.ai/BI-V150.vCore", ...] |