Skip to main content

HAMi Enterprise Chart 2.8.0-rc1

Chart versionDeclared appVersionRelease date
2.8.0-rc12.8.0-rc12026-03-16

This is a patch release. Changes below are compared with HAMi Enterprise v2.8.0-rc0.

Software changes​

  • Alibaba PPU scheduling integration: The scheduler now recognizes PPU node registration information and the alibaba.com/ppu, alibaba.com/ppu-memory, and alibaba.com/gpu-core resources. This Chart does not install a PPU device plugin. Before using this capability, the companion plugin must register the devices and resources on the nodes.
  • Memory registration for NVIDIA unified-memory GPUs: When NVML cannot query device memory, the device plugin can register the device using a preconfigured memory capacity. If no positive value is set, such devices are skipped. Devices whose memory NVML can query continue to use the queried value.
  • Ascend 910C module allocation: When Ascend is enabled and the device configuration contains commonWord: Ascend910C, a request for one NPU by a single container is changed to two to allocate a complete module; other odd-numbered requests are rejected. The default device configuration in the released Chart does not include a 910C entry.
  • Pod admission with an explicitly named scheduler: Fixes an issue where the admission webhook could skip device handling when spec.schedulerName was already set to the HAMi scheduler name. The name is configured by the Chart's schedulerName setting.
  • NVIDIA node status and workload startup: The scheduler clears node device state when the number of allocatable GPUs reaches zero and updates device information when registration annotations change. If NVIDIA configuration enables postNvidiaSmi and a workload already has an Exec-type postStart command, nvidia-smi runs only after the original command succeeds. The container image must provide sh for the command chaining.
  • NVIDIA namespace quotas: During Pod admission, the webhook totals single-GPU NVIDIA memory and compute requests declared by the containers and checks the applicable ResourceQuota configured for the namespace. It returns exceeding resource quota when the quota is exceeded. This admission check does not cover requests for multiple GPUs declared by a container.
  • Monitoring metric updates: The container memory-detail metrics hami_container_device_memory_context_bytes, hami_container_device_memory_module_bytes, and hami_container_device_memory_data_bytes are renamed to vGPU_device_memory_context_size_bytes, vGPU_device_memory_module_size_bytes, and vGPU_device_memory_buffer_size_bytes, respectively. The old hami_container_device_memory_offset_bytes metric has no same-name replacement. The scheduler metrics GPUDeviceMemoryLimit, GPUDeviceCoreLimit, GPUDeviceMemoryAllocated, GPUDeviceCoreAllocated, and vGPULicenseRemains gain a devicetype label. GPUDeviceMemoryAllocated also gains a devicecores label, as does nodeGPUOverview.

Chart changes​

  • Adds devicePlugin.preConfiguredDeviceMemory, measured in MB and defaulting to 0. It is written to the scheduler's device configuration. For a specific node, devicePlugin.nodeConfiguration.config can set preconfigureddevicememory to override the device-plugin configuration. A positive node-level value takes precedence over the general setting.
  • Adds devices.alibaba.enabled (default true) and devices.alibaba.ppuCorePolicy (default default) for PPU resource mapping and compute policy. The supported policies are default, force, and disable.
  • Adds prometheus.enabled, defaulting to false. When set to true and the cluster already provides the monitoring.coreos.com/v1/ServiceMonitor resource type, the Chart creates ServiceMonitors for the scheduler and NVIDIA device plugin. The Chart does not install that resource type.
  • The main Chart's version and appVersion are both 2.8.0-rc1; the default image tag changes to v2.8.0-rc1.

Upgrade notes​

On NVIDIA nodes where NVML cannot query memory, set devicePlugin.preConfiguredDeviceMemory to the actual capacity in MB. To configure different capacities by node, set a positive preconfigureddevicememory for each node in devicePlugin.nodeConfiguration.config; other NVIDIA nodes can retain the default.

For Ascend 910C, in addition to setting devices.ascend.enabled: true, provide a custom Chart package containing files/device-config.yaml with commonWord: Ascend910C and the corresponding resource names, then add those resource names to devices.ascend.customresources. The custom file replaces the entire device configuration generated by the Chart, so preserve existing device entries as needed. The default Chart does not include 910C configuration. For existing 910C workloads, check odd-numbered NPU requests and capacity expectations after single-NPU requests are changed to two NPUs.

For NVIDIA workloads with a custom Exec-type postStart command, confirm that the image provides sh and that the original command and its arguments behave as expected when run by a shell. nvidia-smi will not run if the original command fails.

Deployments using Prometheus Operator can enable prometheus.enabled as needed. Before upgrading, check dashboards, alerts, and query rules that depend on the old metric names or exact label matches, and migrate them to the new metrics and labels. Rules that depend on the memory-offset metric need a separate adjustment.

Obtain the package through the agreed delivery channel. See the Chart and appVersion mapping or the software center.

Values reference​

The following parameter descriptions are taken from the README bundled with the Chart 2.8.0-rc1 release package and are preserved as published. Use this version's values.yaml as the source of default configuration.

Expand values table

Global Configuration

ParameterDescriptionDefault Value
global.imageRegistryGlobal Docker image registry""
global.imagePullSecretsGlobal Docker image pull secrets[]
global.imageTagImage tag"v2.8.0"
global.gpuHookPathGPU Hook path/usr/local
global.labelsGlobal labels{}
global.annotationsGlobal annotations{}
global.managedNodeSelectorEnableWhether to enable managed node selectorfalse
global.managedNodeSelector.usageManaged node selector usage"gpu"
nameOverrideName override""
fullnameOverrideFull name override""
namespaceOverrideNamespace override""

Resource Name Configuration

NVIDIA GPU Resources

ParameterDescriptionDefault Value
resourceNameGPU resource name"nvidia.com/gpu"
resourceMemGPU memory resource name"nvidia.com/gpumem"
resourceMemPercentageGPU memory percentage resource name"nvidia.com/gpumem-percentage"
resourceCoresGPU core resource name"nvidia.com/gpucores"
resourcePriorityGPU priority resource name"nvidia.com/priority"

Cambricon MLU Resources

ParameterDescriptionDefault Value
mluResourceNameMLU resource name"cambricon.com/vmlu"
mluResourceMemMLU memory resource name"cambricon.com/mlu.smlu.vmemory"
mluResourceCoresMLU core resource name"cambricon.com/mlu.smlu.vcore"

Hygon DCU Resources

ParameterDescriptionDefault Value
dcuResourceNameDCU resource name"hygon.com/dcunum"
dcuResourceMemDCU memory resource name"hygon.com/dcumem"
dcuResourceCoresDCU core resource name"hygon.com/dcucores"

Metax GPU Resources

ParameterDescriptionDefault Value
metaxResourceNameGPU resource name"metax-tech.com/sgpu"
metaxResourceCoreGPU core resource name"metax-tech.com/vcore"
metaxResourceMemGPU memory resource name"metax-tech.com/vmemory"
metaxsGPUTopologyAwareGPU topology awareness"false"

Enflame GCU Resources

ParameterDescriptionDefault Value
enflameResourceNameVGCUvGCU resource name"enflame.com/vgcu"
enflameResourceNameVGCUPercentagevGCU percentage resource name"enflame.com/vgcu-percentage"

Kunlunxin XPU Resources

ParameterDescriptionDefault Value
kunlunResourceNameXPU resource name"kunlunxin.com/xpu"

Scheduler Configuration

ParameterDescriptionDefault Value
schedulerNameScheduler name"hami-scheduler"
scheduler.nodeNameDefine node name, scheduler will schedule to this node""
scheduler.overwriteEnvWhether to overwrite environment variables"false"
scheduler.defaultSchedulerPolicy.nodeSchedulerPolicyNode scheduler policybinpack
scheduler.defaultSchedulerPolicy.gpuSchedulerPolicyGPU scheduler policyspread
scheduler.metricsBindAddressMetrics bind address":9395"
scheduler.forceOverwriteDefaultSchedulerWhether to force overwrite default schedulertrue
scheduler.livenessProbeWhether to enable liveness probefalse
scheduler.leaderElectWhether to enable leader electiontrue
scheduler.replicasNumber of replicas1

Kube Scheduler Configuration

ParameterDescriptionDefault Value
scheduler.kubeScheduler.enabledWhether to run kube-scheduler container in scheduler podtrue
scheduler.kubeScheduler.image.registryKube scheduler image registry"registry.cn-hangzhou.aliyuncs.com"
scheduler.kubeScheduler.image.repositoryKube scheduler image repository"google_containers/kube-scheduler"
scheduler.kubeScheduler.image.tagKube scheduler image tag""
scheduler.kubeScheduler.image.pullPolicyKube scheduler image pull policyIfNotPresent
scheduler.kubeScheduler.image.pullSecretsKube scheduler image pull secrets[]
scheduler.kubeScheduler.extraNewArgsExtra new arguments["--config=/config/config.yaml", "-v=4"]
scheduler.kubeScheduler.extraArgsExtra arguments["--policy-config-file=/config/config.json", "-v=4"]

Extender Configuration

ParameterDescriptionDefault Value
scheduler.extender.image.registryScheduler extender image registry"docker.io"
scheduler.extender.image.repositoryScheduler extender image repository"projecthami/hami"
scheduler.extender.image.tagScheduler extender image tag""
scheduler.extender.image.pullPolicyScheduler extender image pull policyIfNotPresent
scheduler.extender.image.pullSecretsScheduler extender image pull secrets[]
scheduler.extender.extraArgsScheduler extender extra arguments["--debug", "-v=4"]

Admission Webhook Configuration

ParameterDescriptionDefault Value
scheduler.admissionWebhook.enabledWhether to enable admission webhooktrue
scheduler.admissionWebhook.customURL.enabledWhether to enable custom URLfalse
scheduler.admissionWebhook.customURL.hostCustom URL host127.0.0.1
scheduler.admissionWebhook.customURL.portCustom URL port31998
scheduler.admissionWebhook.customURL.pathCustom URL path/webhook
scheduler.admissionWebhook.reinvocationPolicyReinvocation policyNever
scheduler.admissionWebhook.failurePolicyFailure policyIgnore

TLS Certificate Configuration

ParameterDescriptionDefault Value
scheduler.certManager.enabledWhether to use cert-manager to generate self-signed certificatesfalse
scheduler.patch.enabledWhether to use kube-webhook-certgen to generate self-signed certificatestrue
scheduler.patch.image.registryCertgen image registry"docker.io"
scheduler.patch.image.repositoryCertgen image repository"jettech/kube-webhook-certgen"
scheduler.patch.image.tagCertgen image tag"v1.5.2"
scheduler.patch.image.pullPolicyCertgen image pull policyIfNotPresent
scheduler.patch.imageNew.registryNew certgen image registry"docker.io"
scheduler.patch.imageNew.repositoryNew certgen image repository"liangjw/kube-webhook-certgen"
scheduler.patch.imageNew.tagNew certgen image tag"v1.1.1"

Scheduler Service Configuration

ParameterDescriptionDefault Value
scheduler.service.typeService typeNodePort
scheduler.service.httpPortHTTP port443
scheduler.service.schedulerPortScheduler NodePort31998
scheduler.service.monitorPortMonitor port31993
scheduler.service.monitorTargetPortMonitor target port9395

Device Plugin Configuration

ParameterDescriptionDefault Value
devicePlugin.image.registryDevice plugin image registry"docker.io"
devicePlugin.image.repositoryDevice plugin image repository"projecthami/hami"
devicePlugin.image.tagDevice plugin image tag""
devicePlugin.image.pullPolicyDevice plugin image pull policyIfNotPresent
devicePlugin.image.pullSecretsDevice plugin image pull secrets[]

Monitor Configuration

ParameterDescriptionDefault Value
devicePlugin.monitor.image.registryMonitor image registry"docker.io"
devicePlugin.monitor.image.repositoryMonitor image repository"projecthami/hami"
devicePlugin.monitor.image.tagMonitor image tag""
devicePlugin.monitor.image.pullPolicyMonitor image pull policyIfNotPresent
devicePlugin.monitor.image.pullSecretsMonitor image pull secrets[]
devicePlugin.monitor.ctrPathContainer path/usr/local/vgpu/containers
devicePlugin.monitor.extraArgsMonitor extra arguments["-v=4"]
devicePlugin.monitor.extraEnvsMonitor extra environments{}

Device Plugin Other Configuration

ParameterDescriptionDefault Value
devicePlugin.deviceSplitCountInteger type, default value: 10. Maximum number of tasks assigned to a single GPU device10
devicePlugin.deviceMemoryScalingDevice memory scaling ratio1
devicePlugin.deviceCoreScalingDevice core scaling ratio1
devicePlugin.runtimeClassNameRuntime class name""
devicePlugin.createRuntimeClassWhether to create runtime classfalse
devicePlugin.migStrategyString type, "none" means ignore MIG functionality, "mixed" means allocate MIG devices through independent resources"none"
devicePlugin.disablecorelimitString type, "true" means disable core limit, "false" means enable core limit"false"
devicePlugin.passDeviceSpecsEnabledWhether to enable passing device specsfalse
devicePlugin.extraArgsDevice plugin extra arguments["-v=4"]
devicePlugin.nodeConfiguration.configNode configuration for device plugin by jsonAn example of default configuration.
devicePlugin.nodeConfiguration.externalConfigNameNode configuration for device plugin by external congimap""
devicePlugin.extraEnvsDevice plugin extra environments{}

Device Plugin Service Configuration

ParameterDescriptionDefault Value
devicePlugin.service.typeService typeNodePort
devicePlugin.service.httpPortHTTP port31992

Device Plugin Deployment Configuration

ParameterDescriptionDefault Value
devicePlugin.pluginPathPlugin path/var/lib/kubelet/device-plugins
devicePlugin.libPathLibrary path/usr/local/vgpu
devicePlugin.nvidiaNodeSelectorNVIDIA node selector{"gpu": "on"}
devicePlugin.updateStrategy.typeUpdate strategy typeRollingUpdate
devicePlugin.updateStrategy.rollingUpdate.maxUnavailableMaximum unavailable count1

Device Configuration

AWS Neuron

ParameterDescriptionDefault Value
devices.awsneuron.customresourcesCustom resources["aws.amazon.com/neuron", "aws.amazon.com/neuroncore"]

Kunlunxin

ParameterDescriptionDefault Value
devices.kunlun.enabledWhether to enabletrue
devices.kunlun.customresourcesCustom resources["kunlunxin.com/xpu"]

Mthreads

ParameterDescriptionDefault Value
devices.mthreads.enabledWhether to enabletrue
devices.mthreads.customresourcesCustom resources["mthreads.com/vgpu"]

NVIDIA

ParameterDescriptionDefault Value
devices.nvidia.gpuCorePolicyGPU core policydefault
devices.nvidia.libCudaLogLevelCUDA library log level1

Huawei Ascend

ParameterDescriptionDefault Value
devices.ascend.enabledWhether to enablefalse
devices.ascend.imageImage""
devices.ascend.imagePullPolicyImage pull policyIfNotPresent
devices.ascend.extraArgsExtra arguments[]
devices.ascend.nodeSelectorNode selector{"ascend": "on"}
devices.ascend.tolerationsTolerations[]
devices.ascend.customresourcesCustom resources["huawei.com/Ascend910A", "huawei.com/Ascend910A-memory", ...]

Iluvatar

ParameterDescriptionDefault Value
devices.iluvatar.enabledWhether to enablefalse
devices.iluvatar.customresourcesCustom resources["iluvatar.ai/BI-V150-vgpu", "iluvatar.ai/BI-V150.vMem","iluvatar.ai/BI-V150.vCore", ...]