跳到主要内容

HAMi Enterprise 2.9.0-rc1

Chart 版本Chart 声明的 appVersion发布日期
2.9.0-rc12.9.0-rc12026 年 5 月 27 日

本次为 patch 版本,以下变化以 HAMi Enterprise v2.9.0-rc0 为比较基线。

软件变更​

修复包含初始化容器的 Pod 中,NVIDIA 设备标注与普通容器的对应关系。 在通过 Downward API 传递设备分配信息时,写入普通容器标注的位置会扣除初始化容器数量,避免将初始化容器的位置误用于普通容器。

此修复适用于设备配置 nvidia.useDownward: true 的部署,本版 Chart 生成的设备配置默认开启该项。本次仍跳过初始化容器自身的 GPU 设备标注,不能据此视为初始化容器的 GPU 使用流程已完整支持。

Chart 变更​

  • 调度器的 ClusterRole 增加对 resource.k8s.io API 组下 deviceclasses、resourceclaims、resourceclaimtemplates、resourceslices 的 get、list、watch 权限,解决 kube-scheduler 监听这些资源时的权限不足问题。这是调度器权限补齐,不会开启默认关闭的 dra.enabled。
  • global.imageTag 默认更新为 v2.9.0-rc1。调度扩展器、NVIDIA 设备插件和监控容器的默认镜像仓库仍为 ghcr.io/dynamia-ai/hami。
  • Chart 名称仍为 hami,安装包为 hami-2.9.0-rc1.tgz;默认 schedulerName 仍为 hami-scheduler。

升级说明​

使用自主管理的调度器 RBAC 时,需要同步补齐上述四类资源的只读权限。由 Chart 管理 RBAC 的部署随升级更新。

对于包含初始化容器的 NVIDIA 工作负载,在升级后新建的 Pod 上检查普通 GPU 容器的设备分配标注及实际可见设备。已有运行中 Pod 的设备分配不会因调度器升级而自动重新执行。

安装包通过约定的交付渠道获取。查看Chart 与 appVersion 对应关系或软件中心。

Values 说明​

以下参数说明取自 Chart 2.9.0-rc1 发布包随附的 README,保留原文。默认配置以该版本的 values.yaml 为准。

展开参数表

Global Configuration

ParameterDescriptionDefault Value
global.imageRegistryGlobal Docker image registry""
global.imagePullSecretsGlobal Docker image pull secrets[]
global.imageTagImage tag"v2.8.0"
global.gpuHookPathGPU Hook path/usr/local
global.labelsGlobal labels{}
global.annotationsGlobal annotations{}
global.managedNodeSelectorEnableWhether to enable managed node selectorfalse
global.managedNodeSelector.usageManaged node selector usage"gpu"
nameOverrideName override""
fullnameOverrideFull name override""
namespaceOverrideNamespace override""

Resource Name Configuration

NVIDIA GPU Resources

ParameterDescriptionDefault Value
resourceNameGPU resource name"nvidia.com/gpu"
resourceMemGPU memory resource name"nvidia.com/gpumem"
resourceMemPercentageGPU memory percentage resource name"nvidia.com/gpumem-percentage"
resourceCoresGPU core resource name"nvidia.com/gpucores"
resourcePriorityGPU priority resource name"nvidia.com/priority"

Cambricon MLU Resources

ParameterDescriptionDefault Value
mluResourceNameMLU resource name"cambricon.com/vmlu"
mluResourceMemMLU memory resource name"cambricon.com/mlu.smlu.vmemory"
mluResourceCoresMLU core resource name"cambricon.com/mlu.smlu.vcore"

Hygon DCU Resources

ParameterDescriptionDefault Value
dcuResourceNameDCU resource name"hygon.com/dcunum"
dcuResourceMemDCU memory resource name"hygon.com/dcumem"
dcuResourceCoresDCU core resource name"hygon.com/dcucores"

Metax GPU Resources

ParameterDescriptionDefault Value
metaxResourceNameGPU resource name"metax-tech.com/sgpu"
metaxResourceCoreGPU core resource name"metax-tech.com/vcore"
metaxResourceMemGPU memory resource name"metax-tech.com/vmemory"
metaxsGPUTopologyAwareGPU topology awareness"false"

Enflame GCU Resources

ParameterDescriptionDefault Value
enflameResourceNameVGCUvGCU resource name"enflame.com/vgcu"
enflameResourceNameVGCUPercentagevGCU percentage resource name"enflame.com/vgcu-percentage"

Kunlunxin XPU Resources

ParameterDescriptionDefault Value
kunlunResourceNameXPU resource name"kunlunxin.com/xpu"

Scheduler Configuration

ParameterDescriptionDefault Value
schedulerNameScheduler name"hami-scheduler"
scheduler.nodeNameDefine node name, scheduler will schedule to this node""
scheduler.overwriteEnvWhether to overwrite environment variables"false"
scheduler.defaultSchedulerPolicy.nodeSchedulerPolicyNode scheduler policybinpack
scheduler.defaultSchedulerPolicy.gpuSchedulerPolicyGPU scheduler policyspread
scheduler.metricsBindAddressMetrics bind address":9395"
scheduler.forceOverwriteDefaultSchedulerWhether to force overwrite default schedulertrue
scheduler.livenessProbeWhether to enable liveness probefalse
scheduler.leaderElectWhether to enable leader electiontrue
scheduler.replicasNumber of replicas1

Kube Scheduler Configuration

ParameterDescriptionDefault Value
scheduler.kubeScheduler.enabledWhether to run kube-scheduler container in scheduler podtrue
scheduler.kubeScheduler.image.registryKube scheduler image registry"registry.cn-hangzhou.aliyuncs.com"
scheduler.kubeScheduler.image.repositoryKube scheduler image repository"google_containers/kube-scheduler"
scheduler.kubeScheduler.image.tagKube scheduler image tag""
scheduler.kubeScheduler.image.pullPolicyKube scheduler image pull policyIfNotPresent
scheduler.kubeScheduler.image.pullSecretsKube scheduler image pull secrets[]
scheduler.kubeScheduler.extraNewArgsExtra new arguments["--config=/config/config.yaml", "-v=4"]
scheduler.kubeScheduler.extraArgsExtra arguments["--policy-config-file=/config/config.json", "-v=4"]

Extender Configuration

ParameterDescriptionDefault Value
scheduler.extender.image.registryScheduler extender image registry"docker.io"
scheduler.extender.image.repositoryScheduler extender image repository"projecthami/hami"
scheduler.extender.image.tagScheduler extender image tag""
scheduler.extender.image.pullPolicyScheduler extender image pull policyIfNotPresent
scheduler.extender.image.pullSecretsScheduler extender image pull secrets[]
scheduler.extender.extraArgsScheduler extender extra arguments["--debug", "-v=4"]

Admission Webhook Configuration

ParameterDescriptionDefault Value
scheduler.admissionWebhook.enabledWhether to enable admission webhooktrue
scheduler.admissionWebhook.customURL.enabledWhether to enable custom URLfalse
scheduler.admissionWebhook.customURL.hostCustom URL host127.0.0.1
scheduler.admissionWebhook.customURL.portCustom URL port31998
scheduler.admissionWebhook.customURL.pathCustom URL path/webhook
scheduler.admissionWebhook.reinvocationPolicyReinvocation policyNever
scheduler.admissionWebhook.failurePolicyFailure policyIgnore

TLS Certificate Configuration

ParameterDescriptionDefault Value
scheduler.certManager.enabledWhether to use cert-manager to generate self-signed certificatesfalse
scheduler.patch.enabledWhether to use kube-webhook-certgen to generate self-signed certificatestrue
scheduler.patch.image.registryCertgen image registry"docker.io"
scheduler.patch.image.repositoryCertgen image repository"jettech/kube-webhook-certgen"
scheduler.patch.image.tagCertgen image tag"v1.5.2"
scheduler.patch.image.pullPolicyCertgen image pull policyIfNotPresent
scheduler.patch.imageNew.registryNew certgen image registry"docker.io"
scheduler.patch.imageNew.repositoryNew certgen image repository"liangjw/kube-webhook-certgen"
scheduler.patch.imageNew.tagNew certgen image tag"v1.1.1"

Scheduler Service Configuration

ParameterDescriptionDefault Value
scheduler.service.typeService typeNodePort
scheduler.service.httpPortHTTP port443
scheduler.service.schedulerPortScheduler NodePort31998
scheduler.service.monitorPortMonitor port31993
scheduler.service.monitorTargetPortMonitor target port9395

Device Plugin Configuration

ParameterDescriptionDefault Value
devicePlugin.image.registryDevice plugin image registry"docker.io"
devicePlugin.image.repositoryDevice plugin image repository"projecthami/hami"
devicePlugin.image.tagDevice plugin image tag""
devicePlugin.image.pullPolicyDevice plugin image pull policyIfNotPresent
devicePlugin.image.pullSecretsDevice plugin image pull secrets[]

Monitor Configuration

ParameterDescriptionDefault Value
devicePlugin.monitor.image.registryMonitor image registry"docker.io"
devicePlugin.monitor.image.repositoryMonitor image repository"projecthami/hami"
devicePlugin.monitor.image.tagMonitor image tag""
devicePlugin.monitor.image.pullPolicyMonitor image pull policyIfNotPresent
devicePlugin.monitor.image.pullSecretsMonitor image pull secrets[]
devicePlugin.monitor.ctrPathContainer path/usr/local/vgpu/containers
devicePlugin.monitor.extraArgsMonitor extra arguments["-v=4"]
devicePlugin.monitor.extraEnvsMonitor extra environments{}

Device Plugin Other Configuration

ParameterDescriptionDefault Value
devicePlugin.deviceSplitCountInteger type, default value: 10. Maximum number of tasks assigned to a single GPU device10
devicePlugin.deviceMemoryScalingDevice memory scaling ratio1
devicePlugin.deviceCoreScalingDevice core scaling ratio1
devicePlugin.runtimeClassNameRuntime class name""
devicePlugin.createRuntimeClassWhether to create runtime classfalse
devicePlugin.migStrategyString type, "none" means ignore MIG functionality, "mixed" means allocate MIG devices through independent resources"none"
devicePlugin.disablecorelimitString type, "true" means disable core limit, "false" means enable core limit"false"
devicePlugin.passDeviceSpecsEnabledWhether to enable passing device specsfalse
devicePlugin.extraArgsDevice plugin extra arguments["-v=4"]
devicePlugin.nodeConfiguration.configNode configuration for device plugin by jsonAn example of default configuration.
devicePlugin.nodeConfiguration.externalConfigNameNode configuration for device plugin by external congimap""
devicePlugin.extraEnvsDevice plugin extra environments{}

Device Plugin Service Configuration

ParameterDescriptionDefault Value
devicePlugin.service.typeService typeNodePort
devicePlugin.service.httpPortHTTP port31992

Device Plugin Deployment Configuration

ParameterDescriptionDefault Value
devicePlugin.pluginPathPlugin path/var/lib/kubelet/device-plugins
devicePlugin.libPathLibrary path/usr/local/vgpu
devicePlugin.nvidiaNodeSelectorNVIDIA node selector{"gpu": "on"}
devicePlugin.updateStrategy.typeUpdate strategy typeRollingUpdate
devicePlugin.updateStrategy.rollingUpdate.maxUnavailableMaximum unavailable count1

Device Configuration

AWS Neuron

ParameterDescriptionDefault Value
devices.awsneuron.customresourcesCustom resources["aws.amazon.com/neuron", "aws.amazon.com/neuroncore"]

Kunlunxin

ParameterDescriptionDefault Value
devices.kunlun.enabledWhether to enabletrue
devices.kunlun.customresourcesCustom resources["kunlunxin.com/xpu"]

Mthreads

ParameterDescriptionDefault Value
devices.mthreads.enabledWhether to enabletrue
devices.mthreads.customresourcesCustom resources["mthreads.com/vgpu"]

NVIDIA

ParameterDescriptionDefault Value
devices.nvidia.gpuCorePolicyGPU core policydefault
devices.nvidia.libCudaLogLevelCUDA library log level1

Huawei Ascend

ParameterDescriptionDefault Value
devices.ascend.enabledWhether to enablefalse
devices.ascend.imageImage""
devices.ascend.imagePullPolicyImage pull policyIfNotPresent
devices.ascend.extraArgsExtra arguments[]
devices.ascend.nodeSelectorNode selector{"ascend": "on"}
devices.ascend.tolerationsTolerations[]
devices.ascend.customresourcesCustom resources["huawei.com/Ascend910A", "huawei.com/Ascend910A-memory", ...]

Iluvatar

ParameterDescriptionDefault Value
devices.iluvatar.enabledWhether to enablefalse
devices.iluvatar.customresourcesCustom resources["iluvatar.ai/BI-V150-vgpu", "iluvatar.ai/BI-V150.vMem","iluvatar.ai/BI-V150.vCore", ...]