k8s优先级priority的使用

admin管理员组

文章数量:1623784

全栈工程师开发手册 （作者：栾鹏）
架构系列文章

直接使用kubectl explain pod.spec命令可以看到当前k8s支持的priority的属性
我这边的是1.15的k8s，可以看到的是

   priority	<integer>
     The priority value. Various system components use this field to find the
     priority of the pod. When Priority Admission Controller is enabled, it
     prevents users from setting this field. The admission controller populates
     this field from PriorityClassName. The higher the value, the higher the
     priority.

   priorityClassName	<string>
     If specified, indicates the pod's priority. "system-node-critical" and
     "system-cluster-critical" are two special keywords which indicate the
     highest priorities with the former being the highest priority. Any other
     name must be defined by creating a PriorityClass object with that name. If
     not specified, the pod priority will be default or zero if there is no
     default.

也就是说如果启动了Priority的抢占控制器，则就只能使用priorityClassName字段。

我们先来看看priorityClassName怎么用

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: test-priority
value: 100
globalDefault: false   # 整个系统只能有一个全局默认，如果没有全局默认则全局默认值为0
description: "This priority class should be used for airflow clickhouse build task pods only."

---
apiVersion: v1
kind: Pod
metadata:
  name: python
spec:
  priorityClassName: test-priority
  ...

这样就使用了预先定义的优先级。

如果我们想直接使用Priority直接定义数值，值越大优先级越高。则必须要关闭Priority抢占控制器，这个控制器是apiserver的启动参数。

在k8s官方文档中
https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#how-do-i-turn-off-an-admission-controller

说需要关闭Priority抢占控制器需要使用--disable-admission-plugins，如果只是在--enable-admission-plugins中去除是不行的，因为抢占控制器默认是开启的。
我这里是用rancher部署的apiserver，所以直接升级集群，修改api-server参数

    kube-api:
      always_pull_images: false
      extra_args:
        disable-admission-plugins: Priority
        enable-admission-plugins: >-
          NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction,TaintNodesByCondition,PersistentVolumeClaimResize
      pod_security_policy: false
      service_node_port_range: 30000-32767

升级后再来看api-server的启动参数

这样就可以在pod中使用Priority字段了。

我们测试一下，先为某个机器添加label：test=false

apiVersion: v1
kind: Pod
metadata:
  name: python-pod1
  labels:
    app: python
  namespace: cloudai-2
spec:
  # priorityClassName: clickhouse-build-priority
  priority: 11
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: test
            operator: In
            values:
            - "true"
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - python
        topologyKey: "kubernetes.io/hostname"
  imagePullSecrets:
    - name: hubsecret
  containers:
  - command: ['sleep','20000']
    name: python
    image: xxxxxxxxxxxx

---

apiVersion: v1
kind: Pod
metadata:
  name: python-pod2
  labels:
    app: python
  namespace: cloudai-2
spec:
  # priorityClassName: clickhouse-build-priority
  priority: 12
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: test
            operator: In
            values:
            - "true"
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - python
        topologyKey: "kubernetes.io/hostname"
  imagePullSecrets:
    - name: hubsecret
  containers:
  - command: ['sleep','20000']
    name: python
    image: xxxxxxxxxxxx

部署上两个pod都是未被成功调度，然后我们将机器的lable改成test=true，发现优先机高的一个pod先被调度。可以看到优先机生效。

本文标签： 优先级K8sPriority