Affinity Practices

Kubernetes provides several methods for allocating node usage, with the four most common ones being:

• Node Selector(nodeSelector)
• Node Affinity and Anti-affinity(nodeAffinity)
• Pod Affinity and Anti-affinity(podAffinity)
• Node Isolation/Restriction

1. Labels

When making node selections, Kubernetes' Label function is often used. Here we show the methods to label nodes and Pods separately.

1. Add the disktype=ssd label to the 10.10.10.10 node

# kubectl label nodes 10.10.10.10 disktype=ssd

2. Add the disktype=ssd label to Pod, which can also be used to add labels to objects such as Deployment and Service.

# kubectl label po unginx-7db67b8c69-zcxmm disktype=ssd

2. Node Selector (nodeSelector)

Node Selector is used when creating Pods (and controllers such as Deployment, StatefulSet, etc.), to assign Pods to the corresponding nodes.

The example yaml below creates a Pod object. In spec.nodeSelector, restrictions on the nodes where the container is deployed are added in the form of a Map. The nodeSelector will filter Nodes with disktype: ssd for Pod deployment.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    env: test
spec:
  containers:
  - name: nginx
    image: nginx
    imagePullPolicy: IfNotPresent
  nodeSelector:
    disktype: ssd

3. Node Affinity and Anti-affinity (nodeAffinity)

To use node affinity and anti-affinity features, add the affinity field under Deployment spec.template.spec. nodeAffinity is divided into hard matching and soft matching.

3.1 Hard Matching

In the following yaml example, requiredDuringSchedulingIgnoredDuringExecution can be interpreted as excluding nodes that do not have the specified Label. If the node does not contain ucloud=yes, the Pod will not be deployed to that node.

Under nodeSelectorTerms, there are matchExpressions (match expressions) and matchFields (match fields), choose either one. Here we use matchExpressions. In the expression below, Key and Values correspond, operator optional parameters are In, NotIn, Exists, DoesNotExist, Gt, Lt. You can also set NotIn, DoesNotExist for anti-Affinity settings. That is, If you wrote operator: NotIn, the Pod will be deployed to nodes without label ucloud=yes.

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    run: ucloud
  name: ucloud
spec:
  replicas: 3
  selector:
    matchLabels:
      run: ucloud
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: ucloud
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution: 
            nodeSelectorTerms: 
            - matchExpressions:
              - key: ucloud
                operator: In
                values:
                - "yes"
      containers:
      - image: nginx
        name: ucloud
        ports:
        - containerPort: 80
        resources: {}

3.2 Soft Matching

In contrast to requiredDuringSchedulingIgnoredDuringExecution, there is preferredDuringSchedulingIgnoredDuringExecution, also known as soft match. This parameter scores corresponding nodes, decreasing the selection probability of nodes without the Label. Here, the weight field ranges from 1-100. For each node that meets all scheduling requirements, the scheduler will compute the sum by iterating over the elements of this field, and adds the "weight" to the total MatchExpressions when the node matches the corresponding node. The scheduler then combines this score with the scores of the node's other priority functions. The node with the highest total score is the optimal choice.

preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
  preference:
    matchExpressions:
    - key: another-node-label-key
      operator: In
      values:
      - another-node-label-value

3.3 Explanation

1. If both nodeSelector and nodeAffinity are specified, the node must meet all conditions for the Pod to be scheduled to that node.

2. If multiple nodeSelectorTerms related to the nodeAffinity type are specified, if one of the nodeSelectorTerms can be satisfied, the Pod can be scheduled to that node.

3. If multiple matchExpressions associated with nodeSelectorTerms are specified, the container can only be scheduled to the node if the node fulfills all matchExpressions requirements.

4. If an existing container node's Label is deleted or changed, Kubernetes will not actively delete the container. Affinitive scheduling selection only works when scheduling Pods.