Virtual Kubelet

Virtual Kubelet (opens in a new tab) is an important open-source project from the Kubernetes community. The Virtual Kubelet virtual node component enables seamless integration of SurferCloud's two major container products, UK8S and Cube. Users can utilize the VK component in the UK8S cluster to create Cube container instances. Each Cube instance is viewed as a Pod on a VK node.

Note: Creating a virtual node is free of charge, while real-time fees are incurred for running Pods on the virtual node; The virtual node is Serverless, backed by SurferCloud's massive physical machine resources (i.e., Cube), so it's not possible to log into or view the monitoring of the virtual node itself. However, you can view the monitoring of the Pods on the virtual node.

Add Virtual Node

On the cluster node list page, click the "Add Virtual Node" button to add a virtual node to the UK8S cluster. A UK8S cluster currently supports adding up to 8 virtual nodes. The added virtual node name is uk8s-xxxxxxxx-vk-xxxxx. This name will be registered as the virtual node's spec.nodeName, where uk8s-xxxxxxxx is the UK8S cluster ID and the last five digits are a randomly generated combination of numbers and letters.

Virtual Node Management

Node Description

Like a regular Node node, a VK node is also a Node object within the UK8S cluster. When managing from the command line, kubectl get nodes and other commands can be used to manage and view nodes. On the cluster list page, click the "Node Description" button to also view the detailed information and status of the VK node, as well as the Cube Pod generated by the node and node events.

Disable and Delete

You can disable and delete the VK node on the control panel page. After disabling the VK node, applications will not be able to create Cube instances through the VK node, while existing Cube instances created through the VK node will be retained. When a VK node is deleted, Cube instances created through the VK node and the attached UDisk will be deleted by default.

Create Cube Instance Via Virtual Node

Creating Cube instances through a VK node is similar to standard Pod resources, but it requires adding nodeName or nodeSelector to point to the VK node and adding taint tolerations in the yaml file Pod spec. It supports directly creating Pods, or managing Pods through controllers such as Deployment and StatefulSet.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  # If the pod does not need to communicate with the api server, it is recommended not to mount the token. To skip the mount, add the following field to the pod spec.
  # If you must use the service account token, please refer to the following regarding the "1.22 version service account token automatic mounting problem"
  automountServiceAccountToken: false
  # Schedule to VK node using nodeSelector, only one of nodeName and nodeSelector needs to be configured.
  # Be sure to use nodeSelector when creating in the form of Deploy or Sts.
  # After creating the virtual node, you can also add designated tags to the virtual node for Pod scheduling management.
  nodeSelector:
    type: virtual-kubelet
  # Schedule to VK node by specifying nodeName.
  # nodeName: uk8s-xxxxxxxx-vk-xxxxx
  # Adding node toleration
  tolerations:
  - key: virtual-kubelet.io/provider
    operator: Equal
    value: SurferCloud
    effect: NoSchedule
  containers:
  - name: nginx
    image: uhub.surfercloud.com/ucloud/nginx:latest
    # Specify the Pod resource configuration through the following parameters (if you do not use the default configuration). For specific support specifications refer to the table below.
    resources:
      requests:
        cpu: "2"
        memory: 2048Mi

When creating a Cube instance, note that certain CPU / memory specifications have certain ratios and restrictions. VK supports creating Cube with the following specification configurations:

Detailed Annotation Description

When creating a Cube via the VK plugin, corresponding fields can be added in Pod annotations, or Deployment / Statefulset spec.template.annotations for further configuration of the Cube. The corresponding annotation explanations are as follows:

Note: 1.22 version supports service account token automatic mounting problem

Starting from 1.22 version, the service account token mount defaults to the projected volume mode, which is not supported by cube.

If the pod does not need to communicate with the api server, the token can be omitted; skip the mount by adding the following field to the pod spec

automountServiceAccountToken: false

If the pod relies on the service account token, it can be explicitly specified to mount in the form of secret

Use default service account as example:

If it is a 1.24 version or higher cluster, since the corresponding token is not automatically generated when creating the service account, a secret needs to be created first:

apiVersion: v1
kind: Secret
metadata:
  name: default-token
  annotations:
    kubernetes.io/service-account.name: default
type: kubernetes.io/service-account-token

In 1.22 version, this step is unnecessary, execute kubectl get secret to find the secret name corresponding to the "default" service account.

When creating Deployment, explicitly specify service account, volume and mount:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      # Specify deployment to virtual node
      nodeSelector:
        type: virtual-kubelet
      tolerations:
      - effect: NoSchedule
        key: virtual-kubelet.io/provider
        operator: Equal
        value: SurferCloud
      serviceAccount: default
      containers:
      - name: nginx
        image: uhub.surfercloud.com/ucloud/nginx:latest
        ports:
        - containerPort: 80
        # Explicit mount
        volumeMounts:
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: default-token
          readOnly: true
      volumes:
      # Explicitly specify secret as volume
      - name: default-token
        secret:
          defaultMode: 420
          secretName: default-token # In 1.22 version, add token suffix
 

UDisk Persistent Volume Support

Users can declare PVC persistent volumes for Cube instances on VK nodes to create and mount UDisk persistent volumes. This task is completed jointly by the CSI UDisk and Virtual Kubelet components. The usage of the storage class and persistent volume claim is the same as when using persistent volumes normally in UK8S (Including creating new UDisk and using existing UDisks, see: Using UDisk in UK8S).

Here's an example:

## Create StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ssd-csi-udisk
provisioner: udisk.csi.ucloud.cn
parameters:
## We recommend using SSD or SATA cloud disks to avoid disk mount failures due to inconsistency between the RSSD cloud disk RDMA area and Cube.
  type: "ssd"
## XFS file system is not supported.
  fsType: "ext4"
## Recycle strategy, support Delete and Retain, default is Delete, not required
reclaimPolicy: Delete
## If the binding mode is set to WaitForFirstConsumer, you can only specify the VK node using pod.spec.nodeSelector
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true # Declare that this storage class supports expandable feature
---
## Create PersistentVolumeClaim PVC
## Volume Expansion (dynamic volume expansion) is not supported.
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: logdisk-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ssd-csi-udisk
  resources:
    requests:
      storage: 20Gi
---
## Use PVC in Pod
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  tolerations:                               
  - effect: NoSchedule
    key: virtual-kubelet.io/provider
    operator: Equal
    value: SurferCloud
## When the binding mode is set to WaitForFirstConsumer, you can only specify VK node using pod.spec.nodeSelector
  nodeSelector:
    type: virtual-kubelet
  containers:
  - name: http
    image: uhub.surfercloud.com/ucloud/nginx:latest
    volumeMounts:
    - name: log
      mountPath: /data
  volumes:
  - name: log
    persistentVolumeClaim:
      claimName: logdisk-claim

Use Own Image Repository

1. Create Own Image Repository

Take own hub address myhub.surfercloud.com as an example, additional information required includes image repository IP address and associated vpc. Username and password are not required. Refer to https://docs.surfercloud.com/cube/userguide/self_repository (opens in a new tab)

cat > .dockerconfigjson <<EOF
{
  "auths": {
    "myhub.surfercloud.com": {
      "username": "user-xxx",
      "password": "passwd-yyy",
      "registryaddr": "10.x.y.z",
      "vpcid": "uvnet-xxx"
    }
  }
}
EOF
kubectl create secret docker-registry myhub --from-file=.dockerconfigjson

2. When creating pods, specify imagePullSecret

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-myhub-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      automountServiceAccountToken: false
      nodeSelector:
        type: virtual-kubelet
      tolerations:
      - key: virtual-kubelet.io/provider
        operator: "Equal"
        value: SurferCloud
        effect: NoSchedule
      containers:
      - name: nginx
        image: myhub.surfercloud.com/test/nginx:latest
        ports:
        - containerPort: 80
      imagePullSecrets:
      - name: myhub
 

Virtual Node Capability Restrictions

Network

Supports the use of VPC, on the same level as cloud hosts; does not support features like hostNetwork, CNI, Calico Policy, etc.

Pod Storage

Containers support 10GiB of temporary storage space, modifications to temporary storage will be lost after Pod restarts. If needed, it is recommended to mount persistent storage. Persistent storage: Supports UDisk, NFS, ConfigMap, Secret mounts, does not support other types of volumes, such as hostPath, projected volume, etc. Note: When mounting ConfigMap and Secret, the total configuration size cannot exceed 30MiB.

Image Repository

Supports pulling images from UHub and self-built image repositories in the same region (only those using UHost as self-built repositories), does not support pulling images from external networks such as DockerHub; and the image size cannot exceed 10GiB.

Other Features

Supports basic features such as logs, exec, monitoring, etc.; can control with Kubernetes controllers like Deployment, StatefulSet etc.
Other compatibilities are the same as native Kubernetes Pods.