Flexvolume Upgrade to CSI

For Kubernetes v1.13 and earlier users, Pods use UDisk block storage volumes mounted through Flexvolume. As it does not support basic features such as topology-aware dynamic scheduling, the Flexvolume solution has ceased to evolve, and CSI has become the standard for container storage implementation.

Early UK8S users who created UDisk mount volumes using Flexvolume are currently facing the problem of converting Flexvolume PV to CSI PV during cluster upgrades. This document provides an example on how to complete this conversion.

⚠️ Upgrades will cause service interruptions, please plan the migration time reasonably, and backup accordingly.

1. Flexvolume UDisk storage volume description

Below is a yaml file nginx-fv.yaml for a Workload that has mounted a Flexvolume UDisk storage volume, which includes a StorageClass, a Pod declaration, and its referenced PVC.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: udisk-ssd-flexvolume
parameters:
  type: ssd # Disk type, enumerations are ssd,sata,rssd
provisioner: ucloud/udisk
reclaimPolicy: Delete
volumeBindingMode: Immediate
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nginx-fv
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: udisk-ssd-flexvolume
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: uhub.surfercloud.com/ucloud/nginx:latest
    imagePullPolicy: Always
    ports:
    - containerPort: 8080
    volumeMounts:
    - name: nginxdisk
      mountPath: /data
  volumes:
  - name: nginxdisk
    persistentVolumeClaim:
      claimName: nginx-fv

Execute kubectl apply -f nginx-fv.yaml and find that the corresponding StorageClass, Pod, and associated PVC have been successfully created. Also, the cloudprovider will create and bind a PV object (name is the same as the Volume value of the PVC object) to this PVC, as shown below:

Name:            pvc-8b7946f7-1214-11ec-8f6b-5254003e805f-bsm-upc4bc0v
Labels:          <none>
Annotations:     pv.kubernetes.io/provisioned-by: ucloud/udisk
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    udisk-ssd-flexvolume
Status:          Bound
Claim:           default/nginx-fv
Reclaim Policy:  Delete
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        10Gi
Node Affinity:   <none>
Message:         
Source:
    Type:       FlexVolume (a generic volume resource that is provisioned/attached using an exec based plugin)
    Driver:     SurferCloud/flexv
    FSType:     
    SecretRef:  nil
    ReadOnly:   false
    Options:    map[diskId:bsm-upc4bc0v]
Events:         <none>

In Source.Options, we can find that the actual UDisk instance corresponding to this PV is bsm-upc4bc0v.

2. Upgrade steps

⚠️ Upgrades will cause service interruptions, please plan the migration time reasonably, prepare all necessary yaml files, and backup accordingly.

2.1 Verify that the original PV recycle policy is Retain

If the PV's recovery policy is not Retain, you will need to use the following command to change its recovery policy to Retain. Even if you delete the Pod and the corresponding PVC, you will find that the PV still exists and the corresponding UDisk instance is also preserved.

⚠️ If the PV created by Flexvolume is deleted, the corresponding UDisk will be deleted, if you need the corresponding UDisk, please ensure that this PV is not deleted.

kubectl patch pv <your-pv-name1> <your-pv-name2> <your-pv-name3> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

2.2 Verify that the CSI plugin is installed in the cluster

First, please refer to the command line upgrade method in the CSI Upgrade Guide to install the latest CSI plugin for the cluster.

2.3 Delete original Pod and PVC

By kubectl delete -f nginx-fv.yaml, delete the original PVC and Pod, you will find that due to the PV recycling policy is Retain, PV and the corresponding UDisk are still retained, and the PV state is Release.

2.4 Create a new PV and PVC using CSI to specify UDisk

Next, we will create a PV with the original UDisk data disk and associate the PVC. See the "Using Existing UDisk Part" in the Using UDisk in UK8S document.

Below is an example file nginx-csi-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nginx-csi
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 10Gi
  csi:
    driver: udisk.csi.ucloud.cn
    volumeAttributes:
      type: ssd
    volumeHandle: bsm-upc4bc0v # Please change it to your own UDiskId
  persistentVolumeReclaimPolicy: Retain
  storageClassName: udisk-ssd
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nginx-csi
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: udisk-ssd
  resources:
    requests:
      storage: 10Gi
  volumeName: nginx-csi

After executing kubectl apply -f nginx-csi-pv.yaml, we can see that the new PV and PVC have been successfully created.

2.5 Mount the PVC to the corresponding Pod

For details, see the Using UDisk in UK8S document. Below is an example yaml file nginx-csi.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: uhub.surfercloud.com/ucloud/nginx:latest
    imagePullPolicy: Always
    ports:
    - containerPort: 8080
    volumeMounts:
    - name: nginxdisk
      mountPath: /data
  volumes:
  - name: nginxdisk
    persistentVolumeClaim:
      claimName: nginx-csi # Note the name corresponds to the new PVC

Execute kubectl apply -f nginx-csi.yaml and find that the new Pod has been successfully created and bound to the corresponding PVC.

After successful upgrades, keep the original FlexVolume installation file, as long as declare your new StorageClass, PV, and PVC follow the respective CSI standards.