2 min read | by Jordi Prats
You can use kubectl drain to evict pods from a node and mark it as unschedulable to prevent new pods from get created on this node. It will allow the pod's containers to gracefully terminate, respecting the PodDisruptionBudgets with a few exceptions. Let's test it suing the following nodes:
$ kubectl get nodes NAME STATUS ROLES AGE VERSION nauvoo.pet2cattle.com Ready control-plane,master 19d v1.20.4+k3s1 tycho.pet2cattle.com Ready <none> 26s v1.20.4+k3s1 If we try to use kubectl drain on one of the nodes we will get the following error:
$ kubectl drain tycho.pet2cattle.com node/tycho.pet2cattle.com cordoned error: unable to drain node "tycho.pet2cattle.com", aborting command... There are pending nodes to be drained: tycho.pet2cattle.com error: cannot delete DaemonSet-managed Pods (use --ignore-daemonsets to ignore): kube-system/svclb-traefik-ndd2s DaemonSet-managed pods cannot be deleted, to be able to safely drain a node containing pods that belong to a DaemonSet we will have to use the option --ignore-daemonsets:
$ kubectl drain tycho.pet2cattle.com --ignore-daemonsets node/tycho.pet2cattle.com already cordoned WARNING: ignoring DaemonSet-managed Pods: kube-system/svclb-traefik-ndd2s node/tycho.pet2cattle.com drained Checking again the list of nodes we will see that the node is marked as SchedulingDisabled: So no new pods are going to be scheduled on this node and since we have evicted all the relevant pods; we will be able to proceed safely with it's maintenance
$ kubectl get nodes NAME STATUS ROLES AGE VERSION nauvoo.pet2cattle.com Ready control-plane,master 19d v1.20.4+k3s1 tycho.pet2cattle.com Ready,SchedulingDisabled <none> 2m59s v1.20.4+k3s1 Once you are ready to put the node back into service, you can use kubectl uncordon to make the node schedulable again.
Posted on 14/04/2021