Scaling, Scheduling, and Resource ManagementLesson 5.3

Kubernetes taints and tolerations: reserving nodes for specific workloads

taint definition, taint effects (NoSchedule, PreferNoSchedule, NoExecute), toleration syntax, use cases for taints, taint-based eviction, control-plane taint, kubectl taint command

Taints Repel Pods; Tolerations Override That

Kubernetes taints and tolerations diagram

Node affinity attracts Pods to nodes. Taints do the opposite — they repel Pods. A node with a taint will not accept Pods unless the Pod has a matching toleration. This is how control plane nodes stay empty of user workloads.

Adding Taints to Nodes

# Taint a node for GPU-only workloads
kubectl taint node gpu-node-1 workload=gpu:NoSchedule

# Remove a taint (trailing dash)
kubectl taint node gpu-node-1 workload=gpu:NoSchedule-

Taint Effects

NoSchedule — new Pods without the toleration will not be scheduled. Existing Pods are unaffected. PreferNoSchedule — scheduler tries to avoid this node but will use it if no other options exist. NoExecute — evicts existing Pods that do not tolerate the taint, in addition to blocking new ones.

Adding Tolerations to Pods

spec:
  tolerations:
  - key: "workload"
    operator: "Equal"
    value: "gpu"
    effect: "NoSchedule"
  # Tolerate ANY taint with NoExecute (for node failure scenarios)
  - key: "node.kubernetes.io/unreachable"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 300   # wait 5 min before evicting