Kubernetes · Chapter 14 of 28
Taints and tolerations
By the end of this chapter you can keep a node for one team with a taint, let only that team's pods onto it with a toleration, take a node out of service with cordon and drain, keep enough copies running while you do with a PodDisruptionBudget, and put the node back. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Scheduling and teams
- 4 tasks
- Kubernetes
- In the all labs plan
The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.
Why you need this
payments-1 is Jhatpat's payments node. The payment gateway's auditors allow card traffic only from machines that run nothing but payments, and that node is the one they audited. If a catalog or notifier pod lands there, Jhatpat fails the audit, and 31% of its orders (paid by card and UPI through that gateway) stop. So two things must be true: no other pod ever runs on payments-1, and the payments pods run there and nowhere else.
The second need is maintenance. worker-2 needs a kernel patch today, which means a reboot. The catalog's three copies all run on worker-2 right now (Ammar made the catalog prefer lucknow-b). Switch the machine off, and the storefront has no catalog until three new pods start somewhere else.
What you know so far does not cover either. A nodeSelector or node affinity (chapter 13) pulls the pods that carry it, but does nothing to the pods that do not: any other pod may still land on payments-1. And deleting the catalog pods by hand removes all three at once.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
payments.yaml: payments only on payments-1
Make payments run on payments-1, and on no other node.
1. Describe payments-1 and read its taint. Show the nodes with their jhatpat.in/pool label as a column.
2. Open payments.yaml with nano. Keep 2 replicas of jhatpat/payments:1.5, and its key from the Secret payments-gateway.
3. In the pod template, add a toleration for the taint dedicated=payments:NoSchedule: key dedicated, value payments, effect NoSchedule.
4. Add a node rule (a nodeSelector or a required node affinity) that payments-1 matches and no worker node matches.
5. Apply the file, and list the pods wide.
To pass: the toleration is for this one taint (not for every taint), no worker node fits the node rule, and 2 payments pods are ready, both on payments-1.
- Task 2
catalog-pdb.yaml: drain worker-2, keep 2 catalog pods
Drain worker-2 safely, and keep at least 2 catalog pods up the whole time.
1. Open catalog-pdb.yaml with nano. Make a PodDisruptionBudget named catalog that selects app: catalog.
2. Make it keep at least 2 pods: minAvailable: 2, or maxUnavailable: 1. Apply the file.
3. Only then, drain worker-2 with kubectl drain. If it stops with an error about some pods, add the flag that the error names.
4. Wait until it prints node/worker-2 drained.
To pass: the budget selects only app: catalog and existed before the drain, your kubectl drain of worker-2 worked (a drain that ended with an error does not count, so run it again), and 3 catalog pods are ready with none of the old ones left on worker-2.
- Task 3
Put worker-2 back
Put worker-2 back in service after the drain.
1. List the nodes. After a drain, worker-2 shows SchedulingDisabled.
2. Make worker-2 schedulable again.
3. Describe worker-2. Its Taints line must say none.
To pass: you drained worker-2 in task 2, worker-2 is now schedulable and has no taints, and 3 catalog pods are ready. The old pods do not move back, and that is fine.
- Task 4 · interview question
Interview: a taint or node affinity
Interview question: to keep other pods off payments-1, would you use a taint or node affinity?
1. Say what a taint does to pods that have no toleration.
2. Say what node affinity or a nodeSelector does, and to which pods.
3. Say why payments needs both a toleration and a node rule.
4. Write two or three sentences of your own below the notes in answer.txt.
To pass: answer.txt talks about the taint, about node affinity (or a nodeSelector), and about the toleration.
What you will learn
- Taints: a mark on a node that turns pods away (NoSchedule, PreferNoSchedule, NoExecute).
- Tolerations: a pod's permission to run on a tainted node, and why it is not a request to run there.
- A dedicated node: a toleration plus a node rule, and why you need both.
- cordon, drain, uncordon: taking a node out of service and putting it back.
- PodDisruptionBudget: how many copies must stay up while nodes are drained.
More in Scheduling and teams
Build "Taints and tolerations" yourself
Jhatpat delivers groceries in ten minutes from 14 dark stores across Lucknow, and its hand-run servers fall over on festival evenings. You move it onto Kubernetes. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.