Kubernetes · Chapter 11 of 28
Probes
By the end of this chapter you can tell the kubelet how to check that an app is alive, ready for traffic and done starting, with readiness, liveness and startup probes; choose their timings; read their events; and keep a slow or sick pod out of a Service without restarting it for no reason. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Running apps
- 4 tasks
- Kubernetes
- In the all labs plan
The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.
Why you need this
A pod that is Running is not the same as an app that answers. Orders takes 25 seconds to load 1,18,402 pin codes before it can take an order. On Dhanteras evening, when the storefront sends about 1,200 requests a second, every new orders pod that gets traffic during those 25 seconds turns customers' checkouts into errors. And an app can hang after a week of running (a deadlock, a full queue): it is still Running, it still gets traffic, and it answers nothing.
The Kubernetes you know so far looks only at the process. Here is orders with no probe, behind a NodePort Service:
$ kubectl get pod orders-demo
NAME READY STATUS RESTARTS AGE
orders-demo 1/1 Running 0 4s
$ lab curl http://10.20.0.12:31200/orders
curl: (7) Failed to connect to 10.20.0.12 port 31200 after 0 ms: Couldn't connect to server
lab: pod orders-demo is not listening yet (Running)
READY says 1/1 while orders cannot answer: with no readiness probe, a container counts as ready the moment it starts, so its Service sends it traffic at once. 25 seconds later the same request works:
$ lab curl http://10.20.0.12:31200/orders
{"orders":[{"id":"JP-LKO-10231","items":4,"total":412,"status":"delivered","minutes":9}]}
Nothing restarts a hung process either, because the process has not exited. Probes let the app tell the kubelet what "working" means.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
catalog.yaml: readiness and liveness for the catalog
Give the catalog two probes: one for traffic, and one for restarts.
1. Open catalog.yaml with nano.
2. Add a readiness probe to the catalog container: an HTTP GET on /ready, port 8080.
3. Add a liveness probe: an HTTP GET on /healthz, port 8080. The timings are your choice.
4. Keep 3 replicas, and apply the file.
To pass: both probes are in the Deployment, 3 catalog pods with both probes are ready, and none of them has restarted.
- Task 2
orders.yaml: a startup probe for the slow starter
Stop orders' restart loop with a startup probe.
1. Describe an orders pod. orders needs 25 seconds to warm up, and its liveness probe kills it before that. Read the events.
2. Open orders.yaml with nano. Keep the liveness probe: an HTTP GET on /healthz, port 8080.
3. Add a startup probe: an HTTP GET on /healthz, port 8080. It must allow at least 30 seconds: initialDelaySeconds plus periodSeconds times failureThreshold.
4. Apply the file. Wait about 30 seconds after the rollout.
To pass: the liveness probe is still there, the startup probe allows 30 seconds or more, and the 2 orders pods with the startup probe are ready and have never restarted.
- Task 3
orders.yaml: keep a warming pod out of traffic
Keep a warming orders pod out of traffic with a readiness probe.
1. Open orders.yaml with nano.
2. Add a readiness probe to the orders container: an HTTP GET on /ready, port 8080.
3. Apply the file. Watch a new pod's READY column while it warms up.
To pass: the probe is in the Deployment, and the newest orders pod became ready 20 seconds or more after it started. It is also a ready address in the orders Service's EndpointSlice.
- Task 4 · interview question
Interview: liveness or readiness
Interview question: the database is down, so should orders fail its liveness probe or its readiness probe?
1. Say what the kubelet does when a liveness probe fails. Does that bring the database back?
2. Say what would happen to every orders pod if its liveness probe checked the database.
3. Say which probe should notice the database instead, if any.
4. Write two or three sentences of your own below the notes in answer.txt.
To pass: answer.txt names the readiness probe, and says what a liveness probe that checks the database would do to the orders pods.
What you will learn
- The kubelet's three probes: readiness (traffic), liveness (restarts) and startup (slow starters).
- Three kinds of check: httpGet, tcpSocket and exec.
- Timings: initialDelaySeconds, periodSeconds, failureThreshold (and timeoutSeconds, successThreshold).
- A liveness probe that kills a slow starter in a loop, with its real events.
- Readiness takes a pod out of the Service's EndpointSlice, without restarting it.
- Startup probes for slow starters, and why probes should not check dependencies.
More in Running apps
Build "Probes" yourself
Jhatpat delivers groceries in ten minutes from 14 dark stores across Lucknow, and its hand-run servers fall over on festival evenings. You move it onto Kubernetes. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.