Kubernetes · Chapter 26 of 28
Troubleshooting
By the end of this chapter you can find out why a pod does not start, why it keeps restarting, why it runs but gets no traffic, and why a request from outside never arrives, with the same few commands in the same order every time, and read each failure's real message. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Packaging and operating
- 5 tasks
- Kubernetes
- In the all labs plan
The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.
Why you need this
Dhanteras is on Friday 6 November. Between 7 and 10 pm Jhatpat expects four times a normal evening's orders, and Ritika's rule is that a broken service is found and fixed in minutes, not the 40 minutes it took last year. Bhola will be on the floor of the Aliganj dark store with a tablet, and when something breaks he does not send a stack trace: he sends "the rider app shows no riders" or "customers get a timeout".
You have met every kind of failure in this course, one chapter at a time, each in a chapter that told you what it was about. A real evening does not: five things break at once in one namespace, and nothing says whether it is the image, the config, a probe, a Service or a policy. What you need now is not a new object but a method: a fixed order of questions, each answered by one command, that narrows any failure down to the one field that is wrong.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
Ticket 1: the rider app shows no riders
The Deployment rider-tracker in ch26 has no running pods. Find why and fix it.
1. List the pods. The STATUS of the rider-tracker pods says which kind of failure it is.
2. Describe one of those pods. Read the last events.
3. Fix the Deployment rider-tracker in place. lab images lists the images the lab's registry has.
4. In tickets.txt, write the cause in a few words of your own, after 1:
To pass: rider-tracker still has 2 replicas, runs a jhatpat/rider-tracker image, and both pods are ready. Line 1: of tickets.txt says what was wrong with the image. Change the Deployment: do not delete it and make it again.
- Task 2
Ticket 2: no SMS to customers
The notifier pod in ch26 never starts. Find why and fix it.
1. List the pods and describe the notifier pod. Read its events.
2. Compare what its env asks for with what the Secret notifier-keys really has.
3. Fix it in place, in the Deployment or in the Secret.
4. In tickets.txt, write the cause after 2:
To pass: the notifier pod is ready. SMS_API_KEY still comes from the Secret notifier-keys with a secretKeyRef, not a value written into the Deployment. Line 2: of tickets.txt says which key of which Secret was wrong. Do not delete the Deployment and make it again.
- Task 3
Tickets 3 and 4: the shop page restarts and cannot reach the catalog
The storefront pods in ch26 keep restarting, and their page says the catalog is unreachable. There are two separate causes.
1. Describe a storefront pod. Its events say who restarts it and why.
2. Read the storefront logs. They name the address it cannot reach.
3. Follow that address to its Service and its endpoints. Compare the Service's selector with the catalog pods' labels.
4. Fix both things in place. Write the causes after 3: and 4: in tickets.txt.
To pass: storefront keeps a liveness probe, on a path its app answers. Its 2 pods are ready and have no restarts. The Service catalog sends to the catalog pods, so the storefront page lists the products. Line 3: says what was wrong with the liveness probe, and line 4: says what was wrong with the catalog Service. Do not delete the Deployments or the Services and make them again.
- Task 4
Ticket 5: customers get a timeout
lab curl http://aliganj.jhatpat.in/ waits and then fails. Find what blocks it and fix it.
1. Do tickets 3 and 4 first. The page needs storefront and the catalog working.
2. Describe the NetworkPolicy storefront-ingress. Read who it lets in.
3. Fix its rule so the ingress controller can come in. Its pods are in the namespace ingress-nginx.
4. Run lab curl http://aliganj.jhatpat.in/ again. Write the cause after 5: in tickets.txt.
To pass: storefront-ingress still exists and still selects the storefront pods (app=storefront). It has no rule that lets everyone in. lab curl http://aliganj.jhatpat.in/ gets the shop page with its products. Line 5: says what blocked the requests.
- Task 5 · interview question
Interview: the first three commands
Interview question: a pod is not Running. What are your first three commands?
1. Ammar will ask: "A pod is not Running and you have two minutes. What are your first three commands, and what does each one tell you?"
2. Write the three commands in answer.txt, below the notes, in order, one on each line.
3. After each command, write a few words on what it tells you.
To pass: answer.txt names the command that lists the pods, the command that describes a pod, and the command that reads its logs. It also says what describe shows you. Lines that start with # do not count.
What you will learn
- The method: get, describe and its events, logs and --previous, exec, -o yaml, endpoints, lab curl.
- Pods that never start: ImagePullBackOff, CreateContainerConfigError, Pending (resources, taints, node selectors, PVCs).
- Pods that start and stop: CrashLoopBackOff, OOMKilled, a failing liveness probe.
- Pods that run but get no traffic: not Ready, a Service with no endpoints, a wrong targetPort.
- Requests from outside: an Ingress 503 and a NetworkPolicy timeout.
More in Packaging and operating
Build "Troubleshooting" yourself
Jhatpat delivers groceries in ten minutes from 14 dark stores across Lucknow, and its hand-run servers fall over on festival evenings. You move it onto Kubernetes. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.