Kubernetes · Chapter 18 of 28

DaemonSets and init containers

By the end of this chapter you can run exactly one pod on every node with a DaemonSet (payments-1 too), roll it out to a new version, make a pod wait for what it needs with init containers, read why a pod is stuck in Init, and say what native sidecars and static pods are. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: More workloads and traffic
  • 4 tasks
  • Kubernetes
  • In the all labs plan

The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.

Why you need this

Every Jhatpat dark store has barcode scanners wired to the machine in its back room, and the store agent reads them: one agent per machine, no more, no fewer. Today Bhola's team counts 4 machines and Tushar keeps 4 copies running by hand. When a fifth node joins for Dhanteras, nobody will remember. The second need is order. After the database was restored on Saturday, the orders service started first and crashed 9 times in a row (connect ECONNREFUSED), and the dashboards showed red for 6 minutes while the database warmed up.

A Deployment cannot say "one per node". You set replicas by hand; chapter 13's anti-affinity keeps two apart, but the count is still yours, and payments-1 is still kept away by its taint. And nothing you know yet says "do not start this container until that one answers": the containers of a pod start together.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    store-agent.yaml: the store agent on every node

    Run one store agent on every node, including the payments node.

    1. Open store-agent.yaml. Change it from a Deployment to a DaemonSet named store-agent. Remove the replicas line.

    2. Keep the image jhatpat/store-agent:1.0 on port 9100.

    3. Add a toleration for the taint dedicated=payments:NoSchedule. Tolerate only that taint, not every taint.

    4. Apply the file. If you made a Deployment store-agent before, delete it.

    To pass: the DaemonSet store-agent in ch18 has 4 ready pods, one on each node: worker-1, worker-2, worker-3 and payments-1. A toleration with operator Exists and no key does not count.

  2. Task 2

    orders.yaml: wait for the database

    Make orders wait until the database answers, so it never starts too early.

    1. Open orders.yaml. Add an init container named wait-for-db that runs busybox:1.37.

    2. Its command loops until nc -z postgres 5432 works, with a short sleep between tries.

    3. Apply orders.yaml while postgres is still at 0. The orders pod shows Init:0/1.

    4. Scale the Deployment postgres to 1. Watch orders start.

    To pass: postgres has 1 ready pod. The orders pod is ready, its wait-for-db ended with exit code 0, and the orders container never restarted. The command must loop (until or while), not try only once.

  3. Task 3

    notifier.yaml: why it is stuck in Init

    Tushar's notifier (2 replicas) is stuck in Init:0/1. Find out why and fix it.

    1. Read the init container's logs. Use kubectl logs with -c wait-for-queue.

    2. List the Services with kubectl get services. Compare the names.

    3. Fix the Service name in the init container's command in notifier.yaml. Keep the init container and its loop.

    4. Apply the file and wait for the rollout.

    To pass: you ran kubectl logs with -c wait-for-queue. The init container waits for the queue Service that really exists, and exactly 2 notifier pods are ready with that fix.

  4. Task 4 · interview question

    Interview: DaemonSet or anti-affinity

    Interview question: why is one store agent per node a DaemonSet, and not a Deployment with pod anti-affinity?

    1. Open answer.txt. Write below the lines that start with #.

    2. In two or three sentences of your own, say what the DaemonSet does when a node joins.

    3. Say what a Deployment with anti-affinity needs instead.

    To pass: your answer names the DaemonSet, says what happens when a new node joins, and says what the Deployment needs. Lines that start with # do not count. Ammar will ask: "A Deployment with anti-affinity also puts one pod on each node. Why a DaemonSet?"

What you will learn

  • DaemonSets: one pod on every node that fits, made and removed as nodes come and go.
  • Tolerations for a DaemonSet: reaching a tainted node like payments-1.
  • Rolling updates of a DaemonSet, node by node.
  • Init containers: run one after another before the app; the Init:0/1 status.
  • Waiting for a dependency with nc -z, and reading why a pod is stuck in Init.
  • Native sidecars (restartPolicy: Always in initContainers), and static pods in one paragraph.

More in More workloads and traffic

Build "DaemonSets and init containers" yourself

Jhatpat delivers groceries in ten minutes from 14 dark stores across Lucknow, and its hand-run servers fall over on festival evenings. You move it onto Kubernetes. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Kubernetes are free. This one comes with the all labs plan.