Go · Chapter 20 of 26
Worker pools
By the end of this chapter you can run many jobs with a fixed number of workers, shut a pool down cleanly, collect results in order and keep the first error, and bound how much runs at once with errgroup's SetLimit and with a semaphore. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Concurrency
- 4 tasks
- Go
- In the all labs plan
The first 2 chapters of Go are free. This one comes with the all labs plan.
Why you need this
In chapter 15 you asked for 50 statuses with 50 goroutines, and it was fast. Then Tenzin ran the same code for all 1,180 chargers against the real central system, and the central system's team called him within a minute. Their database serves a handful of queries at a time; 1,180 arriving together filled its connection pool, every other service that used it slowed down, and the status page timed out anyway. Their rule for every service now: never more than 8 calls at once (16 for small heartbeats, 5 for the status page, which shares the limit with other pages).
A goroutine for each job does not care about anyone's limits. It starts everything at once, because that is what goroutines are for. What you need is the same work done concurrently, but never more than N at a time, however many jobs there are. That is a worker pool, and Go builds one from things you already know: goroutines, channels, a WaitGroup and a context.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
pool.go: 8 workers for all the statuses
Write AllStatuses(ctx context.Context, c *bijli.Central, ids []string) ([]bijli.Status, error) in pool.go. It gets the status of every charger in ids with a pool of exactly 8 worker goroutines that take the jobs from a channel and send the results back on another channel, passing ctx to every c.ChargerStatus call.
When every lookup answers, return the statuses in the order of ids and nil. If any lookup fails, return nil and one of the errors: with C9999 among the ids, an error that wraps bijli.ErrUnknownCharger; with a ctx already cancelled, context.Canceled.
The Check asks for all 1,180 chargers with a fake clock and 40 ms a lookup: exactly 1,180 calls, and at most exactly 8 in flight at once (the central system's MaxInFlight must be 8). When AllStatuses returns, none of its goroutines may still be running, after an error too.
- Task 2
group.go: an errgroup that stops at the first error
Write CheckAll(ctx context.Context, c *bijli.Central, ids []string) error in group.go with an errgroup from golang.org/x/sync/errgroup. It asks c for the status of every charger in ids, at most 8 at once, and returns nil if they all answer. At the first error it stops the rest at once and returns that error.
The Check uses a fake clock and 20 ms a lookup. For all 1,180 chargers with C1003 down, CheckAll must return an error that wraps bijli.ErrChargerDown exactly when C1003 fails, at 20 ms, with at most 8 calls in flight at once. With nothing down it returns nil, after one call for each of the 1,180 chargers, with at most 8 and at least 2 in flight at once. The group made with a context gives you a new context: the lookups must use that one, or they never hear that it is time to stop.
- Task 3
limiter.go: at most 5 calls at once, from anywhere
Write Limited in limiter.go: NewLimited(c bijli.Central, n int64) Limited and (l *Limited) ChargerStatus(ctx context.Context, id string) (bijli.Status, error). However many goroutines call it, at most n calls to c run at once; the others wait for a free slot. ChargerStatus returns what c.ChargerStatus(ctx, id) returns. A caller waiting for a slot gives up when its ctx ends, returning ctx.Err() without calling the central system. A slot is always given back, even when the call fails. A weighted semaphore from golang.org/x/sync/semaphore can hold the slots.
The Check uses a fake clock. 50 goroutines call a Limited with n = 5 at once, 40 ms a lookup: every status must be right, exactly 5 calls in flight at once, 400 ms in all. With n = 1 and one call of 1 s running, a caller whose ctx ends after 30 ms must get context.DeadlineExceeded after 30 ms, and the central system must have had only the one call. With n = 1, after a call for C9999 fails, the next call must still get a slot.
- Task 4 · interview question
Interview: record.go, a goroutine for every heartbeat
Tenzin's RecordAll(ctx context.Context, c *bijli.Central, hbs []bijli.Heartbeat) error records a batch of heartbeats with one goroutine for each, all at once: for 1,180 heartbeats, 1,180 calls arrived at the central system together.
Fix it so that it records every heartbeat with c.RecordHeartbeat(ctx, hb), at most 16 calls in flight at once and at least 2 (still at the same time), and returns nil when all are recorded or one of the errors otherwise (a heartbeat from C9999 gives an error that wraps bijli.ErrUnknownCharger). The Check records one heartbeat for each of the 1,180 chargers with a fake clock, 5 ms each: every charger must then have exactly one heartbeat, with at most 16 calls in flight at once.
Joseph will ask: goroutines are cheap, so why not one goroutine for each of a million jobs?
What you will learn
- A worker pool: N goroutines that take jobs from one channel and send results on another.
- Shutting down cleanly: who closes which channel, and when.
- Results in order and errors: carry the index with the job, keep the first error.
- errgroup.Group: Go, Wait and the first error; WithContext cancels the rest; SetLimit bounds them.
- A semaphore: N slots, shared by any number of goroutines (golang.org/x/sync/semaphore, or a channel).
- How many workers, measured.
More in Concurrency
Build "Worker pools" yourself
BijliGo, a Bengaluru EV charging network, drops chargers at the evening peak. You rewrite its backend in Go. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Go are free. This one comes with the all labs plan.