Rust · Chapter 23 of 26

Async and tokio

By the end of this chapter you can write async functions and await them, run them on tokio's runtime with #[tokio::main] and #[tokio::test], run many at the same time with tokio::spawn, join! and JoinSet, and keep a task from blocking every other task on its thread. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Concurrency and services
  • 4 tasks
  • Rust
  • In the all labs plan

The first 2 chapters of Rust are free. This one comes with the all labs plan.

Why you need this

At 9 am every one of the 120 boxes sends its morning to the plant, and Ishita's screen, Neha's payout run and Lakhan's field app talk to the plant's service at the same time. Most of that time is spent waiting: an upload waits 20 ms for the plant to answer, a box on a village mobile network waits far longer for its next line. In chapter 20 every upload got a thread of its own, and it worked: 120 uploads in about the time of one.

But a thread is a heavy thing to keep for waiting. Each one gets its own stack and is scheduled by the operating system. Measured below, 10,000 threads that each wait 200 ms took 93 MB of memory and 0.68 seconds; 10,000 tokio tasks doing the same took 6 MB and 0.2 seconds, on a single thread. The plant's service will hold a connection open for every box all day (chapter 25), and a thread for each is memory spent on doing nothing. Rust's answer is async: many waits share a few threads, and a thread is never left sleeping while there is work to do.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    uploads.rs: every society's morning, uploaded at once

    Write the async function upload_morning(plant: Arc<Plant>, shifts: Vec<(String, Vec<Collection>)>) -> (usize, u64). It starts one task for each society in a JoinSet, each task calling plant.upload_async with that society's id and pourings, then waits for every task and returns how many uploads the plant stored and the paise of their receipts, added up. A failed upload is not stored and adds nothing.

    The Check runs it on one thread for all 120 societies' mornings, with S014's line down: 120 uploads, at least 60 of them in flight at the same moment, well under 500 ms in all (one at a time takes 2.4 s), and the result must be 119 and the paise of those 119 mornings. uploads.rs must use a JoinSet.

  2. Task 2

    lookups.rs: a farmer's card from two lookups at once

    lookups.rs has two async lookups, farmer_name(id, wait) -> Option<String> and day_paise(id, wait) -> u64, which each wait as long as they are told before they answer. Keep them as they are. Write the async function card(id: &str, wait: Duration) -> String: it runs both lookups at the same time with join! and returns one line, the id, the farmer's name and what they earned today, or, when the farmer does not exist, the id and no such farmer:

    card("F10122", wait) returns "F10122 Rajeshbhai Makwana: 55436 paise today"

    card("F19999", wait) returns "F19999: no such farmer"

    The Check tries F12345, F14800 and F19999 with a wait of 200 ms: each card must take about 200 ms (at least 190 ms, under 330 ms), not the 400 ms of one lookup after the other.

  3. Task 3

    days.rs: each society's day in batches, results in order

    The plant takes at most 50 pourings an upload. Write the async function upload_days(plant: Arc<Plant>, days: Vec<(String, Vec<Collection>)>) -> Vec<u64>. For each society it starts a task with tokio::spawn; that task sends the society's pourings in batches of at most 50 with plant.upload_async, one batch after another, and adds up the paise of the receipts (a failed batch adds 0). upload_days returns each society's paise in the same order as days, whichever task finishes first.

    The Check gives the whole day of S001, S004, S014, S002, S080 and S093, in that order, with S014's line down. It expects 19 uploads (S001's 406 pourings are 9 batches), the six numbers back in that order (S001's 12415897 first, S014's 0 third), more than one upload in flight at once, and well under 300 ms in all (S001's 9 batches take about 180 ms).

  4. Task 4 · interview question

    Interview: pause.rs, the pause that blocked every box

    The plant asks every box to wait 20 ms after it connects, before it sends. Lakhan's async function send_shift(plant: &Plant, society: &str, pourings: &[Collection]) -> Result<Receipt, PlantError> does that, then uploads with plant.upload_async. One box alone takes about 40 ms. But with 120 boxes each in its own task on one thread, they took 2.5 seconds, as if one ran after another.

    Fix send_shift so that 120 calls at once, each in its own task on one thread, take well under 500 ms. Keep the 20 ms pause: one call alone must still take at least 38 ms and return the plant's receipt (for S002's morning, 35 pourings).

    Farhan will ask: why did one sleep make 120 tasks take 2.5 s, when each task only waits 40 ms?

What you will learn

  • async fn and .await: a function that can pause while it waits, and how to wait for one.
  • Futures are lazy: calling an async fn does nothing until it is awaited.
  • tokio: the runtime that runs futures, with #[tokio::main] and #[tokio::test].
  • tokio::spawn and JoinHandle: start a task that runs on its own.
  • join! and JoinSet: wait for several at the same time.
  • Never block the runtime: std::thread::sleep against tokio::time::sleep, measured.

More in Concurrency and services

Build "Async and tokio" yourself

DoodhSetu runs the milk collection boxes of 120 village societies around Anand. The old box program crashed on an empty line, and you rebuild it in Rust. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Rust are free. This one comes with the all labs plan.