Rust · Chapter 26 of 26
Capstone: the plant's collection service
By the end of this chapter you have built the plant's collection service end to end: boxes connect over TCP and send every can as a JSON line; the service checks each line with an error type of its own, prices it through a trait, keeps every society's totals for many connections at once, sends each closed shift to the plant with at most 8 uploads at a time and a time limit, and closes at night without losing an upload. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Concurrency and services
- 5 tasks
- Rust
- In the all labs plan
The first 2 chapters of Rust are free. This one comes with the all labs plan.
Why you need this
This is the service the whole lab has been heading for. On Monday 12 October a box crashed on an empty line and lost the cans since its last save; 1,940 pourings went onto paper and 37 came back wrong. Ishita's answer is a plant service that every box talks to all day: each can reaches the plant's books the moment it is weighed, so a box that crashes loses nothing. On Tuesday the 120 boxes sent 10,164 pourings, ₹29,32,572.64 in all; Karamsad alone sent 406 in two shifts.
Every part has been a chapter of its own: parsing and errors (9 and 14), traits (11), serde (18), shared state (22), tasks (23), limits, time limits and shutdown (24), and TCP (25). A service is all of them at once, and the hard part is where they meet: an error from serde becoming a reply, a ledger shared by every connection, a channel between the connections and the plant, and a closing time that must not drop the last uploads. Lakhan wired a first version; it lost Karamsad's evening on its first night.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
message.rs: a box's line, read and checked
Write parse_line(line: &str) -> Result<Message, LineError>. A box sends two kinds of JSON line, which Message and Pouring (given) already read with serde:
{"type":"pour","society":"S001","farmer":"F10122","shift":"morning","kind":"cow","ml":4880,"fat":42,"snf":86,"at":"06:02:07"} is Message::Pour(Pouring { .. })
{"type":"close","society":"S001","shift":"evening"} is Message::Close { society: "S001", shift: Shift::Evening }
Write LineError's messages with thiserror (derive Error, one #[error] line a variant, #[from] on Json) and remove the stand-in Display. Each message is exactly:
Json(serde_json::Error): not a message: followed by serde's own text (hello gives not a message: expected value at line 1 column 1)
UnknownSociety(id): no society S999
UnknownFarmer(id): no farmer F99999
NotAMember { farmer, society }: F10122 does not pour at S002
NoMilk: a can with no milk
BadReading { field, value }: fat 420 is not a reading
Check in this order, and return the first fault: the line is a message; the society exists (for a close too); the farmer exists; the farmer pours at that society; ml is not 0; fat is at most 150; snf is at most 150. Every one of Tuesday's 10,164 pourings, sent as a pour line, is good and reads back with its own values. The Check also sends a pour with missing fields (Json), a pour for S999 with 0 ml (no society S999), 0 ml with fat 420 (a can with no milk), fat 420 with snf 860 (fat 420 is not a reading), and a close for S999.
- Task 2
pricing.rs: pricing a pouring with the rate charts
PriceRule (given) has accepts(fat, snf) -> bool, per_litre(fat, snf) -> u64 and a default price(ml, fat, snf) -> u64 that is 0 when not accepted and otherwise (ml * per_litre + 500) / 1000. Write its two methods for doodh::Rate: accepts is true when fat is at least min_fat and snf is at least min_snf; per_litre is fat * fat_paise + snf * snf_paise.
Then write priced(p: &Pouring, id: u32) -> Collection: the Collection the pouring becomes, with the id it is given, every field of the pouring (society, farmer, shift, kind, ml, fat, snf, at), status Accepted or Rejected by the chart for its kind (doodh::rate(p.kind)), and paise from that chart's price.
The cow chart's price(6000, 41, 86) is 18192 and price(5000, 30, 80) is 12800 (exactly at the minimums); the buffalo chart's price(5250, 64, 91) is 21084 and price(5000, 45, 90) is 0. The Check prices all of Tuesday's 10,164 pourings, each with its own id: every one must equal Tuesday's Collection.
- Task 3
ledger.rs: one ledger for every connection
Write Ledger, which keeps its books behind an Arc<RwLock<..>>: a Total (pourings, ml and paise, given) for each society, and the pourings of each open shift, by society and shift. Ledger is Clone, and every clone shares the same books. Its methods:
Ledger::new() -> Ledger, with empty books.
pour(&self, c: Collection) -> Total adds the can to its society's Total (one more pouring, its ml, its paise, rejected cans too) and to its open shift (c.society and c.shift), and returns the society's Total after it.
close(&self, society: &str, shift: Shift) -> Vec<Collection> takes that shift's pourings out of the books and returns them (empty when there are none).
total(&self, society: &str) -> Total is the society's day so far, all 0 when it has none. Closing a shift does not change it.
The Check pours two S002 cans into a new Ledger (pour returns 1, then 2 pourings, with both cans' paise), then pours all of Tuesday from 4 threads, each with its own clone: total("S001") must be 406 pourings, 3532350 ml and 12415897 paise, total("S999") all 0, close("S001", Shift::Morning) must return S001's 211 morning pourings once each, a second close none, and the total must stay the same. ledger.rs must use an RwLock.
- Task 4
forward.rs: closed shifts to the plant, 8 at a time
Write the async function forward(plant: Arc<Plant>, shifts: mpsc::Receiver<(String, Vec<Collection>)>, limit: usize, wait: Duration) -> usize. It receives closed shifts (a society's id and its pourings) from the channel, and uploads each with plant.upload_async as soon as it arrives, in a task of its own. A Semaphore lets at most limit uploads run at once, and each upload is given at most wait, after which it is given up and not stored. When the channel is closed and every upload has finished, it returns how many uploads the plant stored.
The Check sends S001's morning, and 60 ms later the plant must already have been asked for it (the channel is still open). Then it sends the mornings of S002 to S030 (S014's line is down) with limit 8: 30 uploads, never more than 8 in flight but more than 1, 29 stored, well under 700 ms. The same with 12 mornings and limit 3: 12 stored. Then 10 shifts to a plant that takes 400 ms, with a wait of 50 ms and limit 8: 0 stored, in well under 350 ms.
- Task 5 · interview question
Interview: service.rs, the uploads lost at closing
Lakhan's run(listener: TcpListener, plant: Arc<Plant>, shutdown: watch::Receiver<bool>) -> Report wires the service together with your Tasks 1 to 4: a task for each box's connection answers every line ({"type":"poured","society":..,"paise":..,"total":..}, {"type":"closed","society":..,"shift":..,"pourings":..,"total":..} or {"type":"error","message":..} with the LineError's text), and a forwarder task sends each closed shift to the plant with forward (limit 8, wait 2 s). On the shutdown signal, run stops accepting, every connection's task stops, and run returns a Report of the pourings and shifts it was sent.
On its first night the service was closed right after the last boxes closed their evening, and the plant never got the last shifts. Fix run so that when it returns, every shift closed before the signal is stored at the plant.
The Check replays Tuesday for S001, S002, S014, S027 and S080 from five boxes at once (every pour and close reply must match Tuesday's data, and a bad line gets an error), against a plant that takes 150 ms an upload. It sends the signal with the five boxes still connected and gives run 3 seconds to return; then the plant must hold all 10 shifts, every pouring of the five societies, with never more than 8 uploads at once, and the Report must count every pouring sent and 10 shifts. Tasks 1 to 4 must work first.
Farhan will ask: what happens to a task that is still running when the program's main returns?
What you will learn
- The shape of a service: connection tasks, shared state, a background worker, a channel between them.
- thiserror in a protocol: every bad line named exactly, and sent back as a reply.
- A trait at the centre: the rate charts price through PriceRule.
- Arc<RwLock<..>>: one ledger, written by every box, read by everyone.
- Forwarding as shifts close: a Semaphore and a timeout around each upload.
- Closing cleanly: what happens to tasks still running when main returns, and how to wait for them.
More in Concurrency and services
Build "Capstone: the plant's collection service" yourself
DoodhSetu runs the milk collection boxes of 120 village societies around Anand. The old box program crashed on an empty line, and you rebuild it in Rust. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Rust are free. This one comes with the all labs plan.