AI engineering traps · Chapter 14 of 16
Streaming bytes cut in half
Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Tools
- 3 tasks
- Python
- In the all labs plan
The first 2 chapters of AI engineering traps are free. This one comes with the all labs plan.
Why you need this
The network decides where one piece of the stream ends and the next begins. On a fast connection, each event of the model's reply tends to arrive whole, and the bug never shows. On a slow one, pieces are small and cut anywhere. An English letter cannot be cut, it is one byte; a Hindi one can. So the bug hides in every test written in English on a laptop.
errors="ignore" does not help either: it drops the bytes of a cut letter instead of marking them, so पानी comes out as पनी, a letter missing and no sign that anything went wrong.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
repro.py: stream the Hindi notice over a slow network
Write reproduce() in repro.py. Ask stream_notice from net.py for the ticket's notice, with these four lines as the facts, one string, a line each: सोसाइटी: गोमती हाइट्स, then - शनिवार 8 नवंबर को सुबह 10 बजे से शाम 4 बजे तक पानी की आपूर्ति बंद रहेगी।, then - कृपया शुक्रवार रात को पानी भर कर रखें।, then - लिफ्ट B सुबह 11 बजे से दोपहर 1 बजे तक बंद रहेगी। Pass the stream through small_chunks(stream, 5), then read it with read_notice from stream.py (with asyncio.run), and return the notice's last line.
The check runs your reproduce() twice: beside the original stream.py the line must have the replacement character � in it (the broken sign-off in the ticket), and beside a fixed stream.py it must be 'प्रबंध समिति, गोमती हाइट्स'. reproduce() must finish within 10 seconds.
- Task 2
stream.py: decode the stream, not each chunk
Fix read_notice in stream.py. Turn the bytes into text with one incremental decoder for the whole stream (codecs.getincrementaldecoder("utf-8")()), decoding each chunk with it, so a character cut between two chunks is kept until the rest of it arrives, and finish with decoder.decode(b"", final=True). The notice must come out exactly as the model wrote it, however the bytes are cut.
Keep the line buffer, the events and the printed line stream: <n> characters as they are. The check streams its own notices, in Hindi and in English, in pieces from 1 to 64 bytes, and compares every character. Dropping the broken bytes with errors="ignore" is not a fix. net.py is not to be changed.
- Task 3
test_stream.py: a test that would have caught it
Write a pytest file, test_stream.py, that would have caught the broken Hindi notices. The check runs your file three times: beside the original stream.py, where at least one test must fail; beside the fixed stream.py, where every test must pass; and beside a wrong fix that drops the broken bytes (errors="ignore"), where at least one test must fail.
So stream a Hindi notice of your own through small_chunks with small pieces and assert its last line is the sign-off word for word: प्रबंध समिति, then your society's name in Hindi. Use plain def tests that call asyncio.run. Each test must finish within 10 seconds.
What you will learn
- repro.py: stream the Hindi notice over a slow network
- stream.py: decode the stream, not each chunk
- test_stream.py: a test that would have caught it
More in Tools
Build "Streaming bytes cut in half" yourself
Sunwai, a Lucknow app for housing societies, runs on a model, and Farah, its CTO, keeps assigning you tickets. Each is a trap AI engineering interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI engineering traps are free. This one comes with the all labs plan.