Build a voice agent · Chapter 21 of 21

Interview round

By the end of this chapter you have answered six timed questions, one idea each, from across the whole lab: a mu-law byte, a VAD that waits for the end of a sentence, a sentence splitter's edges, the "as I said" bug, p95 from samples, and the chipmunk. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 6 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Build a voice agent are free. This one comes with the all labs plan.

Why you need this

Bolo is on Kickstart's line, and Rashmi wants a second voice engineer so Imran can take a holiday. Imran runs the interview the way he reviews code: "Short problems, ten minutes each. I do not care whether you remember an API. I care whether you know the edge that breaks a voice agent on a real call, and whether you measure instead of guessing." Kavita adds the question about commas (her callers pause in the middle of every sentence), Tejas the one about p95 (his dashboard), and the last one is Imran's favourite: the agent that works in every test and hears nonsense on the phone.

Each question uses one idea from an earlier chapter, with no hand-holding: the file has the goal, the check has the edge cases, and several end with real audio: a recording from the line, a live call, a live stream. The ideas are below, as a refresher and as the things an interviewer listens for. The answers are not.

What you build: 6 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1 · interview question

    Interview: mulaw_byte.py, a mu-law byte

    Time yourself: 10 minutes. Write decode(byte) (G.711 mu-law to a 16-bit sample: flip the bits; sign, exponent, mantissa; ((mantissa << 3) + 0x84) << exponent, minus 0x84) and encode(sample) (the other way, sizes capped at 32635). The check tries all 256 bytes and samples across the range, then decodes Mr Pandey's mu-law recording with your decode and listens to it at 8 kHz.

  2. Task 2 · interview question

    Interview: vad.py, a VAD that waits

    Time yourself: 10 minutes. Write segments(frames, threshold=0.02, min_ms=80, hangover_ms=300): speech segments of 20 ms frames, as Segment(start, end) in seconds: a loud frame (RMS over threshold) starts one, hangover_ms of quiet ends it at the end of its last loud frame, and one with less than min_ms of loud frames is dropped. The check makes two sentences with commas and a door bang between them, then reads Mr Gupta's line with its 1.1 s pause.

  3. Task 3 · interview question

    Interview: splitter.py, a splitter edge

    Time yourself: 10 minutes. Write class Sentences: push(delta) returns the sentences now complete (. ? ! or । with white space after; "4.5" and Dr., Mr., Mrs., Rs., No. end none), stripped; rest() returns what is left. The check streams replies cut into pieces anywhere: a full stop in its own piece, a decimal split at the dot, a title, a booking number, a Hindi full stop.

  4. Task 4 · interview question

    Interview: as_i_said.py, the "as I said" bug

    Time yourself: 10 minutes. Write heard_of(words, played_to): the words of a cut reply that ended by played_to (0.05 s to spare), joined, with ... when some were cut, "" when none played. The check tries cases it makes up, then runs Ritesh's call with your heard_of in the given agent: the history must hold what he heard, and he must hear his price.

  5. Task 5 · interview question

    Interview: p95.py, p95 from samples

    Time yourself: 10 minutes. Write percentile(xs, p) (nearest rank: sort a copy, the value at rank ceil(p / 100 x n), math.nan for none) and suite_p95(recs) (every turn of every call in one list). The check tries twenty real turns and compares with the line's own p95 for Ritesh's call.

  6. Task 6 · interview question

    Interview: chipmunk.py, the chipmunk

    Time yourself: 10 minutes. Write stt_query(media_format): the speech-to-text's query string for a line's media_format: its sample_rate, and encoding mulaw for audio/x-mulaw or linear16 for audio/l16; raise ValueError for anything else. The check streams a mu-law line and a PCM line to the speech-to-text with your settings and reads what it heard.

What you will learn

  • How to work a timed question: say the plan, name the edge, build the smallest thing that runs, then try it on real audio.
  • The six ideas again, at interview depth: mu-law (chapter 3), voice activity detection (6), splitting a streamed reply (8), the history after a barge-in (11), percentiles (4 and 18), and sample rates (3).

More in Production

Build "Interview round" yourself

Kickstart services scooters at the door in Nagpur, and its phone line gets 600 calls a day in Hindi, English and Hinglish. You build Bolo, its voice agent, and measure every call: who it understood, how long callers waited and who it cut off. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Build a voice agent are free. This one comes with the all labs plan.