Build a voice agent (TypeScript) · Chapter 3 of 21
Phone audio: 8 kHz and mu-law
By the end of this chapter you can say why phone calls carry 8,000 samples a second, encode and decode G.711 mu-law by hand, change a sample rate without changing the sound, take a call on a phone network's line in both directions, and recognise the "wrong sample rate" bug from what the recogniser hears. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Sound and the pipeline
- 4 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of Build a voice agent (TypeScript) are free. This one comes with the all labs plan.
Why you need this
In chapters 1 and 2, Kickstart's line sent Bolo clean 16-bit audio at 16,000 samples a second. That is the lab's friendly default. Kickstart's real number is different: a customer dials from a mobile phone, the call crosses the phone network, and a telephony provider hands it to Bolo over a WebSocket. What arrives there is 8,000 samples a second, and often 8 bits a sample in mu-law, not 16.
Imran has seen the same week happen at three companies: "The demo worked on the laptop. On the first real call the bot heard nothing at all. Somebody had told the recogniser the audio was 16,000 a second when the phone network sent 8,000. Every voice sounded like a chipmunk, twice as high and twice as fast, and the recogniser gave up."
Bolo's code so far assumes one format. In this chapter it learns the phone network's: why it is 8 kHz, what mu-law does to a sample, how to change the rate, and what each mistake sounds like to a recogniser.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
mulaw.ts: mu-law, the phone network's 8 bits
Write and export mulawEncode(pcm) giving a Buffer of one byte a sample, and mulawDecode(bytes) giving an Int16Array, exactly as G.711 does them (bias 132, clip 32635; the steps are in the starter).
Under if (import.meta.main), print pandey-price-1-mulaw.wav: N bytes, S s at 8000 Hz; your mu-law of pandey-price-1-8k.wav: M of N bytes match; round trip: largest error E, on a sample of S (the first sample where |decoded - original| is largest); and the STT heard: TEXT for the file's bytes decoded and sent as 16-bit PCM at 8000 Hz.
- Task 2
resample.ts: from 8,000 to 16,000 samples a second
Write and export resample(pcm, from, to): the same samples when the rates are equal; otherwise floor(length x to / from) samples, where sample i is the input at x = i x from / to, mixed linearly between samples floor(x) and the next (the last sample when there is no next), rounded.
Under if (import.meta.main), resample ../../calls/pandey-price-1-8k.wav to 16000 and print pandey-price-1-8k.wav: N samples at 8000 Hz, at 16000 Hz: N samples, S s, first 6: and your first six samples, and the STT heard: TEXT (sent with sample_rate=16000).
- Task 3
mulaw-call.ts: a call on a phone network's line
Turn the greeting into 16 kHz speech (POST VOICE + /v1/speak), then open WS + /v1/line?caller=shabnam-sunday&codec=mulaw. On start, send the greeting as one media message (resampled to 8000, mu-law, base64) and a mark. Once the mark comes back, keep 300 media payloads and hang up. Print call ID: ENCODING at RATE Hz, B bytes a frame; caller (decoded and resampled): TEXT (each frame decoded, resampled to 16000, joined, sent with sample_rate=16000); caller (mu-law to the STT): TEXT (the bytes as they came, with sample_rate=8000&encoding=mulaw).
- Task 4 · interview question
Interview: chipmunk.ts, the wrong format
POST raw bytes (no WAV header) of Mr Pandey's line to VOICE + /v1/listen four ways and print, for each, NAME: D s, heard "TEXT" (D the answer's duration, 2 decimals): 8 kHz sent as 16000 (the 8 kHz samples with sample_rate=16000), 16 kHz sent as 8000, mu-law sent as 16-bit PCM (the mu-law bytes with sample_rate=8000), and 8 kHz sent as 8000.
What you will learn
- The phone band: why 8,000 samples a second is enough for speech, and what it loses.
- G.711 mu-law: 8 bits a sample, with fine steps for quiet sounds and coarse ones for loud sounds.
- Resampling between 8,000 and 16,000, and why going down needs care.
- A call on a mu-law line: decode what comes in, encode what goes out.
- Codecs at a glance: G.711, AMR, Opus, MP3, and why a live call avoids some of them.
- The chipmunk bug, and its cousins, measured.
More in Sound and the pipeline
Build "Phone audio: 8 kHz and mu-law" yourself
Kickstart services scooters at the door in Nagpur, and its phone line gets 600 calls a day in Hindi, English and Hinglish. You build Bolo, its voice agent, and measure every call: who it understood, how long callers waited and who it cut off. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Build a voice agent (TypeScript) are free. This one comes with the all labs plan.