Build a voice agent · Chapter 14 of 21

Hindi, English and Hinglish

By the end of this chapter Bolo hears which language each caller speaks, turn by turn, and answers in a voice that caller can follow: Hinglish to Shabnam, English to Mr Fernandes, with the numbers in the right language. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Real time
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Build a voice agent are free. This one comes with the all labs plan.

Why you need this

Rashmi's first rule for Bolo, back in chapter 1: every call answered in the caller's language. Kickstart's callers are mostly Hinglish speakers, who mix Hindi and English in every sentence ("achha, toh saturday ko shaam ko aa sakte ho?"). Some speak only English: Mr Fernandes, who has just moved to Nagpur, and Farhan, who books from his office.

The model already does half of this: it answers in English when the caller speaks English. Here is Mr Fernandes anyway, with Bolo as it is:

  5.72-  9.36  caller  hi, i have a royal enfield classic. what does a general service cost?
 10.44- 16.40  Bolo    a general service for a bike costs ek hazaar ek sau ninyaanve rupaye that includes an o...  (1084 ms after the caller)
 16.86- 19.26  caller  sorry, how much was that? in english please.
 20.28- 25.56  Bolo    sorry i didnt catch that could you say it again  (1019 ms after the caller)
 ...
 28.66- 32.36  Bolo    most services are done the same day and your bike is back by saat p m  (1213 ms after the caller)
outcome: caller_hung_up, not a success
  ✖ heard the price in English (one thousand one hundred ninety nine)
  ✖ heard when it comes back (seven)

The model wrote English: "A general service for a bike costs ₹1,199." He heard "ek hazaar ek sau ninyaanve rupaye", asked for English, and still never got his price. The words were right. The voice was wrong: Bolo speaks every reply with the Hindi voice, and a voice reads numbers in its own language.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    detect.py: which language is this?

    Write language_of(words: list[str]) -> str | None. Count the words that are in HINDI and the words that are in ENGLISH (from words.py); every other word (names, brands, and words both languages use, like "bike", "service", "ok") counts for neither. With no counted word it gives None; it gives "en" when the English words are at least 80% of the counted ones, and "hi" otherwise.

  2. Task 2

    call_language.py: Bolo's voice follows the caller

    Write class CallLanguage: CallLanguage(start="hi"); heard(words: list[str]) switches to the language of a turn's words and returns it ("hi" or "en"), keeping the language the call already had when language_of gives None; the property voice gives "en-IN" for English and "hi-IN" for Hindi.

    The given agent.py calls heard() on every turn and speaks every reply and fixed phrase in that voice. Mr Fernandes (fernandes-english), Farhan (farhan-status) and Shabnam (shabnam-sunday) must all succeed, and no English caller may ask for English again.

  3. Task 3 · interview question

    Interview: two_voices.py, the same text in two voices

    Send "A general service for a bike costs ₹1,199." and then "Bike ki general service ₹1,199 ki hai." to the text-to-speech service (POST VOICE/v1/speak with JSON {"text", "language", "format": "json"}), each with language "hi-IN" and then "en-IN". For each, print LANGUAGE: and the answer's words (the "word" of each entry of its "words" list, joined with spaces): four lines in all.

What you will learn

  • How a voice reads text: words as written, numbers and symbols in the voice's language.
  • Hinglish: why a few English words do not make a caller English.
  • Detecting the language of a turn, the way the model does.
  • Following the caller: the language decided again on every turn, kept when a turn says nothing.

More in Real time

Build "Hindi, English and Hinglish" yourself

Kickstart services scooters at the door in Nagpur, and its phone line gets 600 calls a day in Hindi, English and Hinglish. You build Bolo, its voice agent, and measure every call: who it understood, how long callers waited and who it cut off. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Build a voice agent are free. This one comes with the all labs plan.