AI engineering traps (TypeScript) · Chapter 9 of 16

Counting tokens with string length

Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Calling the model
  • 3 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of AI engineering traps (TypeScript) are free. This one comes with the all labs plan.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    repro.ts: break Gomti Heights' digest on purpose

    Write reproduce() in repro.ts. Ask digest from digest.ts for the digest of 'Gomti Heights', with today's messages from whatsapp.ts (gomtiHeightsToday()). Resolve to the digest, or, if digest throws, to the error's message.

    The check runs your reproduce() twice: beside the original digest.ts it must resolve to the 400 message, "400 This model's maximum context length is 8192 tokens. However, you requested 12221 tokens (11721 in the messages, 500 in the completion). Please reduce the length of the messages or completion.", and beside a fixed digest.ts to 'Digest for Gomti Heights: 309 messages. Most talked about: water (81), parking (37).' digest uses the lab model when you leave out its client; reproduce() must finish within 10 seconds.

  2. Task 2

    digest.ts: count tokens the way the model does

    Fix fitMessages in digest.ts so it measures each line the way the lab model does: a line sent as a message of its own costs its tokens in the o200k tokenizer plus 3. It keeps the newest lines, in their order, whose costs add up to at most budget (BUDGET, 6000, when left out), and no fewer than fit. js-tiktoken is installed: Tiktoken from 'js-tiktoken/lite' with 'js-tiktoken/ranks/o200k_base' is the lab model's own tokenizer.

    Keep BUDGET, digest and its log line digest: <kept> of <n> messages sent. The check uses lines and budgets of its own (Hindi, English, emoji), and a big Hindi day with the lab model. It also type-checks digest.ts: fitMessages takes the lines and an optional budget and returns string[]. Dividing the length by 2 instead of 4 is not a fix.

  3. Task 3

    digest.test.ts: a test that would have caught it

    Write a vitest test file, digest.test.ts, that would have caught this bug. The check runs your file three times: beside the original digest.ts, where at least one test must fail; beside the fixed digest.ts, where every test must pass; and beside a wrong fix that guesses 2 characters a token, where at least one test must fail.

    So, for both groups in whatsapp.ts, take the lines digest would send ('[time] name: text') and check fitMessages with the real tokenizer (tokens plus 3 a line): what it keeps costs at most BUDGET, and one more line would not fit. Each test must finish within 10 seconds.

What you will learn

  • repro.ts: break Gomti Heights' digest on purpose
  • digest.ts: count tokens the way the model does
  • digest.test.ts: a test that would have caught it

More in Calling the model

Build "Counting tokens with string length" yourself

Sunwai, a Lucknow app for housing societies, runs on a model, and Farah, its CTO, keeps assigning you tickets. Each is a trap AI engineering interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI engineering traps (TypeScript) are free. This one comes with the all labs plan.