AI support copilot project (TypeScript) · Chapter 12 of 12
Cost, speed and the take-home
By the end of this chapter you can count a request's tokens before you send it and price it in rupees after, keep embeddings in a Postgres cache keyed by what they were made from, give each conversation a token budget, and add to a counter in a way two requests at once cannot break. Then the whole copilot is yours, to run on your own machine with a real model. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Ready for members
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of AI support copilot project (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Sayali asks the question every AI feature meets in the end: what does an answer cost? Today nobody can say. The model reports tokens and the app passes them on, but nothing turns them into rupees. One answer to "How much is the booking fee?" read 398 tokens and wrote 40.
And the copilot pays for the same work again and again. "How much is the booking fee?" is one of Datta's most common questions, and every time it is embedded, searched, reranked and answered from scratch. POST /ingest embeds all 179 chunks again when Asha fixes one comma. A member who keeps a conversation going has no limit at all: every turn costs another answer. Diwali week will double all of it.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
tokens.ts: tokens before, rupees after
In src/cost/tokens.ts write messageTokens(messages): 3 for every message plus its content's tokens (countTokens), plus 3 for the start of the reply, the way OpenAI counts a chat request; messageTokens([]) is 3. Then costInr(tokens, priceInput, priceOutput): tokens is { input, output }, the prices are rupees per million tokens; return (input × priceInput + output × priceOutput) / 1,000,000 rounded to 4 places, as a number.
The Check counts a system and a user message and wants the same number the lab model reports as prompt_tokens, then counts an empty list, then prices four usages, among them 398 in and 40 out at ₹200 and ₹800 a million (0.1116).
- Task 2
cache.ts: embed a text once, ever
In src/cost/cache.ts write cacheKey(text): the sha256 hex of config.embedModel, a new line and the text. Then cachedEmbed(ai, texts): one embedding per text, in order. Look every key up in embedding_cache in one query; send the texts not found (each once, in order) to the model in one embedTexts call, only when there are any; store the new ones (key, embedding) with .onConflictDoNothing(); return the vectors in the texts' order. No texts: [] with no request.
The Check calls it four times and counts the texts sent to the model: [a, b, a] sends 2, [a, b, c] sends 1, [c, a] and [] send nothing; every vector must be its text's, and all three must be in embedding_cache. Then it stores a new text in a transaction of its own and calls cachedEmbed with the same text before committing (the insert must not fail), and sends POST /ask twice (your cachedEmbed embeds the question; Task 1's costInr prices the answer).
- Task 3 · interview question
Interview: budget.ts, a token budget per conversation
In src/cost/budget.ts write charge(q, conversationId, tokens): one UPDATE of conversations, written with q (the db or tx it is given), whose new tokensUsed Postgres computes from the row (sql
${conversations.tokensUsed} + ${tokens}), with .returning() to read the new total; return it. Then checkBudget(conversationId, budget): read tokensUsed, and when it is at or over the budget throw new AppError(429, 'budget_exceeded',This conversation has used its ${budget} tokens: start a new one); a conversation that does not exist is not checked.The Check charges 250 inside a transaction it rolls back (the total must be 350, and nothing may stay), then charges 50 while a transaction of its own has added 100 and holds the row (the total must be 150), then checks a conversation at 2999 and at 3000 tokens, one that does not exist, and POST /conversations/:id/ask over the budget (429).
Nikhil will ask: charge could read tokensUsed, add in TypeScript and save the total. Why must the adding happen inside the UPDATE?
What you will learn
- Tokens before you send: counting a chat request's tokens exactly as the provider does.
- Rupees after: prices per million tokens, and why four places.
- Caches keyed by content: a hash of what a result was made from, model name included.
- On conflict do nothing: storing a cache entry that another request may have stored a moment ago.
- A token budget: a limit per conversation, refused with a 429.
- A lost update: what read, add and write does when two requests meet, and the one UPDATE that cannot lose.
More in Ready for members
Build "Cost, speed and the take-home" yourself
Parda answers the same audience questions all day. You build the copilot that answers them from its own help pages, with sources, and says so when it does not know. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI support copilot project (TypeScript) are free. This one comes with the all labs plan.