The Vercel AI SDK · Chapter 19 of 21

Tokens and cost

By the end of this chapter you can say what every one of Mali's answers cost, in tokens and in rupees, and stop a chat that would cost too much. You will read the usage of a multi-step answer, price tokens with a model's prices (cached ones too), send each answer's cost with it, and give a chat a budget. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Ready for customers
  • 4 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of The Vercel AI SDK are free. This one comes with the all labs plan.

Why you need this

Ananya wants Mali on every page of the website by Diwali. Vikram wants a number first: "Six hundred chats a day. What does one cost? And is there a chat that could cost us ₹50 by itself?"

Chapter 1 priced a single call: input tokens times one price, output tokens times another. Mali is not a single call any more. One question can take three requests (search, add to cart, answer), and a long chat sends its whole history again every turn. Which tokens is HariBhari really paying for?

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    usage.ts: one step, or all of them?

    Ask with generateText: lab('lab-chat'), MALI_INSTRUCTIONS, tools: maliTools(), stopWhen: isStepCount(5), and the prompt Find a pet-safe plant under ₹500 and add it to my cart.

    Print one line per step from result.steps' usage, step N: IN in, OUT out; then final step: IN in, OUT out from result.finalStep.usage; then all steps: IN in, OUT out from result.usage.

  2. Task 2

    cost.ts: tokens into rupees

    Export PRICES (lab-chat: input 200, cachedInput 50, output 800; lab-chat-mini: input 15, cachedInput 4, output 60, rupees a million tokens) and rupees(usage, modelId): cached input tokens (usage.inputTokenDetails.cacheReadTokens, maybe undefined) at the cachedInput price, the other input tokens at the input price, output tokens at the output price, over a million, not rounded. A model with no price throws an Error.

  3. Task 3

    server.ts: the cost of each answer, sent with it

    Export POST, the chat route of chapter 9 (lab('lab-chat'), MALI_INSTRUCTIONS, the converted UI messages, maliTools(), isStepCount(5), abortSignal: req.signal), with a messageMetadata function on toUIMessageStream: for the finish part, return { model: 'lab-chat', inputTokens, cachedTokens, outputTokens, rupees } from part.totalUsage (cachedTokens from inputTokenDetails.cacheReadTokens ?? 0, rupees from your ./cost.ts); for other parts, undefined.

  4. Task 4 · interview question

    Interview: budget.ts, a budget for a chat

    Export overBudget(maxRupees): a stop condition, ({ steps }) => boolean, true once the steps so far cost maxRupees or more, adding up rupees(step.usage, step.model.modelId) from ./cost.ts.

    The Check runs a three-step chat with stopWhen: [isStepCount(5), overBudget(max)] for budgets of ₹0.15, ₹0.40 and ₹1 and compares the steps it ran with what each step really cost.

    Vikram will ask: the budget stopped the chat after the plant went into the cart, before Mali said so. Is a stop condition the right place for a budget?

What you will learn

  • usage across steps: result.usage (all steps) against result.finalStep.usage (the last).
  • Rupees from tokens: a model's prices, and a function that applies them.
  • Cached input tokens: why a long chat's next turn can be cheaper, and how to count it.
  • Cost in message metadata: each answer carries its tokens and rupees.
  • A budget: a stop condition that ends a chat before it costs too much.

More in Ready for customers

Build "Tokens and cost" yourself

HariBhari, a Bengaluru plant shop, wants Mali, a plant helper on its website. You build it end to end, from the first call to a chat that streams cards, asks before it adds to the cart and stays within budget. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of The Vercel AI SDK are free. This one comes with the all labs plan.