The Vercel AI SDK · Chapter 2 of 21

Streaming text

By the end of this chapter you can stream a model's answer with streamText: print it piece by piece as it is written, read every part of the stream (where a step starts, the text, why it finished, the tokens), run code when the answer ends with onEnd, and measure how long a customer waits for the first word. You will stream a long answer, count a stream's parts, write Vikram's cost line from a callback, and time the first word against the whole answer. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: The SDK on the server
  • 4 tasks
  • TypeScript
  • Free chapter

Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.

Why you need this

Ananya tried chapter 1's Mali on a long question, "Tell me everything about caring for a Peace Lily". She typed it, pressed Enter, and stared at an empty box. Three seconds later the whole answer landed at once, all 243 tokens of it. "Is it broken?" she asked. It was not broken. It was waiting.

generateText waits for the model to finish the whole answer before it gives you anything. A model writes one token at a time, and a long answer takes seconds. Every chat product you have used shows the words as they are written instead, so the customer starts reading at once. Mali needs the same, and generateText cannot give it: by the time it returns, the writing is over.

The lesson

How a model writes

A model does not think up the whole answer and then send it. It writes one token at a time: it reads everything so far, picks the next token, adds it, and goes again. A 243-token answer is 243 of those turns. At the lab model's pace (about one token every 15 ms, close to a real hosted model) that is a few seconds.

Model APIs can send each token the moment it is written. That is streaming: the reply comes as a series of small messages (server-sent events) over one open HTTP response, instead of one JSON body at the end. The AI SDK reads those events for you and hands you the pieces.

streamText

streamText takes the same options as generateText:

import { streamText } from 'ai';

const result = streamText({
  model: lab('lab-chat'),
  instructions: MALI_INSTRUCTIONS,
  prompt: 'Tell me everything about caring for a Peace Lily',
});

for await (const piece of result.textStream) {
  process.stdout.write(piece);
}
Here is everything about caring for a Peace Lily. Light: it likes low light. Keep it out of harsh afternoon sun, which scorches the leaves. Water: water it twice a week. ...

(Trimmed: the whole answer is nine short paragraphs of care.) Two things are different from generateText:

  • No await on the call. streamText returns a result at once and starts the request in the background. You read the answer from the result as it arrives.
  • result.textStream is the answer's text in pieces. for await waits for each piece and runs the loop body as soon as it comes. Here a piece is a token: "Here", " is", " everything".

process.stdout.write prints without a new line, so the pieces join up into sentences on the screen. With console.log(piece) every token would land on its own line.

What the customer feels

Farhan timed the same long question both ways in the lab:

generateText: whole answer after 3055 ms
streamText: first piece after 33 ms, whole answer after 4935 ms, 243 pieces

With generateText the customer sees nothing for three seconds. With streamText the first word is on the screen in 33 ms, and they read along while the rest is written. (The lab model writes a stream at a steady pace, which makes its whole streamed answer a little slower than its non-streamed one; with a real model the totals are about the same.) The number that decides how fast a chat feels is the time to first token: how long until something appears. The interview task measures it.

After the stream: the result's promises

The result also has promises that settle once the stream is over: result.text (the whole text), result.finishReason, result.usage, and more. Await them after reading the stream:

for await (const piece of result.textStream) process.stdout.write(piece);
console.log();
const usage = await result.usage;
console.log(`tokens: ${usage.inputTokens} in, ${usage.outputTokens} out`);
... Pets: keep it away from cats and dogs, who should not chew it.
tokens: 247 in, 243 out

Awaiting result.text alone also reads the whole stream, but then you print everything at the end, and you are back to the empty box.

Every part: result.stream

textStream keeps only the text. A stream carries more than text: where a step starts, the text in pieces, where it ends, why the model stopped and the tokens. result.stream gives you every part, each with a type:

const result = streamText({
  model: lab('lab-chat'),
  instructions: MALI_INSTRUCTIONS,
  prompt: 'What happens if my plant arrives broken?',
});
for await (const part of result.stream) {
  if (part.type === 'text-delta') console.log(part.type, JSON.stringify(part.text));
  else if (part.type === 'finish') console.log(part.type, part.finishReason, part.totalUsage.inputTokens, part.totalUsage.outputTokens);
  else console.log(part.type);
}
start
start-step
text-start
text-delta "If"
text-delta " a"
text-delta " plant"
...
text-delta "."
text-end
finish-step
finish stop 246 22

(Trimmed: 22 text-delta parts in all.) The parts, in order:

PartWhat it saysWhat it carries
startthe call has begun
start-stepa request to the model has begun
text-starta block of text beginsan id
text-deltaa piece of texttext
text-endthat block of text is done
finish-stepthe request is donefinishReason, usage
finishthe whole call is donefinishReason, totalUsage

A step is one request to the model. Here there is one; in chapter 4 a call makes several, and each gets its own start-step and finish-step. Later chapters add more part types: tool-call and tool-result (chapter 3), and errors. The chat route of chapter 7 sends these same parts to the browser, so this table is the shape of everything Mali shows.

Older examples online call result.stream by its old name, fullStream.

Callbacks: onChunk and onEnd

Sometimes the code that reads the stream is not yours. In chapter 7 Mali's server hands the stream straight to the browser and never loops over it. To run your own code along the way, streamText takes callbacks:

  • onChunk({ chunk }) runs for every part as it passes.
  • onEnd(...) runs once, when the answer is over, with the whole text, the finishReason, the usage and the steps.
const result = streamText({
  model: lab('lab-chat'),
  instructions: MALI_INSTRUCTIONS,
  prompt: 'What happens if my plant arrives broken?',
  onChunk({ chunk }) {
    if (chunk.type === 'text-delta') process.stdout.write('|');
  },
  onEnd({ text, finishReason, usage }) {
    console.log(`\nonEnd: ${finishReason}, ${usage.inputTokens} in, ${usage.outputTokens} out, ${text.length} characters`);
  },
});
for await (const piece of result.textStream) process.stdout.write(piece);
console.log('after the loop');
|If| a| plant| or| pot| arrives| damaged|,| send| a| photo| within| |48| hours| and| we| replace| it| for| free|.
onEnd: stop, 246 in, 22 out, 91 characters
after the loop

onChunk printed a bar before each piece, and onEnd ran once, after the last piece and before the loop ended. Vikram wants exactly this: one line in his cost log per answer, written when it ends, without slowing the stream down. Older examples call onEnd by its old name, onFinish.

Step through a stream, part by part:

When something goes wrong

streamText does not throw when the model fails: the stream has already been handed out, and a throw in the middle could crash a server. The error becomes an error part, the text comes back empty, and the SDK logs it. Give streamText an onError({ error }) callback to handle it your way (chapter 18 makes this customer-friendly):

onError: The server is overloaded or not ready yet.

That came from lab('lab-chat-down'), a lab model that is always down.

Your turn

Stream a long answer piece by piece in task 1. Read every part of a stream, and count them, in task 2. Write Vikram's cost line from onEnd in task 3. In the interview question, time the first word against the whole answer, and explain to Ananya why streaming feels so much faster.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    stream.ts: stream Mali's answer as it is written

    Call streamText (from 'ai'), not awaited, with lab('lab-chat'), instructions MALI_INSTRUCTIONS (both from ../../shared/mali.ts) and the prompt, exactly: Tell me everything about caring for a Money Plant

    Loop over result.textStream with for await and print each piece the moment it arrives, with process.stdout.write. After the loop print a new line, then the tokens from await result.usage, as:

    tokens: IN in, OUT out

    The Check runs your file at the real pace and watches when your output arrives: the answer must come out in many pieces over seconds, not all at once at the end.

  2. Task 2

    parts.ts: every part of the stream

    Export an async function readStream(question) from parts.ts. It calls streamText with lab('lab-chat'), instructions MALI_INSTRUCTIONS and the question as the prompt, then loops over result.stream (every part, not only the text). It counts each part by part.type, adds up part.text of the text-delta parts, and keeps part.finishReason from the finish part. It returns { counts, text, finishReason }.

    Under if (import.meta.main), call it with Do you charge for shipping? and print counts, the text, then finish: REASON. The Check calls readStream with a question of its own and compares the counts with the parts the stream really carried.

  3. Task 3

    cost-log.ts: a cost line when each answer ends

    Export an async function streamWithCost(question, log) from cost-log.ts. It calls streamText with lab('lab-chat'), instructions MALI_INSTRUCTIONS, the question as the prompt, and an onEnd callback that calls log once with:

    IN in, OUT out, ₹COST

    from onEnd's usage (inputTokens, outputTokens), at ₹200 a million input tokens and ₹800 a million output tokens, rupees with 4 decimals. It reads result.textStream to the end and returns the whole text.

    Under if (import.meta.main), call it with Which payment methods do you accept? and a log that prints cost: LINE, then print the answer.

  4. Task 4 · interview question

    Interview: first-token.ts, how long until the first word?

    Time a streamed long answer: streamText with lab('lab-chat'), instructions MALI_INSTRUCTIONS and the prompt, exactly: Tell me everything about caring for a Snake Plant

    Note Date.now() just before the call. Loop over result.textStream: when the first piece arrives, keep the milliseconds passed (once). After the loop, keep the milliseconds passed in all. Print:

    first text: N ms

    whole answer: N ms

    Ananya will ask: the whole answer takes seconds either way, so why does streaming feel so much faster?

What you will learn

  • streamText: the same call as generateText, with the answer arriving in pieces.
  • textStream: the text alone, piece by piece, for printing or showing.
  • stream: every part, in order: start, the text, the end of a step, the finish and its tokens.
  • The result's promises: await result.usage and await result.text after the stream.
  • onChunk and onEnd: code that runs on each part, and once when the answer ends.
  • Time to first token: the number that decides how fast a chat feels.

More in The SDK on the server

Build "Streaming text" yourself

HariBhari, a Bengaluru plant shop, wants Mali, a plant helper on its website. You build it end to end, from the first call to a chat that streams cards, asks before it adds to the cart and stays within budget. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.