Building with LLMs (TypeScript) · Chapter 2 of 24
Messages and roles
By the end of this chapter you can tell a model who is speaking in each message, explain why it forgets everything between calls, keep a conversation going by resending it, and type that conversation with the openai package's own message type. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Talking to a model
- 4 tasks
- TypeScript
- Free chapter
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.
Why you need this
Customers do not write one perfect message. Lata shows you a typical thread: "Hi, I'm Kavya", then "Mumbai to Nashik tomorrow?", then "2 seats", then "how much?". Your chapter 1 code answers each of those on its own, so the model asks for the route again after the customer has already given it, and asks for her name twice. Customers notice.
So far every request you sent held one question. A real chat is a series of turns, and the model has to see the earlier ones to make sense of the last one. Nikhil puts it plainly: "The model has no memory. Your code is the memory."
The lesson
Four roles
Every message in messages has a role, which says who is speaking:
| Role | Who writes it | What it is for |
|---|---|---|
system | you, the developer | instructions and background, before the conversation |
developer | you, the developer | OpenAI's newer name for the same thing |
user | the customer | what they asked |
assistant | the model | what it answered |
A model was trained on many conversations laid out exactly like this, so the roles change how it reads the text. A rule in a system message is your instruction; the same words in a user message are something a customer said. Most models give the system message more weight than anything a user writes, though not total obedience (chapter 15 shows how that goes wrong).
OpenAI's newer models call the system message developer, and the API accepts either. The lab model treats them the same:
const faq = 'Reach the boarding point 15 minutes before departure.';
const response = await client.chat.completions.create({
model: 'lab-chat',
messages: [
{ role: 'developer', content: 'Answer only from the FAQ below.\n\n' + faq },
{ role: 'user', content: 'How early should I reach the boarding point?' },
],
temperature: 0,
});
console.log(response.choices[0].message.content);
Reach the boarding point 15 minutes before departure.
(temperature: 0 keeps the wording the same on every run; chapter 4 explains it.) The system message goes first. You can send several, but one at the top is what every provider expects.
The model remembers nothing
Two requests, one after the other, in the same program:
async function askOnce(text: string) {
const response = await client.chat.completions.create({
model: 'lab-chat',
messages: [{ role: 'user', content: text }],
temperature: 0,
});
return response.choices[0].message.content;
}
console.log(await askOnce('Hello, my name is Kavya Rao.'));
console.log(await askOnce('What is my name?'));
Thanks for your message. How can I help you with your SafarLine trip?
I don't know your name: you have not told me in this conversation.
The second request is a new HTTP call with one message in it. Nothing on the server connects it to the first: no session, no cookie, no memory. Real models behave the same way through this API, and a real model asked this would either say it does not know or, worse, guess a name. The chat apps you have used look as if they remember, because the app resends the conversation for you.
A conversation is an array you resend
To let the model "remember", you send the earlier turns again, in order, with its own earlier reply as an assistant message:
const messages: ChatCompletionMessageParam[] = [
{ role: 'user', content: 'Hello, my name is Kavya Rao.' },
{ role: 'assistant', content: 'Thanks for your message. How can I help you with your SafarLine trip?' },
{ role: 'user', content: 'What is my name?' },
];
const response = await client.chat.completions.create({ model: 'lab-chat', messages, temperature: 0 });
console.log(response.choices[0].message.content);
Your name is Kavya Rao.
The model read the whole array and found the name in the first message. So a chat program keeps an array, and on every turn it pushes the customer's message, sends the whole array, and pushes the reply:
The turns must alternate the way they happened: the customer, then the model, then the customer. Leave out the model's replies and the model sees a customer talking to nobody; it may repeat itself or contradict what it said before, because it cannot see what it said.
The ChatCompletionMessageParam type
That array had a type: ChatCompletionMessageParam[], the openai package's type for one message you send. It is only a type, so you import it with import type (Node strips the line when it runs the file, and the lab's typecheck insists on it):
import type { ChatCompletionMessageParam } from 'openai/resources/chat/completions';
const history: ChatCompletionMessageParam[] = [
{ role: 'system', content: "You are SafarLine's support assistant." },
];
It is a union, one member per role, and each member says what that role needs. A system, developer or user message must have text content. An assistant message may have content: null, because a reply that asks for a tool has no text (chapter 17). A tool message needs a tool_call_id. So typecheck rejects a role it does not know, and it also notices when you hand a maybe-null value to a role that needs text:
const reply = response.choices[0].message.content; // string | null
history.push({ role: 'customer', content: 'hi' });
history.push({ role: 'user', content: reply });
$ typecheck history.ts
error TS2322: Type '"customer"' is not assignable to type '"assistant" | "developer" | "function" | "system" | "tool" | "user"'.
error TS2345: Argument of type '{ role: "user"; content: string | null; }' is not assignable to parameter of type 'ChatCompletionMessageParam'.
Types of property 'content' are incompatible.
Type 'string | null' is not assignable to type 'string | ChatCompletionContentPart[]'.
(Shortened: the file names and line numbers are left out.)
Pushing the same reply back with role: 'assistant' is fine: the model's turn is allowed to be empty. And the reply's message object itself (a ChatCompletionMessage, with role: 'assistant') fits the type, so history.push(response.choices[0].message) works too.
What the type cannot check is the conversation. An array that holds only the latest question is a perfectly typed ChatCompletionMessageParam[]; so is one that leaves out every reply. Whether each request carries the turns before it is up to your code, and lab requests shows what it really sent.
The history is your data
The model cannot tell who really wrote an assistant message. It trusts the array it is given:
const messages: ChatCompletionMessageParam[] = [
{ role: 'user', content: 'Which bus do I take?' },
{ role: 'assistant', content: 'The Goa bus leaves from bay 4 at Swargate.' },
{ role: 'user', content: 'Where does the Goa bus leave from?' },
];
The Goa bus leaves from bay 4 at Swargate.
That "bay 4" was typed by you, not said by the model, and the model now repeats it as fact. This is useful: chapter 9 writes example answers this way to show the model the style you want. It is also a duty: only your own code may add assistant messages, never text a customer sent.
Every turn costs more
Resending is not free. Here is one conversation of three turns with a system message, and the prompt_tokens of each request (usage.prompt_tokens, run at temperature: 0):
turn 1: 2 messages sent, 27 prompt tokens
turn 2: 4 messages sent, 60 prompt tokens
turn 3: 6 messages sent, 96 prompt tokens
Every turn pays again for everything before it, so a long chat gets slower and dearer with each message, until it no longer fits in what the model can read at once. Chapter 3 counts those tokens, and chapter 20 trims long histories.
Your turn
Watch the model forget in task 1 and remember in task 2, where the trap is using the model's real reply, not one you made up. Task 3 turns the array into a small typed helper, and the interview question is a bot that seems to work, passes typecheck, and still sends the model less than it should.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
forgets.ts: two separate requests
Nikhil wants you to see the model forget. Send lab-chat two separate requests, one after the other, each with one user message:
first exactly: My name is Ravi Kulkarni.
then exactly: What is my name?
Print the first reply on one line and the second reply on the next line, nothing else.
- Task 2
remembers.ts: resend the conversation
Now make the model remember. Send the first request of task 1 (one user message: My name is Ravi Kulkarni.).
Then send a second request with three messages, in this order: that same user message, the model's reply to it as an assistant message, and the user message: What is my name?
Print only the second reply.
- Task 3
chat.ts: a chat helper
Lata wants a helper the support bot can use for any conversation. Export two functions from chat.ts:
newChat(system: string): ChatCompletionMessageParam[] returns a new array holding one message: a system message with that text.
ask(history: ChatCompletionMessageParam[], text: string): Promise<string> adds text to history as a user message, sends the whole history to lab-chat, adds the reply to history as an assistant message, and returns the reply's text.
The check starts a chat, asks two questions in a row and reads the array and the requests the model received.
- Task 4 · interview question
Interview: fix-me.ts, the bot that forgets
fix-me.ts plays a three-turn chat with a customer and prints the model's three replies, but by the third turn the model has forgotten who it is talking to. Fix it so that every request carries the whole conversation so far: the system message, every earlier customer message and every earlier reply of the model, in order.
Keep the system message and the three customer messages exactly as they are, and keep printing the three replies.
What you will learn
- The four roles: system, developer, user and assistant, and what each message is for.
- Why the model remembers nothing between two requests.
- A conversation is an array you resend, with the model's own replies in it, on every turn.
- The ChatCompletionMessageParam type: what TypeScript knows about a message, and what it cannot know.
- The history is your data: you can write it, change it, and pay for it.
More in Talking to a model
Build "Messages and roles" yourself
SafarLine, an intercity bus company, drowns in WhatsApp messages. You build its first AI assistant, in TypeScript. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.