RAG: search that answers · Chapter 1 of 37
The model does not know Paisa Setu
By the end of this chapter you can call a chat model from TypeScript, send it a whole conversation, read what it cost, and see for yourself why it cannot answer questions about Paisa Setu on its own. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Why RAG
- 4 tasks
- Free chapter
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.
Why you need this
Kabir, who runs support at Paisa Setu, gets about 600 merchant questions a day: when does my money arrive, why did this refund fail, what does U30 mean. The answers are somewhere in 145 help articles, policies and runbooks, and his team spends most of the day searching them. Arjun, the CTO, has a plan: "Put a chat model on it."
So you try it. You ask a model when UPI payments are settled, and it answers at once, politely and confidently: two working days. Paisa Setu moved to the next working day by 11 am on 1 March 2026. A merchant who trusts that answer plans their cash for the wrong day and calls back angry. This chapter shows why that happens, and why the obvious fix does not work.
The lesson
A client that works anywhere
import OpenAI from 'openai';
const ai = new OpenAI();
With no arguments, the client reads two environment variables: OPENAI_BASE_URL (where to send requests) and OPENAI_API_KEY. In the lab they point at the lab model, lab-chat, which runs inside your lab with no internet. Point them at OpenAI, or at Gemini or Claude's OpenAI-compatible endpoints, and the same code talks to those. Keys stay in the environment, never in the code.
Messages and roles
A request is a model name and a list of messages. The system message sets the rules; user messages are what the person typed:
const res = await ai.chat.completions.create({
model: 'lab-chat',
messages: [
{ role: 'system', content: 'You are a polite assistant. Reply in one sentence.' },
{ role: 'user', content: 'What does UPI stand for?' },
],
});
console.log(res.choices[0].message.content);
console.log(res.choices[0].finish_reason, res.usage);
$ node demo.ts
UPI stands for Unified Payments Interface, the real-time payment system in India run by NPCI (the National Payments Corporation of India).
stop { prompt_tokens: 27, completion_tokens: 27, total_tokens: 54 }
The reply is in choices[0].message. finish_reason is stop when the model finished its answer (length means it ran out of room). usage counts tokens, the pieces of text models read and write, and providers charge per token. Run lab requests in the terminal to see exactly what the model received.
The model remembers nothing
Each request stands alone. Tell the model something in one request and ask about it in the next:
$ node demo.ts
Thanks for letting me know. How can I help you today?
I don't know that. You have not told me in this conversation.
A chat app "remembers" because it sends the whole conversation every time, the model's own replies included, as assistant messages. Press Next to follow what the model sees in each request:
The third request, in code:
messages: [
{ role: 'user', content: 'My shop is Sharma Sweets in Indore.' },
{ role: 'assistant', content: 'Nice to meet you! How can I help?' },
{ role: 'user', content: 'Which city is my shop in?' },
],
$ node demo.ts
Your shop is Sharma Sweets in Indore.
Confident and wrong
Now ask about Paisa Setu:
$ node demo.ts
Settlements are usually made on a T+2 basis, two working days after the transaction, directly to your registered bank account.
A model learns from text that existed before it was trained. Paisa Setu's policies were never in it, so it answers with what payment companies usually do, in the same confident voice as when it is right. Nothing in the reply tells you it guessed.
Too much to paste
The obvious fix: send the documents with the question. All 96 help articles together:
$ node demo.ts
400 context_length_exceeded
400 This model's maximum context length is 8192 tokens. However, your messages resulted in 52023 tokens. Please reduce the length of the messages.
Every model has a context window, the most tokens one request can hold. Bigger models have bigger windows, but you pay for every token on every question, and the documents grow every week. The real fix is to send only the few passages that answer this question. Finding them is what the rest of this lab builds.
A mistake to avoid when you handle that error: catching every error. A wrong model name is an error too, and it must not look like "too long":
$ node demo.ts
true 404
404 The model `gpt-lab` does not exist or you do not have access to it.
Check the error's code before you treat it as one you expected.
Your turn
Your four tasks are at the top, all in exercise.ts, each with its own comment. The last is Arjun's interview question: why not paste every document into every prompt? Run lab requests after each one to see what you sent.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
ask: ask the model a question
Send the model two messages with ai.chat.completions.create: SYSTEM as the system message, then the question as a user message, with the model MODEL (lab-chat). Return the text of the reply, the content of the first choice's message.
await ask('What does UPI stand for?') is the model's one-sentence answer about the Unified Payments Interface. The content's type allows null: return '' then.
- Task 2
askWithUsage: the answer, and what it cost
Make the same request as ask, and return an Answer: answer is the reply's text, tokensIn is the response's usage.prompt_tokens, tokensOut is usage.completion_tokens, and finishReason is the first choice's finish_reason.
For a short question the model finishes its answer, so finishReason is 'stop'. Return the numbers exactly as the response reports them.
- Task 3
chatTurn: a conversation the model can follow
The model remembers nothing between requests, so send SYSTEM, then every message of history in order, then the question as a user message.
Return a new list: the history, then { role: 'user', content: question }, then { role: 'assistant', content: the reply }. Do not change the history you were given (no push): the app keeps the old list too.
With a history where the merchant said their id is M-1042, chatTurn(history, 'What is my merchant id?') ends with the model's reply containing M-1042.
- Task 4 · interview question
Interview: askWithDocs, paste in the documents
Put every document, word for word, in the system message after SYSTEM (join them with blank lines), send the question as a user message, and return { ok: true, answer } when the request works.
All of Paisa Setu's documents are far more than the model's 8,192-token context window, so the request fails with an OpenAI.APIError whose code is 'context_length_exceeded'. Return { ok: false, reason: 'too-long' } for that error only. Any other error must still be thrown: with the model 'lab-chatt', which does not exist, the call must reject with status 404. Use the model parameter, not MODEL, in the request.
Arjun will ask why you cannot paste every document into every prompt.
What you will learn
- The chat API: a client, a model and a list of messages with roles (system, user, assistant), through the official openai package.
- The reply: the text, why the model stopped, and the tokens it used.
- Conversations: the model remembers nothing, so you send the history.
- Context limits: why you cannot paste every document into the prompt.
More in Why RAG
Build The model does not know Paisa Setu yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.