RAG in Python · Chapter 1 of 37
The model does not know Paisa Setu
By the end of this chapter you can call a chat model from Python, send it a whole conversation, read what each answer cost, and show with a real error why a model cannot answer questions about Paisa Setu on its own. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Why RAG
- 4 tasks
- Python
- Free chapter
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.
Why you need this
Kabir, who runs support at Paisa Setu, gets about 600 merchant questions a day: when does my money arrive, why did this refund fail, what does U30 mean. The answers are somewhere in 145 help articles, policies and runbooks, and his team spends most of the day searching them. Arjun, the CTO, has a plan: "Put a chat model on it."
So you try it. You ask a model when UPI payments are settled, and it answers at once, politely and confidently: two working days. Paisa Setu moved to the next working day by 11 am on 1 March 2026. A merchant who trusts that answer plans their cash for the wrong day and calls back angry. Everything you know from Python so far works on data you can see. A model works on text it was trained on, and Paisa Setu's documents were never part of it. This chapter shows why that matters, and why the obvious fix fails.
The lesson
A client that works anywhere
from openai import OpenAI
client = OpenAI()
With no arguments, the client reads two environment variables: OPENAI_BASE_URL (where to send requests) and OPENAI_API_KEY. In the lab they point at the lab model, lab-chat, which runs inside your lab with no internet. Point them at OpenAI, or at Gemini's or Claude's OpenAI-compatible endpoints, with your own key, and the same code talks to those. Keys live in the environment, never in the code.
Messages and roles
A request is a model name and a list of messages, each a dictionary with a role and its content. The system message sets the rules; user messages are what the person typed:
response = client.chat.completions.create(
model="lab-chat",
messages=[
{"role": "system", "content": "You are a polite assistant. Reply in one sentence."},
{"role": "user", "content": "What does UPI stand for?"},
],
)
print(response.choices[0].message.content)
print(response.choices[0].finish_reason)
print(response.usage)
$ python3 demo.py
UPI stands for Unified Payments Interface, the real-time payment system in India run by NPCI (the National Payments Corporation of India).
stop
CompletionUsage(completion_tokens=27, prompt_tokens=27, total_tokens=54, completion_tokens_details=None, prompt_tokens_details=None)
The reply is in choices[0].message: choices is a list because you can ask for several answers at once. finish_reason is "stop" when the model finished its answer ("length" means it ran out of room). usage counts tokens, the pieces of text models read and write, and providers charge for every one of them, in and out. Run lab requests in the terminal to see exactly what the model received.
The model remembers nothing
Each request stands alone. Tell the model something in one request and ask about it in the next:
$ python3 demo.py
Thanks for letting me know. How can I help you today?
I don't know that. You have not told me in this conversation.
A chat app "remembers" because it sends the whole conversation every time, the model's own replies included, as assistant messages. Press Next to follow what the model sees in each request:
The third request carries the history, so the answer is in what it was sent:
$ python3 demo.py
Your shop is Sharma Sweets in Indore.
Keeping that history is the app's job. In Python, watch for one trap: a list you append to is the caller's list, changed in place. Build a new list instead ([*history, asked, reply]), so whoever passed you the history still has the old one.
Confident and wrong
Now ask about Paisa Setu, with no system message and no documents:
$ python3 demo.py
Settlements are usually made on a T+2 basis, two working days after the transaction, directly to your registered bank account.
A model learns from text that existed before it was trained. Paisa Setu's policies were never in it, so it answers with what payment companies usually do, in the same confident voice as when it is right. Nothing in the reply tells you it guessed. Real models do the same: this is called a hallucination, and no setting turns it off. The model knows two things: its training, and what is in the request.
Too much to paste
The obvious fix: send the documents with the question. All 96 help articles in the system message:
$ python3 demo.py
96 help articles
Traceback (most recent call last):
...
openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 8192 tokens. However, your messages resulted in 52028 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}
Every model has a context window, the most tokens one request can hold. Bigger models have bigger windows, but you pay for every token on every question, and the documents grow every week. The real fix is to send only the few passages that answer this question. Finding them is what the rest of this lab builds.
The openai package raises one exception class per HTTP status (BadRequestError for 400, NotFoundError for 404), and each carries the API's own code. A mistake to avoid when you handle the too-long error: catching every error. Here except Exception decides a wrong model name is "too long":
def too_long(model: str) -> bool:
try:
client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "When are refunds sent?"}],
)
return False
except Exception:
return True
print(too_long("gpt-lab"))
$ python3 demo.py
True
Catch openai.APIStatusError as e instead and print the exception's class, e.status_code, e.code and e.body["message"]: the real error was a different one.
NotFoundError 404 model_not_found
The model `gpt-lab` does not exist or you do not have access to it.
Catch the narrowest class, check its code, and raise everything else, so a real failure is seen the moment it happens.
Your turn
Your four tasks are at the top, one file each. The last is Arjun's interview question: why not paste every document into every prompt? Run lab requests after each one to see what you sent.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
ask.py: ask the model a question
Kabir wants to see the model answer a merchant. Write ask(question) in ask.py: send lab-chat (MODEL) two messages, SYSTEM as the system message, then the question as a user message, and return the text of the reply, the content of the first choice's message.
ask("What does UPI stand for?") is the model's one-sentence answer about the Unified Payments Interface. The content can be None: return "" then.
- Task 2
usage.py: the answer, and what it cost
Write ask_with_usage(question) in usage.py: the same request as ask, returning an Answer whose answer is the reply's text, tokens_in is the response's usage.prompt_tokens, tokens_out is usage.completion_tokens, and finish_reason is the first choice's finish_reason.
For a short question the model finishes its answer, so finish_reason is "stop". Return the numbers exactly as the response reports them.
- Task 3
chat_turn.py: a conversation the model can follow
The model remembers nothing between requests. Write chat_turn(history, question) in chat_turn.py: send SYSTEM, then every message of history in order, then the question as a user message.
Return a new list: the history, then {"role": "user", "content": question}, then {"role": "assistant", "content": the reply's text}. Do not change the history you were given (no append): the app keeps the old list too.
With a history where the merchant said their id is M-1042, chat_turn(history, "What is my merchant id?") ends with the model's reply containing M-1042.
- Task 4 · interview question
Interview: paste_docs.py, paste in the documents
Write ask_with_docs(question, docs, model) in paste_docs.py: the system message is SYSTEM followed by every document, word for word, joined with blank lines; the question is a user message; send it to the model parameter, not MODEL. Return DocsAnswer(ok=True, answer=the reply's text) when the request works.
All of Paisa Setu's documents are far more than the model's 8,192-token context window, so the request fails with openai.BadRequestError whose code is "context_length_exceeded". Return DocsAnswer(ok=False, reason="too-long") for that error only. Any other error must still be raised: with the model "lab-chatt", which does not exist, the call must raise openai.NotFoundError.
Arjun will ask why you cannot paste every document into every prompt.
What you will learn
- The chat API: a client, a model and a list of messages with roles (system, user, assistant), through the official openai package.
- The reply: the text, why the model stopped, and the tokens it used.
- Conversations: the model remembers nothing, so you send the history.
- Context limits: why you cannot paste every document into the prompt, and how to catch exactly that error and no other.
More in Why RAG
Build The model does not know Paisa Setu yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly, with sources. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.