OpenAI Agents SDK · Chapter 17 of 24
RAG as a tool
By the end of this chapter you can turn Shikhar's policy into embeddings with the openai client, store them in Postgres with pgvector, give an agent a search tool that finds the rules nearest a parent's question, keep the rules behind each answer as a record, and explain why chapter 1's refund question is now answered right. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Memory
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of OpenAI Agents SDK are free. This one comes with the all labs plan.
Why you need this
In chapter 1 a parent asked how much of the fees comes back if their son leaves in the second week, and the agent said "Fees are fully refunded if a student leaves within 30 days of joining." Nidhi still thinks about that answer. Rule 2.2 says that in the first 14 days the family gets back the tuition paid minus ₹5,000, and from day 15 only half. A family that believed the agent would have planned around money that was never coming.
The agent was not lying: it had never seen the policy. Karthik's fix is the one this lab promised: give the agent Shikhar's facts. The policy has 21 rules today and will have more; pasting all of it into every request costs tokens on every turn and gets worse as it grows. So the agent will search the policy and read only the rules that answer the question. That pattern is called RAG, retrieval augmented generation: find the right text first, then let the model write from it.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
embed_store.py: the policy in pgvector
Embed every rule of /workspace/data/policy.md (the lines that start with a digit, like 2.2 A student who leaves...) with the openai client's embeddings for the model lab-embed, all in one call, and store them in Postgres (psycopg.connect() finds the lab's database).
Make the pgvector extension if it is missing, and a table policy_rules if it is missing, with the columns rule (text, the primary key: the rule's number, like 2.2), text (the rule without its number) and embedding (vector(384), the embedding of the whole line). Insert one row per rule; when the rule is already there, replace its text and embedding. Then print Stored, the number of rows in the table, then rules: Stored 21 rules.
The Check runs your program twice: the table must still hold each rule once.
- Task 2
search_tool.py: a search tool the agent calls
Make an agent for lab-chat with the instructions You are Shikhar Classes' helpdesk assistant. Answer only from the policy rules search_policy finds, and give the rule number.
Give it a function tool search_policy with one parameter, query (a string), and the docstring Search Shikhar Classes' student and parent policy for the rules about a question: admissions, scholarships, fees, instalments, refunds, classes, tests, results, doubt sessions, attendance and the hostel. The tool embeds the query with lab-embed, finds the 3 rules of policy_rules nearest to it by cosine distance, and returns them as text, one rule per line: its number, a space, its text.
Run the agent on Is the registration fee refunded? and print the final output. Use the table task 1 made.
- Task 3
sources.py: answers with their sources
Make the agent and the search_policy tool of task 2 again, with a context: a dataclass Sources with one field, rules, a list of strings that starts empty. Each time search_policy runs, it adds the numbers of the rules it found to the context's rules, in order. search_policy still has one parameter for the model, query.
Run the agent on When will the refund be paid, and can I get it in cash? with a new Sources as its context, and print the final output, then a last line Sources: and the rule numbers, joined by a comma and a space (like Sources: 4.1, 2.3, 1.2).
- Task 4 · interview question
Interview: refund_right.py, the refund question, now right
Chapter 1's agent answered this confidently and wrongly: My son wants to leave in the second week. How much of the fees do we get back?
Ask it twice. First an agent for lab-chat with no instructions and no tools, as in chapter 1. Then the agent and search_policy tool of task 2. Print Before: and the first final output, then After: and the second (the second may run over several lines).
The interviewer will ask: why is the second answer right, and what could still make it wrong?
What you will learn
- Embeddings: text as a list of numbers, near for near meanings, made with the openai client and lab-embed.
- pgvector: embeddings in a Postgres table, searched by distance.
- A search tool: a function tool that embeds the query and returns the nearest rules, with their numbers.
- Sources: the rules behind an answer, collected in the run's context.
More in Memory
Build "RAG as a tool" yourself
Shikhar Classes, a Kota coaching institute, answers parents and students on WhatsApp all day. You build its helpdesk assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of OpenAI Agents SDK are free. This one comes with the all labs plan.