RAG: search that answers · Chapter 6 of 37
Embedding text
By the end of this chapter you can turn Paisa Setu's text into real embeddings with the embeddings endpoint: one text, all 866 chunks in batches and in order, with the right errors retried, and the question embedded by the same model as the documents. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Embeddings
- 5 tasks
- In the all labs plan
The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.
Why you need this
Chapter 5 compared vectors that Ira made by hand. Now Kabir wants the real thing: every one of the 866 chunks of the help centre, policies and runbooks as a vector, so that "money not reached in bank" can find the settlement articles. A model makes the vectors; your code has to feed it.
That is where it goes wrong in production. The endpoint takes at most 256 texts per request, so a loop that sends everything at once fails, and a loop that mixes up the order stores the refund chunk's vector under the login chunk's id, with no error at all. A chunk that is an empty heading gets the whole batch refused. A rate limit in the middle of the night stops the ingest job. And when Arjun switches models next quarter, every old vector becomes meaningless.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
embedOne: embed one text
Send text to ai.embeddings.create with the model EMBED_MODEL (lab-embed) and return its vector: the embedding of the first item in the response's data, unchanged.
await embedOne('I just started a week ago and not a single rupee has reached my bank') is an array of 384 numbers starting 0.0626..., -0.1067..., with length 1 (lab-embed normalises). One request, one text.
- Task 2
embedMany: every chunk, in batches
Embed all the texts with at most MAX_BATCH (256) per request, one request after the other, using client (it is ai unless the Check passes its own). Return { vectors, tokens }: vectors[i] is the embedding of texts[i], and tokens is the sum of every response's usage.prompt_tokens.
The 866 chunk texts take 4 requests, in order: 256, 256, 256, then 98. Each item of a response's data has index, the position of its text in that request: put each embedding at the batch's start plus its index. The Check's own client returns a batch's items in reverse order, so code that pushes them as they come fails.
- Task 3
embedMany: no empty texts
Also in embedMany: the endpoint refuses an empty string with a 400 for the whole request. Before the first request, check every text, and throw an Error 'Text 1 is empty' (with the text's position) when one is empty or only spaces.
embedMany(['UPI refund', ' ', 'card fees']) rejects with 'Text 1 is empty'. With 300 texts where text 280 is empty, it rejects with 'Text 280 is empty' and sends nothing: checking batch by batch would send the first 256 and leave the job half done.
- Task 4
withRetry: retry what is worth retrying
Call fn and return its result. When it throws an OpenAI.APIError whose status is 429 or 500 and above, wait and call it again: baseMs before the first retry, then twice baseMs, then four times, doubling each time. After retries retries (so retries plus one calls), throw the last error.
Any other error is thrown at once, with no retry: a 400 (an empty string sent with a client whose maxRetries is 0) is called once, and so is a TypeError from your own code. Two 429s then success calls fn 3 times.
- Task 5 · interview question
Interview: embedQuery, the same model as the index
A stored index remembers which model built its vectors. Embed the question with index.model, not EMBED_MODEL, and return its vector. With { model: 'lab-embed', dimensions: 384 } it returns the question's 384 numbers.
With { model: 'text-embedding-3-small', dimensions: 1536 } the request must go to that model, which the lab does not have, so the 404 reaches the caller. With { model: 'lab-embed', dimensions: 1536 } the vector's length does not match: throw an Error naming both sizes, like 'The index has 1536 dimensions, but lab-embed returned 384'.
Arjun will ask: when the documents move to a better embedding model, do the questions have to change too, and why?
What you will learn
- The embeddings endpoint: one or many texts in, one vector each out.
- Batches and order: the 256 limit, and the index on every result.
- Errors: empty strings, which errors to retry and which never to.
- One model for both: why questions and documents must share a model.
More in Embeddings
Build "Embedding text" yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.