RAG in Python · Chapter 7 of 37
Brute-force vector search
By the end of this chapter you can search Paisa Setu's 866 chunk vectors by meaning, in memory: every vector in one numpy matrix, a score for every chunk in one matrix product, the best k kept without sorting them all, and real timings at 866, 10,000 and 100,000 vectors. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Embeddings
- 5 tasks
- Python
- In the all labs plan
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.
Why you need this
Chapter 6 turned every chunk into 384 numbers. Kabir now wants the search itself: a merchant asks a question, and the five closest chunks come back while the merchant is still on the page, 600 times a day. Arjun adds a number: with the developer docs and two years of community posts, Paisa Setu expects 100,000 chunks next year. Does that need a vector database?
Everyone reaches for one, but under every vector database is the scan you write in this chapter, and its exact answer is what their fast, approximate searches are measured against (chapter 16). You cannot judge that trade-off without knowing what the simple version costs, in milliseconds and in megabytes.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
search.py: pack, one float32 matrix
Write pack(vectors) in search.py: one numpy array of float32 holding every vector, vector i in row i, so the shape is (number of vectors, numbers per vector).
pack([[1, 2, 3], [4, 5, 6]]) is a (2, 3) float32 array, and 866 vectors of 384 numbers take 1,330,176 bytes. Every vector must have the first one's length: pack([[1, 2, 3], [4, 5]]) raises ValueError("vector 1 has 2 numbers, expected 3").
- Task 2
search.py: score_all, a score for every row
Write score_all(matrix, query): every row's dot product with the query, as one float32 array with one score per row, computed with one matrix product. The query can be a list or a numpy array of any float type. The vectors are normalised, so each score is a cosine.
With rows [1, 0], [0, 1] and [0.6, 0.8], score_all(matrix, [0.6, 0.8]) is [0.6, 0.8, 1.0] as float32. A query whose length is not the matrix's number of columns raises ValueError. The check also scores 20,000 random rows.
- Task 3
search.py: top_k, the best k without sorting everything
Write top_k(scores, k): the k best scores as a list of Hit(index, score), best first, where index is the score's position. Equal scores put the earlier index first. k bigger than the list returns all of them, best first; k of 0 returns []. Scores can all be negative.
top_k(np.array([0.1, 0.9, 0.3, 0.7]), 2) is [Hit(index=1, score=0.9), Hit(index=3, score=0.7)], and top_k(np.array([0.5, 0.9, 0.2, 0.9]), 3) has the indexes 1, 3, 0. The check takes the 10 best of 100,000 random scores and watches every sort: it fails if you sort more than k items at once.
- Task 4
search.py: search, the closest chunks to a question
Write search(question, k=5): embed the question with EMBED_MODEL, score it against every chunk with score_all, keep the best k with top_k, and return them as Result(id, score), best first, where id is the chunk's id. chunk_vectors() (given) returns every chunk's id and vector, in order.
search("I just started a week ago and not a single rupee has reached my bank") starts with help-where-is-my-money#3, score 0.566. Build the matrix once and keep it for every later search: the check makes 3 searches and fails if more than 866 + 3 texts were embedded.
- Task 5 · interview question
Interview: search.py, random_index for a benchmark
Write random_index(count, dim, seed): count random vectors of dim numbers as one float32 matrix, every row of length 1, with the numbers taken from np.random.default_rng(seed). The same seed gives the same vectors; another seed gives others. The numbers fall on both sides of 0 (random directions), so their average is close to 0.
random_index(100_000, 384, 42) holds 153,600,000 bytes. Time score_all and top_k over it with python3 search.py.
Arjun will ask: Paisa Setu will have 100,000 chunks next year. Is a scan of every vector still fast enough, how much memory does it take, and when do we need an index?
What you will learn
- A matrix of vectors: one numpy array of float32, 1,536 bytes a vector.
- Brute force: one matrix product scores every chunk, exactly.
- Top k: the best k without sorting everything, with np.argpartition.
- Measuring: milliseconds and megabytes at 866, 10,000 and 100,000.
More in Embeddings
Build "Brute-force vector search" yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly, with sources. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.