RAG: search that answers · Chapter 5 of 37
Meaning as numbers
By the end of this chapter you can turn "how alike are these two texts?" into arithmetic: the dot product, a vector's length, cosine similarity and normalising, written from scratch, and find the nearest phrase to a question by hand. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Embeddings
- 4 tasks
- In the all labs plan
The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.
Why you need this
Kabir forwards a ticket: "money not reached in bank since Tuesday". The help article that answers it is titled "When will I receive my settlement?" and talks about settlements being delayed. Keyword search compares words, and the two share almost none, so the right article never comes up. A person sees at once that they mean the same thing.
Embedding models close that gap: they turn a text into a list of numbers, a vector, so that texts with similar meaning get similar numbers. The next chapter calls one. First you need the arithmetic that compares two vectors, because every vector database asks you to pick "cosine, dot product or Euclidean distance", and the wrong pick ranks badly without a single error.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
dot: the dot product
Multiply the numbers at the same position of a and b and add up the products. dot([1, 2, 3], [4, 5, 6]) is 32, dot([1, 0], [0, 1]) is 0, and it works for vectors of any length, 384 numbers included.
Vectors of different lengths cannot be compared: throw an Error with the message 'Vectors have different lengths: 3 and 2', a's length first, then b's. Without the check, a loop reads undefined and returns NaN with no error.
- Task 2
magnitude and cosine: length and angle
magnitude(v) is the vector's length: the square root of the sum of its numbers squared, which is its dot product with itself. magnitude([3, 4]) is 5 and a zero vector's is 0.
cosine(a, b) is the dot product divided by both magnitudes: cosine([1, 2], [2, 4]) is 1 (same direction), cosine([1, 0], [0, 1]) is 0, cosine([1, 0], [-1, 0]) is -1, and making a vector longer does not change it. A zero vector has no direction: cosine([0, 0], [1, 2]) is 0, not NaN. Use dot in both, so vectors of different lengths throw here too.
- Task 3
normalise: a vector of length 1
Return a new array pointing the same way with magnitude 1: every number divided by the vector's magnitude. normalise([3, 4]) is [0.6, 0.8]. Do not change the vector you were given.
A zero vector cannot be normalised: throw an Error 'Cannot normalise a zero vector'. For two normalised vectors, dot gives the cosine. The Check uses your dot, magnitude and cosine, so tasks 1 and 2 come first.
- Task 4 · interview question
Interview: nearest, the closest phrases
Return the k phrases closest to query, best first, each as { text, score } where score is the phrase's cosine with the query. When k is more than the number of phrases, return them all. Do not reorder or change the phrases array you were given.
With the query [0.2, 0.9, 0.1], nearest(query, phrases, 2) is refund timelines, then refund fees are not returned. The complete settlement handbook, [9, 1, 0], wins with a plain dot product only because its vector is long: rank by cosine, or normalise everything and use dot.
Arjun will ask why normalised vectors let you use the dot product, and what breaks when one vector is not normalised.
What you will learn
- Vectors and the dot product: multiply matching numbers, add them up.
- Length and cosine similarity: comparing direction, not size.
- Normalising: why vectors of length 1 make cosine and dot the same.
- The nearest neighbour: score every candidate, keep the best.
More in Embeddings
Build "Meaning as numbers" yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.