RAG in Python · Chapter 5 of 37

Meaning as numbers

By the end of this chapter you can turn "how alike are these two texts?" into arithmetic: the dot product, a vector's length, cosine similarity and normalising, written from scratch, and find the nearest phrase to a question by hand. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Embeddings
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.

Why you need this

Kabir forwards a ticket: "money not reached in bank since Tuesday". The help article that answers it is titled "When will I receive my settlement?" and talks about settlements being delayed. Keyword search compares words, and the two share almost none, so the right article never comes up. A person sees at once that they mean the same thing.

Embedding models close that gap: they turn a text into a list of numbers, a vector, so that texts with similar meaning get similar numbers. The next chapter calls one. First you need the arithmetic that compares two vectors, because every vector database asks you to pick "cosine, dot product or Euclidean distance", and the wrong pick ranks badly without a single error.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    vectors.py: dot, the dot product

    Write dot(a, b) in vectors.py: multiply the numbers at the same position of a and b and add up the products. dot([1, 2, 3], [4, 5, 6]) is 32, dot([1, 0], [0, 1]) is 0, and it works for vectors of any length, 384 numbers included.

    Vectors of different lengths cannot be compared: raise ValueError with a message that names both lengths, like "Vectors have different lengths: 3 and 2", whichever one is shorter. zip alone stops at the shorter list without a word, so dot([1, 2, 3], [4, 5]) would quietly be 14.

  2. Task 2

    vectors.py: magnitude and cosine, length and angle

    Write magnitude(v) in vectors.py, the vector's length: the square root of the sum of its numbers squared, which is its dot product with itself. magnitude([3, 4]) is 5.0 and a zero vector's is 0.0.

    Write cosine(a, b), the dot product divided by both magnitudes: cosine([1, 2], [2, 4]) is 1 (same direction), cosine([1, 0], [0, 1]) is 0, cosine([1, 0], [-1, 0]) is -1, and making a vector longer does not change it. A zero vector has no direction: cosine([0, 0], [1, 2]) is 0.0, not a ZeroDivisionError. Use dot in both, so vectors of different lengths raise ValueError here too.

  3. Task 3

    vectors.py: normalise, a vector of length 1

    Write normalise(v) in vectors.py: return a new list pointing the same way with magnitude 1, every number divided by the vector's magnitude. normalise([3, 4]) is [0.6, 0.8]. Do not change the list you were given.

    A zero vector cannot be normalised: raise ValueError, for example "Cannot normalise a zero vector". For two normalised vectors, dot gives the cosine.

  4. Task 4 · interview question

    Interview: vectors.py, nearest phrases

    Write nearest(query, phrases, k) in vectors.py: return the k phrases closest to query, best first, each as Match(text=..., score=...) where score is the phrase's cosine with the query. When k is more than the number of phrases, return them all. Do not reorder or change the phrases list you were given.

    With the query [0.2, 0.9, 0.1], nearest(query, PHRASES, 2) is refund timelines, then refund fees. The complete settlement handbook, [9, 1, 0], wins with a plain dot product only because its vector is long: rank by cosine, or normalise everything and use dot.

    Arjun will ask why normalised vectors let you use the dot product, and what breaks when one vector is not normalised.

What you will learn

  • Vectors and the dot product: multiply matching numbers, add them up.
  • Length and cosine similarity: comparing direction, not size.
  • Normalising: why vectors of length 1 make cosine and dot the same.
  • The nearest neighbour: score every candidate, keep the best.

More in Embeddings

Build "Meaning as numbers" yourself

Paisa Setu, a payments company, needs its help centre to answer merchants correctly, with sources. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.