Building with LLMs · Chapter 18 of 24

Embeddings

By the end of this chapter you can turn text into an embedding, compare two texts by meaning with cosine similarity, and search SafarLine's help centre for the entries that answer a customer's question, even when they share no words with it. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Building with LLMs
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.

Why you need this

The help centre (faq.md) has 26 answers, and Lata pastes the right one into a reply many times a day. Zoya wants the assistant to do it: find the entry that answers the message, and answer from it (chapter 1 showed why the model must be sent the facts).

Finding the entry is the hard part. A customer writes "my dog wants to come along"; the entry is called "Can I bring my pet?". The words in and == work on characters, and the two share no useful word. You could send all 26 entries with every message, but the whole help centre on every request costs tokens, and a real help centre has thousands of entries, not 26. You need to compare texts by meaning, cheaply, and send the model only the few that matter.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    embed.py: a sentence as numbers

    Ask lab-embed for the embedding of this sentence, exactly:

    Is there a sleeper bus to Hyderabad tonight?

    Print three lines: how many numbers the vector has; its first three numbers, each with 4 decimals, separated by spaces; and its length (the Euclidean norm) with 3 decimals.

  2. Task 2

    similar.py: how alike are two texts?

    Write cosine(a, b) in similar.py: the cosine similarity of two vectors of the same length (lists or numpy arrays), as a float, for any vectors: cosine([1, 0], [2, 0]) is 1.0.

    Then, under the main guard, embed these four texts with lab-embed in one request: My money hasn't come back yet / When will my refund arrive? / Can I change my travel date? / How much luggage can I bring?

    Print the last three, each with its cosine similarity to the first, most similar first, one per line: the score with 3 decimals, a space, and the question.

  3. Task 3

    search.py: search the FAQ by meaning

    Write search(query, k=3, min_score=0.3) in search.py. It returns the questions of the k entries of /workspace/data/faq.md closest in meaning to the query, most similar first, leaving out any that score below min_score, so a query the FAQ cannot answer returns an empty list.

    An entry is a question in bold, between two pairs of asterisks, and its answer on the next line. Embed each entry as the question, a space, and the answer, without the asterisks. The check compares your results with its own for several queries.

    Embed the FAQ once, in one request, however many times search is called: only the query is new each time.

  4. Task 4 · interview question

    Interview: fix_search.py, wrong and wasteful

    best_matches(query) in fix_search.py should return the 3 FAQ questions most similar in meaning to the query, most similar first. It runs, but it returns the wrong questions, and it sends the embeddings endpoint one request per FAQ entry: 27 requests where two would do.

    Fix both. Loading the file and one search must make at most two requests to the embeddings endpoint: one for the whole FAQ and one for the query. Keep what is embedded the same.

What you will learn

  • What an embedding is: a text as a list of numbers, placed by meaning.
  • The embeddings endpoint, one text or many in one request.
  • Cosine similarity with numpy: how alike two meanings are.
  • Semantic search: embed once, compare every time, keep the best.
  • Where it fails: other languages, and questions nothing answers.

More in Building with LLMs

Build "Embeddings" yourself

SafarLine, an intercity bus company, drowns in WhatsApp messages. You build its first AI assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.