Building with LLMs (TypeScript) · Chapter 18 of 24
Embeddings
By the end of this chapter you can turn text into an embedding, compare two texts by meaning with cosine similarity, and search SafarLine's help centre for the entries that answer a customer's question, even when they share no words with it. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Building with LLMs
- 4 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of Building with LLMs (TypeScript) are free. This one comes with the all labs plan.
Why you need this
The help centre (faq.md) has 26 answers, and Lata pastes the right one into a reply many times a day. Zoya wants the assistant to do it: find the entry that answers the message, and answer from it (chapter 1 showed why the model must be sent the facts).
Finding the entry is the hard part. A customer writes "my dog wants to come along"; the entry is called "Can I bring my pet?". includes, === and regular expressions work on characters, and the two share no useful word. You could send all 26 entries with every message, but the whole help centre on every request costs tokens, and a real help centre has thousands of entries, not 26. You need to compare texts by meaning, cheaply, and send the model only the few that matter.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
embed.ts: a sentence as numbers
Ask lab-embed for the embedding of this sentence, exactly:
Is there a sleeper bus to Hyderabad tonight?
Print three lines: how many numbers the vector has; its first three numbers, each with 4 decimals, separated by spaces; and its length (the Euclidean norm) with 3 decimals.
- Task 2
similar.ts: how alike are two texts?
Write and export cosine(a: number[], b: number[]): number in similar.ts: the cosine similarity of two vectors of the same length, for any vectors: cosine([1, 0], [2, 0]) is 1.
Then, inside if (import.meta.main), embed these four texts with lab-embed in one request: My money hasn't come back yet / When will my refund arrive? / Can I change my travel date? / How much luggage can I bring?
Print the last three, each with its cosine similarity to the first, most similar first, one per line: the score with 3 decimals, a space, and the question.
- Task 3
search.ts: search the FAQ by meaning
Write and export search(query: string, k = 3, minScore = 0.3): Promise<string[]> in search.ts. It returns the questions of the k entries of /workspace/data/faq.md closest in meaning to the query, most similar first, leaving out any that score below minScore, so a query the FAQ cannot answer returns an empty array.
An entry is a question in bold, between two pairs of asterisks, and its answer on the next line. Embed each entry as the question, a space, and the answer, without the asterisks. The check compares your results with its own for several queries.
Embed the FAQ once, in one request, however many times search is called: only the query is new each time.
- Task 4 · interview question
Interview: fix-search.ts, wrong and wasteful
bestMatches(query) in fix-search.ts should return the 3 FAQ questions most similar in meaning to the query, most similar first. It runs, but it returns the wrong questions, and it sends the embeddings endpoint one request per FAQ entry: 27 requests where two would do.
Fix both. Loading the file and one search must make at most two requests to the embeddings endpoint: one for the whole FAQ and one for the query. Keep what is embedded the same.
What you will learn
- What an embedding is: a text as an array of numbers, placed by meaning.
- The embeddings endpoint, one text or many in one request.
- Cosine similarity on plain number[] arrays: how alike two meanings are.
- Semantic search: embed once, compare every time, keep the best.
- Where it fails: other languages, and questions nothing answers.
More in Building with LLMs
Build "Embeddings" yourself
SafarLine, an intercity bus company, drowns in WhatsApp messages. You build its first AI assistant, in TypeScript. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Building with LLMs (TypeScript) are free. This one comes with the all labs plan.