AI support copilot project · Chapter 4 of 12

Vector search

By the end of this chapter you can measure how alike two embeddings are with numpy, ask Postgres for the chunks nearest a question with pgvector's cosine distance, from SQLAlchemy, in a stable order, and say what an HNSW index does, what it gives up, and when DabbaDrop needs one. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Search
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of AI support copilot project are free. This one comes with the all labs plan.

Why you need this

Postgres now holds DabbaDrop's 162 chunks, each with the 384 numbers of its meaning. A member asks "Can I pay cash on delivery?". Somewhere in those 162 rows is the chunk that answers (payment-methods-03: "Cash on delivery is not available"), and the copilot must find it among the five it hands the model, for every one of Sana's 180 questions a day, in milliseconds.

Chapter 3 stored the arrows but nothing compares them. There is no way yet to say how near two arrows are, no query that asks Postgres for the nearest ones, and no route a member's question can reach.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    vector.py: cosine similarity with numpy

    Write cosine_similarity(a, b) in app/search/vector.py: the dot product of a and b divided by the length of a times the length of b, as a plain float (1.0 the same direction, 0.0 unrelated, -1.0 opposite). Use numpy: np.asarray(a, dtype=float), the @ operator, np.linalg.norm.

    The Check runs six small pairs ([3, 4] and [6, 8] point the same way, so 1.0), then embeds the question "Can I pay cash on delivery?" and two chunks and wants the cash chunk nearer than the holidays chunk.

  2. Task 2

    vector.py: the nearest chunks, in Postgres

    Write vector_search(db, query_vector, k=5) in app/search/vector.py. Return the k chunks nearest the query vector, nearest first, as Hit(chunk_id, page_id, title, heading, text, score). Select the chunk's id, page_id, heading and text, the page's title (join HelpPage), and the cosine distance (HelpChunk.embedding.cosine_distance(query_vector), pgvector's <=>); order by the distance, then the chunk's id; limit k. score is 1 - distance.

    The Check compares your 5 hits for "Can I pay cash on delivery?" with its own SQL, adds two chunks at the same distance (they must come back in id order), and calls GET /search?q=...&k=3.

  3. Task 3 · interview question

    Interview: the HNSW index migration

    Write the migration alembic/versions/0002_chunks_embedding_hnsw.py. upgrade() creates an index called chunks_embedding_hnsw on the table chunks, the column embedding, with postgresql_using="hnsw" and postgresql_ops={"embedding": "vector_cosine_ops"}; downgrade() drops it (op.drop_index with table_name="chunks").

    The Check runs alembic upgrade head, reads the index's definition from pg_indexes, wants EXPLAIN of a nearest-5 query (sequential scans off) to use it, then runs alembic downgrade 0001 and wants it gone.

    Kunal will ask: HNSW is an approximate index. Why would DabbaDrop accept a search that may miss the true nearest chunk, and when would you not add the index at all?

What you will learn

  • Cosine similarity: the dot product and the lengths, and why only the direction counts; numpy's arrays, @ and np.linalg.norm.
  • Cosine distance in pgvector: the <=> operator, and its cousins <-> and <#>.
  • The nearest k: ORDER BY a distance and LIMIT, and why a second sort key makes the order stable.
  • A SQLAlchemy select: cosine_distance, label, join, order_by, limit, and rows you read by name.
  • HNSW: what an approximate index does inside, what it trades away, and how EXPLAIN shows whether Postgres uses it.

More in Search

Build "Vector search" yourself

DabbaDrop answers the same customer questions all day. You build the copilot that answers them from its own help pages, with sources, and says so when it does not know. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI support copilot project are free. This one comes with the all labs plan.