AI support copilot project · Chapter 3 of 12

Embeddings in Postgres

By the end of this chapter you know what an embedding is and how to ask a model for many of them at once, you can store 384 numbers in a Postgres column with pgvector and map it in SQLAlchemy, and you can load every help page and chunk into Postgres in one transaction that a failing model cannot break. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: The pages
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of AI support copilot project are free. This one comes with the all labs plan.

Why you need this

Chapter 2 cut DabbaDrop's 34 current pages into 162 chunks. The search in chapter 4 must find, among those 162, the few that answer a member's question, and it must find them by meaning, not by spelling. Tara writes "can I stop my dabbas while I'm in Pune for a week?". The chunk that answers her says "You can pause your plan for 3 to 14 days at a time". The two share almost no words: stop and pause, a week and 3 to 14 days.

So every chunk needs a form a program can compare by meaning, and it needs to live somewhere a search can reach it in milliseconds, for every question, all day. Today the chunks are cut from the files on every request and thrown away, nothing in the project can turn text into numbers, and the database has no tables.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    pages.py: the pages and chunks tables

    Write the two classes in app/db/models/pages.py to match the given migration (alembic/versions/0001_pages_and_chunks.py). Every column is not null.

    HelpPage (the table pages): id, text and the primary key; title, text; updated, a date; tags, an array of text (ARRAY(String)) that Postgres defaults to an empty array ("{}"); body, text; ingested_at, timestamptz that Postgres defaults to now().

    HelpChunk (the table chunks): id, text and the primary key; page_id, text, a foreign key to pages.id with ondelete CASCADE; position and tokens, integers; heading and text, text; embedding, Vector(384) from pgvector.sqlalchemy (annotated Mapped[list[float]]); and in __table_args__ a unique constraint called chunks_page_position on (page_id, position).

    The Check compares each class with this list, runs the migration, and saves a page and a chunk of 384 numbers through your classes.

  2. Task 2

    embeddings.py: embed texts in batches

    Write embed_texts(ai, texts, batch_size=64) in app/corpus/embeddings.py. Return one embedding per text, a list of 384 floats, in the texts' order. Send the texts batch_size at a time, one ai.embeddings.create call per batch with model=settings.embed_model, input=the batch and dimensions=settings.embed_dims; put each response's items (res.data, each with .embedding and .index) back in index order. No texts: return [] without a request.

    The Check embeds 10 texts with batch_size 4 and reads the lab model's request log: 3 requests of 4, 4 and 2 texts, each embedding the right text's; then an empty list.

  3. Task 3 · interview question

    Interview: ingest.py, the pages into Postgres in one transaction

    Write ingest(db, ai, pages) in app/corpus/ingest.py. Chunk every page (chunk_page with settings.chunk_tokens and settings.chunk_overlap) and embed every chunk's embedding_text with one embed_texts call, before you change the database. Then delete every HelpPage (their chunks go with them), add a HelpPage for each page (tags as a list), flush, add a HelpChunk for each chunk with its embedding, and commit once. Return {"pages": n, "chunks": m}.

    The Check ingests data/help (34 pages, 162 chunks; its own embed_texts, so this task stands alone, but it needs Task 1's classes), reads jain-food-04 back, ingests again (nothing doubled), ingests without jain-food (it is gone), then makes the model fail and wants the database unchanged.

    Kunal will ask: why embed everything before the first delete, and commit only once at the end?

What you will learn

  • Embeddings: what the 384 numbers are, and how the embeddings API answers many texts in one request.
  • Batches: why and how to send texts a batch at a time, in order.
  • pgvector: Postgres's vector(384) type, and arrays of text.
  • The tables in SQLAlchemy: Vector(384), ARRAY(String), server defaults, a cascading foreign key, matching a given migration.
  • One transaction: delete(), flush, commit, rollback, what other connections see meanwhile, and why the model is asked first.

More in The pages

Build "Embeddings in Postgres" yourself

DabbaDrop answers the same customer questions all day. You build the copilot that answers them from its own help pages, with sources, and says so when it does not know. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI support copilot project are free. This one comes with the all labs plan.