Memory and multi-agent systems · Chapter 3 of 16

Summary memory

By the end of this chapter you can summarise a conversation with a model, measure what the summary kept against the facts that were really stated, keep a summary rolling across months of sessions, and replace prose with a structured summary that cannot quietly lose a fact. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Memory
  • 5 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Memory and multi-agent systems are free. This one comes with the all labs plan.

Why you need this

Chapter 2 kept the newest messages and dropped the oldest. The oldest is where a wedding's decisions start, so the next idea is the one every long chat product uses: summarise what you drop, and send the summary instead.

Baraat Bliss tried it over the summer. In August, Rekha brought a complaint to the planning meeting:

  • The assistant answered a cousin with 10 December for the mehndi. The mehndi had moved to the 9th.
  • Karan found why: "It read an old summary." The move was in the thread. The summary, written weeks earlier and carried forward ever since, still had the old date.

The meeting decided that dates and counts come from the fact store, not summaries. This chapter shows why, with numbers: you measure exactly which facts a summary keeps, drops and gets stale, against the labels in threads.jsonl.

What you build: 5 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    transcript.py: a transcript to summarise

    Write transcript(messages) in transcript.py for thread messages (the dicts baraat's thread gives). It returns one line per message: the sender, a space, the role in brackets, a colon, a space and the text, for example:

    Rekha (planner): Good question. I will share the details by email.

    A text with line breaks becomes one line, its line breaks and runs of spaces turned into single spaces. Then, under if __name__ == "__main__":, print the transcript of WD1063's session 2. The check also tries other sessions and a message of its own.

  2. Task 2

    summarise.py: what one summary keeps

    Write summarise(messages, words=60) in summarise.py: one request to lab-chat at temperature 0, with the system message Summarise this conversation in N words. (N is words) and the messages' transcript (task 1) as the user message; it returns the reply's text.

    Write kept(summary, wedding_id, session): the keys of the facts stated in that session (each message's facts label, in order) that the summary mentions, decided with mentions from mentions.py (given).

    Under if __name__ == "__main__":, summarise each session of WD1063 in 60 words and print one line per session, for example:

    session 1: 2 of 3 facts kept, lost budget

    with a comma, lost, and the lost keys in the order they were stated (joined with a comma and a space) only when some were lost.

  3. Task 3

    rolling.py: a rolling summary over a whole thread

    Write the class RollingSummary in rolling.py. RollingSummary(words=60) starts with self.text as an empty string. add(messages) sends one summary request like task 2's (temperature 0, Summarise this conversation in N words.) whose user message is the new session's transcript, with one line before it when there is a summary so far: Baraat Bliss (summary so far): and the summary so far. add keeps the reply in self.text and returns it.

    Under if __name__ == "__main__":, add every session of WD1063 in order, print the final summary, then three lines about WD1063's facts in facts.json: holds: the keys whose current value the summary mentions; stale: the other keys whose older value (in the fact's history) it mentions; lost: the rest. Each list is sorted and joined with a comma and a space, or none when empty.

  4. Task 4

    structured.py: a structured summary

    Write update(memory, message) in structured.py: for each fact in the message's facts label, memory[key] becomes a dict with value, by (the sender) and at (the first 10 characters of the message's at); a message with tentative true changes nothing; it returns memory. Write notes(memory): one line per key, sorted by key, like

    - budget: 6500000 (said by Gurpreet Singh on 2026-06-18)

    Under if __name__ == "__main__":, build task 3's rolling summary of WD1063 and a memory from every message of WD1063, then ask lab-chat What is the budget? twice at temperature 0, with the system message You are Baraat Bliss's planning assistant. Answer from these notes. then a blank line, Notes: on its own line and the memory: first the rolling summary, then notes(memory). Print rolling: and the first reply, then structured: and the second, each on its own line.

  5. Task 5 · interview question

    Interview: fix_me.py, a compaction that forgets the last one

    compact(messages, keep) in fix_me.py summarises a replayed thread's old messages into a second system message that starts with Earlier in this conversation: and keeps the last keep messages. Run it: after five compactions the summary holds nothing from the first sessions.

    Fix compact so that the summary is carried forward: when the message after the system message is an earlier summary, the transcript compact sends starts with the line Baraat Bliss (summary so far): and that summary's text, then the old messages' contents, one per line. Keep the function's name, parameters and what it returns. The check calls it with and without an earlier summary.

What you will learn

  • A summary is one more request: a transcript in, a short text out.
  • What a summary keeps: measured fact by fact against the labels.
  • Compaction: a summary plus the recent messages.
  • Rolling summaries, and how they go stale over months.
  • A structured summary: the latest value per fact, with who said it.

More in Memory

Build "Summary memory" yourself

Baraat Bliss, a Jaipur wedding planner, must remember months of family decisions across many agents. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Memory and multi-agent systems are free. This one comes with the all labs plan.