LangChain and LangGraph · Chapter 18 of 25
Streaming graphs
By the end of this chapter you can watch a graph while it runs: see each node's result the moment it finishes, see the whole state grow step by step, show the model's answer word by word from inside a node, and choose the right kind of stream for a progress bar, a debugger or a chat screen. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: LangGraph
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.
Why you need this
In chapter 5 you streamed one model's reply: llm.stream(...) gave you the answer a few tokens at a time, so a tenant saw it being typed. Since then the assistant has become a graph: several nodes, more than one model call, a lookup in between. Call invoke on it and the tenant sees nothing at all until the very last node has finished.
Mrunal watched a tenant send the same question three times, because the screen stayed blank for a few seconds and he thought the app had hung. Sid wants two things: the help desk screen should show each step as it finishes, and the tenant should see the answer appear as it is written, like a person typing. LangGraph streams a graph the way LangChain streams a model, with one difference: there is more than one thing to watch, so you choose a stream mode.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
progress.py: node by node
helpdesk.py (given, do not change it) has RoomRaja's help desk graph: classify, find_rule, answer. Run it with stream in the updates mode for the message What is the lock-in period? (the input is a dictionary with the key message).
For each node that finishes, print its name, a space, done:, a space, and the keys it returned joined by a comma and a space. When the answer node finishes, also print Answer:, a space and its answer. Four lines in all.
- Task 2
state_after.py: the whole state, step by step
Run helpdesk.py's graph with stream in the values mode for the message I paid rent on the 8th, why is there a late fee. After each step, print the state's keys joined by a comma and a space; then print the answer from the last state.
- Task 3
tokens.py: the answer as it is written
Run helpdesk.py's graph with stream in the messages mode for the message Is the deposit two months or one month? and print the content of each chunk that came from the answer node as it arrives, with nothing between them, then one newline at the end. Only the answer's text is printed, on one line.
- Task 4 · interview question
Interview: chat_ui.py, what a chat UI needs
Write a generator chat(message) in chat_ui.py that runs helpdesk.py's graph once, with stream and the two modes updates and messages together, and yields, in the order they happen: the pair ("status", node) when a node finishes, and ("token", text) for each non-empty chunk of the answer node's reply.
Under if __name__ == "__main__":, for Can I change rooms for free?, print each status except answer's as the node's name in square brackets on its own line, and the tokens one after another on one line, then a newline. The check calls chat with other messages.
The interviewer will ask which stream mode you would use for a chat UI, and why not values.
What you will learn
- stream instead of invoke: the same run, in pieces.
- updates: what each node returned, as it finishes.
- values: the whole state after each step.
- messages: the model's tokens, from inside any node.
- Several modes at once, and which one a chat screen needs.
More in LangGraph
Build "Streaming graphs" yourself
RoomRaja, a Pune co-living company, answers tenants on WhatsApp by hand. You build its assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.