AI support copilot project (TypeScript) · Chapter 7 of 12
The answer
By the end of this chapter you can turn a question and its five chunks into the messages a chat model reads, ask the model for an answer in a fixed JSON shape with structured outputs, read back its answer and the tokens it used, and keep only the citations that point at a source a member can really open. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Answers
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of AI support copilot project (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Chapter 6's search hands back the five chunks that answer a question best. Pranav still has no answer: he asked "My show was cancelled. When do I get my money back?", and the app can only show him five chunks of help pages. Sayali's three rules for the copilot are about the answer itself: from the help pages only, with the page it used, and "I don't know" when the pages do not say.
Nothing so far talks to a chat model. And Parda's WhatsApp tool needs more than a paragraph of text: it needs the answer, the ids it can link to, and a yes or no for "did the pages say", every time, in the same shape. A model that writes free text sometimes puts the ids at the end, sometimes in the middle, and sometimes cites a page that does not exist.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
prompt.ts: the answer's shape and the messages
In src/answer/prompt.ts write AnswerOut: a zod object with three required fields, answer (a string), citations (an array of strings) and found (a boolean), made strict so that an object with any other field is refused.
Then formatSource(hit): two lines, "[<chunkId>] <title> > <heading>", a new line, then the hit's text. Then buildMessages(question, hits): an array of two messages, a system message whose content is INSTRUCTIONS, a blank line, "Sources:", a blank line, then every hit's formatSource in order with a blank line between them; then a user message whose content is the question.
The Check parses a good answer, one with a field missing, one with wrong types and one with an extra field; then it formats the first source for "How much is the booking fee?" and compares both messages with its own, character for character.
- Task 2
service.ts: a structured answer, with its tokens
Write askModel(ai, messages) in src/answer/service.ts. Call ai.chat.completions.parse with model config.chatModel, the messages given, temperature 0 and response_format zodResponseFormat(AnswerOut, 'answer'). Return { out, usage }: out is the first choice's message.parsed, and usage is { input: usage.prompt_tokens, output: usage.completion_tokens }. When message.parsed is null (the model refused), throw an Error.
The Check reads the lab model's request log: one chat request with a strict json_schema response format that has answer, citations and found, the model and temperature 0; your out must equal the model's reply and your usage its usage. Then it gives your function a stand-in model that refuses.
- Task 3 · interview question
Interview: service.ts, only real citations
Write answerQuestion(ai, question) in src/answer/service.ts. hits = await retrieve(ai, question); then askModel with buildMessages(question, hits). Keep only the citations that are chunk ids of those hits, each once, in the model's order. Return { question, answer (the model's), found (the model's found, and only when a citation was kept), citations (for each kept id { chunkId, pageId, title, heading } from its hit), tokens (askModel's usage) }.
The Check gives your function the app's client with a stand-in parse, so it can make the model cite a made-up id, two real ones and a repeat, then only a made-up id; then it calls POST /ask.
Nikhil will ask: the answer schema already says citations is an array of strings, and the model is told to cite. Why check the ids again in TypeScript?
What you will learn
- Messages: a system message and a user message, and what goes in each.
- Sources with ids: why each chunk is sent with its id in square brackets, best first.
- Instructions that matter: what "answer only from the sources" changes, measured.
- Structured outputs: a zod object turned into a strict JSON schema by zodResponseFormat, chat.completions.parse, message.parsed, temperature: 0, and the tokens in usage.
- What a schema cannot promise: that the ids are real, or that the answer is true.
More in Answers
Build The answer yourself
Parda answers the same audience questions all day. You build the copilot that answers them from its own help pages, with sources, and says so when it does not know. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI support copilot project (TypeScript) are free. This one comes with the all labs plan.