Building with LLMs · Chapter 17 of 24

Tool calling

By the end of this chapter you can give a model tools: describe your own Python functions to it, read the calls it asks for, run them, send the results back, and keep the loop safe when the model asks for the wrong thing or the same thing forever. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Building with LLMs
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.

Why you need this

Half of SafarLine's WhatsApp queue is about one booking or one bus: "is my booking confirmed?", "any seat to Nashik tomorrow?". Lata can answer them because she can open the bookings and the timetable. The model cannot. It knows its training and what is in the request, and you cannot paste 300 bookings and 1,512 departures into every request: it would cost a fortune, it would not fit, and it would send every customer's phone number to the model.

What you want is the other way round: the model reads the question, works out which lookup would answer it, and asks your code to do it. Your code runs the lookup and hands back only that one result. That is tool calling (also called function calling), and it is how every assistant that "does things" works.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    describe.py: describe a tool

    Lata wants the model to look bookings up instead of guessing. Describe a tool for lab-chat named get_booking, with a one-sentence description and one required parameter, pnr, of type string.

    Send lab-chat one user message, exactly:

    Can you check my booking SLNJ5GCA?

    with your tool, and print two lines: the name of the tool the model asked for, then the value of its pnr argument.

  2. Task 2

    round_trip.py: the whole round trip

    Offer lab-chat the get_booking tool, with one user message, exactly:

    Is my booking SLASGU7P still on?

    When the model asks for get_booking, run it: find the booking with that PNR in /workspace/data/bookings.json and give back its record as JSON, without the phone key. If there is no such booking, give back a JSON object with an error key saying so.

    Then send a second request with the user's message, the model's own message with its tool call, and a tool message holding your result. Print only the text of the model's final answer.

  3. Task 3

    two_pnrs.py: two calls at once

    Offer lab-chat the get_booking tool, with one user message, exactly:

    Please check SL8ZG82E and SLQQQQQQ for me.

    The model asks for get_booking twice in one reply. Run every call it asks for and send back one tool message per call: the booking's record as JSON without the phone key, or a JSON object with an error key for a PNR that is not in bookings.json.

    Print only the text of the model's final answer.

  4. Task 4 · interview question

    Interview: agent.py, a tool loop that cannot be fooled

    Write answer(question, model) in agent.py. It sends the question as a user message with the get_booking tool and keeps going until the model answers in text, which it returns.

    Each time the model asks for tools, run every call and send the results back. Before running a call, check its arguments: get_booking takes exactly one argument, pnr, as text; for anything else, do not run it, and send back a JSON object with an error key as that call's result.

    If the model asks again for calls it already made (the same tool with the same arguments), do not run them again: send the conversation once more with a tool choice that forbids tools, and return that answer. Never send more than 5 requests for one question.

    The check asks with lab-chat, lab-chat-loopy (asks for the same tool after every result) and lab-chat-confused (gets one argument wrong).

What you will learn

  • Describing a tool: a name, a description and a JSON Schema for its arguments.
  • The model asks, your code runs: tool_calls, and arguments that arrive as JSON text.
  • The round trip: sending the result back in a tool message, and the rules the API enforces.
  • Several calls at once, and tool_choice.
  • When the model gets it wrong: bad arguments and loops.

More in Building with LLMs

Build "Tool calling" yourself

SafarLine, an intercity bus company, drowns in WhatsApp messages. You build its first AI assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.