Claude and the Anthropic SDK · Chapter 5 of 21
Choosing a model
By the end of this chapter you can list Claude's models and their limits from the API, measure the same work on all three in right answers, tokens, dollars and time, work out the cost of a right answer rather than of a request, and route each question to the cheapest model that gets it right. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: The Messages API
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of Claude and the Anthropic SDK are free. This one comes with the all labs plan.
Why you need this
Farhan has done the sums from chapter 1: Haiku 5.5 costs a fortieth of Opus 5.5 per token. "Use Haiku for everything," he says, "and we save 97%." Meher has one question back: "Does it get the amounts right?" A wrong amount sent to a family costs Parchi far more than any token: a family that plans for ₹80,000 and receives ₹72,000 does not come back.
So far every request in this lab has gone to Opus 5.5, because the first chapter said so. Nothing you have measured tells you whether a cheaper model would do, for which questions, or what "cheaper" means once wrong answers count. This chapter measures it.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
list_models.py: the models, from the API
Loop over client.models.list() and print one line per model, in the API's order: ID (NAME): context C, output O, $IN in, $OUT out.
NAME is display_name, C is max_input_tokens, O is max_tokens, IN and OUT are PRICES[id].input and .output written with :g (4, not 4.0). Numbers as they are, without commas.
- Task 2
compare_models.py: the same claims on three models
Send each claim in CLAIMS (as bill_message(id), system=SYSTEM, max_tokens 4096, default effort) to claude-opus-5-5, then claude-sonnet-5-5, then claude-haiku-5-5. A reply is right when amount_in(its text) equals CHECKED[id].
Print MODEL: R/8 right, I in, O out, $COST, $PER per right answer for each model (COST and PER with 6 decimals), then all right, cheapest: MODEL.
- Task 3
router.py: a router
Define route(question) -> str: "claude-sonnet-5-5" when the question names a claim id (CLM- and four digits, in any case), otherwise "claude-haiku-5-5".
Define ask(question) -> str: send it to route(question)'s model (max_tokens 4096, system=SYSTEM, one user message) and return the reply's text blocks, joined.
- Task 4 · interview question
Interview: model_ids.py
"A teammate's 'claude-opus-5.5' gets a 404. Why, and how do you check an id?" For each id in GUESSES, call client.models.retrieve(id) and print ID: NAME, or ID: not found when it raises anthropic.NotFoundError.
Then send "Namaste!" (max_tokens 1024) to the model "claude-opus-5.5", catch the error and print messages: STATUS TYPE.
What you will learn
- The three models and their ids, and client.models.list() for their limits.
- Measuring a model: right answers, tokens, dollars and time on the same work.
- Cost per right answer, and why the cheapest request is not the cheapest answer.
- Routing: easy questions to the cheap model, hard ones to the one that gets them right.
- Model ids are exact: the 404 for a guessed id, and how to check one first.
More in The Messages API
Build "Choosing a model" yourself
Parchi, a Jaipur startup, helps families claim health insurance. You build Sahay, a claims assistant that reads bills and policies with Claude, and measure what caching, models and effort cost. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Claude and the Anthropic SDK are free. This one comes with the all labs plan.