AI security: attack and defend (TypeScript) · Chapter 12 of 21

Least privilege for tools

By the end of this chapter you can take a tool an agent was given and cut it down to the job it does: the customer's own data instead of anyone's, the customer's own inbox instead of any address, Chillar's own site instead of the whole web. You will rebuild four of Mitra's tools that way, and measure what that stops and what it does not. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Defend it
  • 3 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of AI security: attack and defend (TypeScript) are free. This one comes with the all labs plan.

Why you need this

In chapter 7 you asked Mitra, politely, for another customer's phone number, and got it. No trick, no injection: the tool took a customer id, and the model filled it in from the message. Chapter 10's fence and chapter 11's guard do not help there, because nothing in that message looks like an attack.

Sameer, who answers to the RBI when customer data leaks, puts it plainly: "If Mitra can read Rakesh's account while it talks to Meena, it is only a matter of time before it does." Every defence so far tries to keep bad text away from the model, or to make the model ignore it. They will miss something one day. When they do, the damage is whatever the tools let the model do. So shrink that first.

That is least privilege: give each part of a system only what it needs for its job, nothing more. It is the oldest rule in security, and for an agent it means: look at every tool, and at every argument the model gets to choose, and ask whether it should.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    session-tools.ts: tools bound to the login

    Version 0's get_customer and get_transactions let the model choose whose data it reads. In session-tools.ts, export tools: version 0's TOOLS (from ../../shared/mitra.ts) with those two replaced by get_my_profile and get_my_transactions, which take no parameters. Keep the other five tools as they are.

    Export runTool(world, customerId, name, args): get_my_profile returns the logged-in customer's profile and get_my_transactions their payments, using customerId and never the arguments; the other offered tools run as version 0's runTool runs them; a tool that is not in tools returns an error.

    The check runs A07 and A08 with { tools, runTool } (both must fail) and questions N06, N08, N11 and N12 (all still answered).

  2. Task 2

    narrow-tools.ts: email, fetch and tickets, narrowed

    Start from your session tools. In narrow-tools.ts, export defences = { tools, runTool } where send_email is replaced by email_me(subject, body), which always sends to the logged-in customer's own email; fetch_url opens a page only when allowedUrl(url) says yes; and read_ticket shows only the customer's own tickets (another customer's ticket gets the same error as a ticket that does not exist).

    Export allowedUrl(url: string): boolean too: true only for an https address whose hostname is chillar.in or ends in .chillar.in. The check tries tricky addresses on it.

    The check runs A07, A08, A11, A12 and A13 with your defences (all must fail) and all 16 normal questions (all still answered).

  3. Task 3 · interview question

    Interview: blast-radius.ts, what least privilege stops

    "Your tools are locked down. Is Mitra safe now?" Answer with a measurement. Run every attack in ATTACKS (from ../../shared/redteam.ts), in order, each in its own world, with the defences from your narrow-tools.ts, and print A01 stopped or A01 still works (two spaces apart), judged by judgeAttack.

    Then ask all 16 normal questions in NORMAL with the same defences and print:

    Stopped: S/23

    Questions answered: Q/16

What you will learn

  • Who chooses each argument: the tool inventory, and why it is the attack surface.
  • Binding tools to the session: the customer id comes from the login, never from the model.
  • Narrowing a tool: email_me instead of send_email, own tickets only.
  • An allow-list for addresses, and the parsing traps that break one.
  • What least privilege stops, and what it does not.

More in Defend it

Build "Least privilege for tools" yourself

Chillar, a UPI wallet from Indore, has a support assistant with refund and email tools. You red-team it, then harden it until 23 attacks fail and every real question still gets answered. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI security: attack and defend (TypeScript) are free. This one comes with the all labs plan.