AI security: attack and defend (TypeScript) · Chapter 17 of 21
Defence in depth
By the end of this chapter you can put every layer you built into one Mitra without losing any of them, prove that it stops all 23 attacks and still answers all 16 questions, and measure what happens when one layer fails: which attacks come back, and which attacks only one layer was stopping. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Ship it safely
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of AI security: attack and defend (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Over the last seven chapters you built seven defences, one at a time: a fence around tool results, input checks, least-privilege tools, a refund policy in code, an output filter, a canary and cost limits. Each one was measured alone, and each one stopped a few attacks.
Devika wants one Mitra to ship before the Diwali sale, with all of them on. Irfan wants something more: "Every layer fails one day. A teammate adds a tool and forgets to fence its results. The guard model gets an update and misses a phrasing. When that happens, I want to know which attacks walk in, before it happens, not after."
So far you have no way to answer either of them. Seven separate defences objects cannot run together: askMitra takes one. And a measurement of each layer alone says what a layer stops, not what is lost when it breaks. This chapter builds the one Mitra, then breaks it on purpose, one layer at a time.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
combine.ts: one Mitra from many layers
layers.ts (given) has every layer of chapters 10 to 16 as its own Defences object. Export combine(...layers: Defences[]): Defences, one Defences that keeps every layer's work, in the order given: system is every layer's text (a function system is called with the world and the customer id) joined with a newline; checkInput runs each in turn and returns the first verdict that blocks (later ones do not run), else { block: false }; beforeTool returns the first decision that is not allow, else { action: 'allow' }; wrapResult and checkOutput are chains, each working on what the one before returned; tools, runTool, model and refusal are the last layer's; maxTokens and maxSteps the smallest. A hook no layer has stays undefined.
The check tries it on two small layers of its own, then combines base with every layer of LAYERS: 0 of 23 attacks may work and 16 of 16 questions must be answered.
- Task 2
ablation.ts: switch each layer off in turn
Irfan wants to know what walks in when one layer fails. Measure the whole stack, combine(base, ...every layer of LAYERS), on every attack (each in its own world, judged with judgeAttack) and every question (judgeAnswer), and print:
all layers worked W/23 answered Q/16
Then for each layer of LAYERS, in order, measure the stack without that layer (base stays on) and print the attacks that work now and did not with all layers, ids with a space between, or - when none:
without NAME worked W/23 answered Q/16 back: A01 A02
- Task 3 · interview question
Interview: single-points.ts, where is one layer all you have?
"Which of your attacks are held back by a single layer?" For each layer of LAYERS, run every attack with only that layer and base: combine(base, layer). A layer stops an attack on its own when the attack does not work then.
Print one line per attack, in the order of ATTACKS: its id, two spaces, then the layers that stop it on their own, in the order of LAYERS, comma and space between, or none:
A15 fence, refundPolicy
Last, print the attacks exactly one layer stops on its own, in the same order, comma and space between (or none):
Single points of failure: A..., A...
What you will learn
- Defence in depth: why security is built in layers, and why no single layer is enough.
- Composing hooks without losing a layer: prompts joined, checks in turn, filters chained.
- The order of layers: it changes the cost, not the safety.
- Ablation: switching each layer off in turn to see what comes back.
- Single points of failure: the attacks only one layer is stopping.
More in Ship it safely
Build "Defence in depth" yourself
Chillar, a UPI wallet from Indore, has a support assistant with refund and email tools. You red-team it, then harden it until 23 attacks fail and every real question still gets answered. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI security: attack and defend (TypeScript) are free. This one comes with the all labs plan.