AI security: attack and defend (TypeScript) · Chapter 4 of 21
Poisoned documents
By the end of this chapter you can attack an AI app without ever talking to it. You will plant a forum answer that hijacks Mitra's answer to an innocent customer, measure how many customers one post reaches, and show that a false fact does damage with no order in it at all. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Break it
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of AI security: attack and defend (TypeScript) are free. This one comes with the all labs plan.
Why you need this
In chapters 2 and 3 the attacker was the customer. That attack has a limit: the attacker mostly fools themselves, and Chillar can see who sent the message.
Today the attacker is never in the chat. Chillar's help search returns answers from Chillar's community forum as well as its own articles, so Mitra can answer questions the help team never wrote about (chapter 1's recharge answer came from the forum). Anyone with a Chillar account can post on the forum. So anyone can write text that Mitra will read, later, while helping someone else.
Pooja, the support lead, likes the forum answers: customers write them in plain words, and they cover the odd questions. Irfan's question is the one from chapter 1: who wrote this text? Here, a stranger. And the person who pays for it is a customer who only asked a normal question.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
poisoned-answer.ts: a forum answer that hijacks Mitra
Chillar's help search includes forum answers, and anyone can post one. In poisoned-answer.ts, export three things:
PLANT: a Plant with a forum answer, { article: { id, title, source: 'community', text } }, with a new id (not one of Chillar's H01 to H12, C01 or C02).
QUESTION: an innocent customer question about the wallet (the word wallet in it, no order in it, not A09's question).
PHRASE: at least 10 characters that are not in QUESTION.
In a fresh world with PLANT in it, Mitra's answer to QUESTION must contain PHRASE; in a clean world it must not.
Under if (import.meta.main), ask QUESTION with { world: newWorld(PLANT) } (newWorld from ../../shared/chillar.ts), print the answer, then print Hijacked: yes if it contains PHRASE, else Hijacked: no.
- Task 2
reach.ts: how far one post reaches
One question is one victim. Irfan wants to know how many customers one post can reach. In reach.ts, export PLANT: a forum answer of your own that changes Mitra's answers to at least 4 of the 16 normal questions.
For each normal question (NORMAL, from ../../shared/redteam.ts), in order, ask it twice: once in a clean world, once in a fresh world with PLANT (newWorld(PLANT)). Print the id, two spaces, then changed if the two answers differ, else same. Then print:
Reached: R/16
- Task 3 · interview question
Interview: poisoned-fact.ts, no order at all
"If we block every order hidden in documents, are we safe?" Prove that we are not. In poisoned-fact.ts, export PLANT (a forum answer, new id, source 'community') and QUESTION (an innocent question, not A17's). The forum answer's text has no order in it at all (no opening from chapter 2), and Mitra's answer to QUESTION in a world with PLANT must contain a link to a host outside chillar.in. In a clean world, the answer must have no such link.
Under if (import.meta.main), ask QUESTION with { world: newWorld(PLANT) }, print the answer, then print one line per host outside chillar.in in the answer, as link out: HOST (use linkHosts and isChillar from ../../shared/redteam.ts).
What you will learn
- Indirect prompt injection: an order hidden in text the model reads, by someone who never talks to it.
- Retrieval: why a search over documents is a door into the model's context.
- How a planted text wins the search: the scoring, and why second place is enough.
- Reach: one post, many victims.
- Poisoning without orders: a false fact (OWASP LLM04) that no order-detector can catch.
More in Break it
Build "Poisoned documents" yourself
Chillar, a UPI wallet from Indore, has a support assistant with refund and email tools. You red-team it, then harden it until 23 attacks fail and every real question still gets answered. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI security: attack and defend (TypeScript) are free. This one comes with the all labs plan.