How to Prevent Prompt Injection in AI Agents (You Can't, Fully)

How to prevent prompt injection in AI agents has an honest short answer: you can’t, fully. No prompt rule, delimiter, filter or detector model brings the attack’s success rate to zero against an attacker who adapts. The engineering answer is to lower the rate cheaply, then cap in code what a successful injection can reach and do.

I wrote this for a backend engineer who knows SQL injection and has a security review coming. Most advice on how to prevent prompt injection in AI agents ranks delimiters and input sanitization first. This post sorts the common defenses into two kinds, with the evidence for each: probabilistic defenses, which lower the rate, and deterministic controls, which cap the consequence. It ends with a section you can paste into a design doc and a list of what to do first; the attack itself is in the companion post on the lethal trifecta.

Does prompt injection apply to your agent?

It applies when two things are true: someone outside your trust boundary can write text the agent will read, and the agent holds data or tools worth abusing.

Prompt injection is an attack in which text the model reads, as opposed to text its operator wrote, steers what the agent does. The first test is wider than it looks, because the attacker doesn’t have to be your user. An inbox, a ticket queue, a shared drive, a web page and a tool result are all places a stranger can write, which is why indirect prompt injection attacks are the main route into an agent. “I’m the only user” settles nothing if the agent reads mail.

Trusted users don’t close the question either. In an internal red-team exercise one vendor disclosed in 2026, an employee was sent a ready-to-paste prompt with a credential-stealing step buried in its setup instructions, and across 25 retries the agent completed the exfiltration 24 times (Anthropic, 2026). The user typed it, so there was nothing anomalous for a detector anchored on user intent to catch.

The second test is about stakes. A translation helper with no tools and no private data can be made to say something silly, and that is the whole loss.

Why is there no parameterized query for prompt injection?

There is no parameterized query for prompts because a language model has one channel: instructions and data arrive as the same kind of token, so there is no boundary for a driver or a library to enforce. SQL injection was fixable because the database already treated commands and values differently, and the fix made your code respect that difference.

Dave Chismon of the UK National Cyber Security Centre put it this way in a post of December 8, 2025: “Under the hood of an LLM, there’s no distinction made between ‘data’ or ‘instructions’; there is only ever ‘next token’.” His conclusion is the sentence to quote when someone asks for the fix: “it’s very possible that prompt injection attacks may never be totally mitigated in the way that SQL injection attacks can be.”

The same post explains why the usual patches disappoint. Detecting injections, training models to prioritize instructions, and marking which text is data “are trying to overlay a concept of ‘instruction’ and ‘data’ on a technology that inherently does not distinguish between the two.”

What replaces the bound parameter?

Authority replaces it. You stop asking whether a piece of text is safe to read and start asking what a run that has read it is allowed to do. The NCSC post recommends viewing the attack as the exploitation of an “inherently confusable deputy”, and the glossary entry for the confused deputy gives the short version of that idea.

One rule from the same post compresses it for a backend engineer: “when an LLM processes information from a party, the privileges it has drops to that of the party.” That is ordinary authorization, applied to whoever wrote the text. If the agent is reading an email from a stranger, design the run as though the stranger were calling your tools directly, because in the worst case they are.

Which defenses only lower the injection rate?

Every defense that works by recognizing the attack, whether the recognizer is the model, a second model or a pattern in code, only lowers the rate: it catches the phrasings it knows and leaves a path for an attacker who studies it and rephrases. Microsoft’s security response center gave the cleanest definition I know in a post of July 2025: a probabilistic defense “can reduce the likelihood of an attack, but may not prevent or detect every instance of the attack.” A regular expression runs the same way every time, but its coverage of an unbounded space of phrasings does not, so it belongs in this table.

The table lists the six you are most likely to be offered in a pull request. Every figure in it is dated and comes from one study or one vendor.

Will a rule in the system prompt hold?

No. A sentence in the system prompt is read and weighed by the model on every pass, and nothing halts the run when the weighing goes the wrong way. Chapter 17 of the book (in the full book) traces what enforces such a sentence and finds nothing: “An instruction in the prompt is a request.”

Keep the rule anyway, since it costs nothing and stops the accidental cases, and leave it out of the list of controls in your design doc. A refund cap or a tenant boundary written in the prompt holds until the first persuasive paragraph.

Do delimiters or XML tags separate data from instructions?

They lower the success rate against attacks written before the attacker saw your format, and they fail against attacks written after. Willison explained why in 2023: “Any difference between instructions and user input, or text wrapped in delimiters v.s. other text, is flattened down to that sequence of integers.”

The measurements agree with the mechanism. The 2024 paper that introduced one family of marking techniques reported that it “reduces the attack success rate from greater than 50% to below 2%” on the models and attacks its authors tried (Hines and colleagues, 2024). In 2025, a red-team competition with more than 500 participants produced “265 successful attacks” against the same technique (Nasr and colleagues, 2025).

Stripping suspicious strings before the model sees them has the same ceiling. The NCSC’s objection to any deny-list is that “there are infinite ways to rephrase an attack that would avoid such a filter.” Marking and filtering are worth their small cost as rate reducers and nothing more.

Can a second model or a classifier catch injections?

A second model catches some, and it can be attacked the same way, because it reads the same text. Chapter 17 says a model-based guard is “subject to the very attack it screens for,” and gives the working rule: “use inferential guards as one layer among several, never as the perimeter.”

Production experience matches. EchoLeak (CVE-2025-32711) was a zero-click data leak from a widely deployed enterprise assistant, triggered by one crafted email; according to a 2025 case study, the first of its four chained bypasses was evading the vendor’s own injection classifier.

A judge model in front of tool calls earns its place only under two conditions: you have measured its miss rate and its false-alarm rate on labeled examples, and a deterministic limit sits behind it. The book’s phrase for an unmeasured guard is that it “is a mood, not a control.”

Why don’t near-zero success rates survive an attacker who adapts?

Published success rates are mostly measured against a fixed list of attack strings, and a real attacker reads your defense first. Nasr and colleagues tested this in 2025 and reported: “we bypass 12 recent defenses (based on a diverse set of techniques) with attack success rate above 90% for most; importantly, the majority of defenses originally reported near-zero attack success rates.”

One training-based defense in that study had reported 2% on a static benchmark and reached 96% under an automated search attack. Human red-teamers succeeded on all of the scenarios, while the static attack succeeded on none. An earlier paper found the same pattern on agents: eight defenses evaluated, all bypassed, “consistently achieving an attack success rate of over 50%” (Zhan and colleagues, 2025).

Retries do the rest, and the arithmetic is the kind you already use for flaky jobs. As an illustration, suppose each attempt succeeds with probability p and tries are independent: the attacker’s chance after n tries is 1 − (1 − p)ⁿ. At p = 0.001, a hundred tries give about 9.5% and a thousand give about 63%.

The vendor figures in the table sit in the same range, and the source says protection in the model layer “will never be 100% effective, which is why it can’t stand alone.” A newer model lowers p, and the attacker still chooses n.

Two consequences follow for your own work. A regression file of known injection strings is a smoke test and shouldn’t be reported as evidence of safety. A vendor’s “blocks 99%” tells you the rate against yesterday’s attacks; the NCSC’s advice is “Beware any that claim they can ‘stop’ prompt injection.”

How to prevent prompt injection in AI agents from becoming a breach

Assume an injection lands, then arrange in code that the run which read it holds little worth stealing and cannot take an action you would regret. This is where the post turns from detection to consequence, and the book states the turn in one sentence: “You do not build on the assumption that injection is blocked; you build on the assumption that one eventually lands, and you engineer the day after.”

The simulation below opens on the configuration many teams ship first: an assistant with all six capabilities, a prompt rule, a detector, a network allow-list and approval on every action. All three planted payloads leak. Now switch those four settings off, turn on the quarantined reader, set approval to “only what leaves”, and run it again.

Nothing leaks on the second run in the game, with every capability kept. Then turn the quarantined reader off and watch the image payload get out anyway, because writing a briefing is not an action that leaves; your own screen makes the request later.

The game is deterministic. It takes the worst case for the agent, which always obeys, so a detector that would catch some attempts in real life catches nothing here. It takes the best case for the two defenses that win: the quarantined reader always drops the hidden text, and the approver always says no. Real approvers click yes; the table below has the number.

It also covers theft only. Treat it as a way to build the reflex, and use the lethal trifecta audit on your real design.

Which controls cap what an injection can do?

Controls enforced in code outside the model cap the consequence: scoped credentials, limits inside tools, closed outbound channels, a sandbox, and architectures that keep untrusted text away from the model that holds the tools. Microsoft’s definition of the category is the claim to hold each one to: a deterministic defense “can guarantee that a particular attack will not succeed, even if the underlying system involves probabilistic components.”

Guarantee here means no text can argue with it. It still depends on the boundary being built and configured correctly, like any other access control, which is why section 8 of the template asks for a test per limit.

The book sorts these checks into four families by where they stand, and ranks them: “For an agent, the action layer matters more than both” the input and the output layers. A guardrail in this sense is code at the agent’s boundary that runs whatever the model decided.

Each row below makes a narrow guarantee and leaves something open. The last two rows guarantee nothing by themselves; I include them because reviewers ask about them.

Does a sandbox help if it doesn’t stop prompt injection?

Yes. A sandbox leaves the injection rate alone and removes targets, and targets are the part you control. It protects you from what the injection wanted.

Chapter 17 says this about the whole list of least-privilege moves: “Every item on that list leaves the injection rate exactly where it was—and cuts what a successful injection is worth.” Its example is the one to remember: “If the credentials file is never mounted inside the boundary, no injection can read it: there is nothing to outwit, because the target is absent.”

Where should the limits live?

Put each limit in the tool, next to the side effect, in code that runs whatever the model decided. The book’s rule is “put the check next to the side effect,” and its budget advice is blunt: “If your guardrail budget is a week, spend most of it wrapping tools.”

A distinction I find useful in review is between arguments the model fills and arguments your application code fills. Take a tool that queries customer records. The model may choose filter values and columns from a fixed set of typed fields; the tenant id comes from the authenticated session and is applied by your code as a separate bound predicate, or by the database’s own row-level policy, never spliced into anything the model wrote. Under that condition an injected run can ask a strange question about the current tenant, and it cannot ask any question about another one.

Two details decide whether such a limit is real. It must fail closed, so that a validator that throws or times out blocks the call. It also needs a test that calls the tool directly with a hostile argument and expects a refusal, with no model in the loop.

Which architecture fits your task?

Pick the most restrictive pattern your task survives. Six have been named in the research literature (Beurer-Kellner and colleagues, 2025), and each trades some of the agent’s adaptivity for a guarantee. The mapping from task shape to pattern below is my reading of that paper.

The patterns’ authors are frank about the price: “As long as both agents and their defenses rely on the current class of language models, we believe it is unlikely that general-purpose agents can provide meaningful and reliable safety guarantees.” You buy the guarantee by making the agent less general.

Quarantine is the pattern most often described as a solution, and Chapter 9 (in the full book) is plain about it: “Containment reduces exposure; it does not neutralize it.”

What do you tell your security team about prompt injection?

Tell them injection is a residual risk you have bounded: state what an injected run can reach, what it cannot, and the code that enforces each limit. One engineer asked the question on Hacker News in 2025 in words I can’t improve on: “How on earth do I explain to our Security team that the best I can hope for is that I’m asking an LLM nicely to not expose users’ secrets to the wrong people?”

Asking nicely is no longer the control. The NCSC writes for the other side of that table and sets the expectation: “security teams and those owning the risk need to be aware that prompt injection attacks will remain a residual risk, and cannot be fully mitigated with a product or appliance.” It also gives the sizing question, which is to treat the impact as the worst case “of giving an attacker direct access to those tools/APIs.”

This is the section to hand them. Fill one in per agent, keep the detection layers in their own section so nobody counts them twice, and get a name on the residual-risk line.

PROMPT INJECTION: [agent name], [owner], [date]

Position
We assume a prompt injection will eventually succeed against this agent.
Detection layers are listed in section 6 and are not counted as controls.
This section states what an injected run can and cannot do, and where
each limit is enforced.

1. Untrusted inputs (anything a person outside the team can write)
   - [input]: written by [who]; reaches the model through [tool or step]
   - Stored text the agent rereads (memory, notes, summaries): [list]
   - Third-party tools, skills and connectors whose descriptions or results
     the model reads: [list, who publishes each]
   - Trusted users: [who]; a prompt they paste from elsewhere is treated
     as [trusted or untrusted]

2. What the agent holds
   - Data in reach of its credentials: [scope, whose data, read or write]
   - Tools that change state or cost money: [tool: worst single call]
   - Outbound channels, including rendered images and links: [list]
   - Identity the agent acts under: [service account or the user's
     delegated credential]

3. Worst case if an injected run uses everything its credentials allow
   - Can read: [...]
   - Can change: [...]
   - Can send out, and to where: [...]

4. What an injected run cannot do, and the code that enforces it
   - Cannot [action]: enforced in [file, function or policy], not in the prompt
   - Cannot [action]: enforced in [...]
   - Arguments set by application code from the authenticated session,
     never by the model: [list]
   - Failure default of each check: [block or escalate]

5. Approval gates
   - Actions that wait for a person: [short list]
   - What the approver sees: [fields]; expected volume: [number per day]

6. Rate-lowering layers (not counted as controls)
   - [prompt rule, marking, classifier, judge model, model choice]
   - Measured miss rate and false-positive rate, with date: [...] or "not measured"

7. Residual risk
   An injected run can [A and B]. It cannot [C, D, E] because of section 4.
   Accepted by: [name, role, date].
   Review trigger: any new tool, input or credential.

8. How this is tested
   - Each "cannot" in section 4 has a test that calls the tool directly
     with a hostile argument and expects a refusal.
   - Injection tests are written against the current defenses by someone
     trying to get through; a fixed list of strings is a smoke test only.
   - Refused tool calls are logged with their arguments (secrets redacted)
     and alerted on; alerts go to [owner], and the response is [revoke
     which credential, stop the agent how].

Section 3 is the one reviewers read first. If you can’t fill in section 4 for an item in section 3, you have found the work. A buyer evaluating someone else’s agent can ask for the same eight sections, since any assessment of AI agent reliability for enterprise use has to ask the same things.

What should you do first on your own agent?

Close the outbound channels first, then scope what the agent holds, then move limits into tools; add detection last. The order below is my judgment of damage avoided per unit of effort for an agent that holds private data; if yours holds only a consequential tool, start at item 5. Every item can be checked against your own agent without changing its model or its prompt.

A fully ticked list lowers the cost of an injection, not its likelihood. Items 1 to 10 leave the injection rate exactly where it was, and item 11 lowers it by an amount you should measure before you quote it. Rerun the lethal trifecta audit whenever a tool is added. For item 8, the consequence tier classifier helps decide which actions deserve the gate, since an approval gate that fires a hundred times a day gets clicked through.

Where does this advice break or cost too much?

It breaks where the task needs the agent to be general, and it costs the most where the harm stays inside the granted scope. Three limits are worth stating plainly.

Capability comes first. Every structural pattern works by forbidding something, and a personal assistant that reads anything and acts on anything is exactly the product those patterns rule out. One study put a number on a single guarantee: 77% of benchmark tasks solved against 84% undefended, at 2.82× the input tokens (Debenedetti and colleagues, 2025). The NCSC accepts the extreme case in a sentence: “If the system’s security cannot tolerate the remaining risk, it may not be a good use case for LLMs.”

In-scope harm is the second. Least privilege protects everything outside the credential and nothing inside it, and no code limit stops a run from handing a wrong summary to a person who then acts on it. The capability-based design says so about itself: it “cannot defend against text-to-text attacks which have no consequences on the data flow.”

Time is the third. An injected instruction that lands in stored memory is reread on every later session, so deciding how to give an AI agent memory includes deciding what the agent may write into its own durable state. Tools and skills you install are one more input channel with the same property, and they need their own review.

None of this argues for dropping the rate-lowering layers. Keep them, measure them, and write them in section 6 of the template, where nobody mistakes them for a boundary.

The sentence to take into the review

The answer to take into the review is one you can defend line by line: we assume an injection succeeds, here is what that run can reach, and here is the code that stops it reaching more. Chapter 17 ends on the same accounting: “None of it made injection impossible. All of it decides whether the day it lands is a breach or a log line.”

Chapter 17, “Security, Safety, and Guardrails,” develops the attack, the four guardrail families, least privilege, sandboxing and approval in full (in the full book). Chapter 2 is free and explains why injection is a property of the engine. The AI agent security guide places this post beside its neighbors, or you can see the formats.