Limit the Damage · bigonotes

You cannot fully stop prompt injection. So ask a different question: what happens when it succeeds? Design so that a successful injection cannot do much.

Least-privilege tools#

Give the model only the tools it needs for the task, each with the minimum permissions required.

An email-reading assistant does not need a tool that sends emails. If the tool is not there, an injection cannot trigger it.

Task	Give it	Leave out
Summarize the inbox	Read emails	Send, delete, forward
Answer questions about orders	Read the signed-in user's orders	Read every customer's orders, issue refunds

Confirm before risky actions#

Any action that sends, deletes, pays or publishes needs human confirmation. Even if an injection convinces the model to call a dangerous tool, a person has to approve it before it runs. That breaks the attack. See Risky actions for the preview pattern.

Treat model output as untrusted#

Never pass model output straight into SQL, a shell, HTML or any other place where it would be executed. The model might have been injected, and its output might be an attack payload.

Output goes into	Risk	Treat it like user input
SQL	SQL injection	Use parameterized queries
A shell	Command injection	Never build commands from it, or allow-list exact values
HTML	Cross-site scripting	Escape it before rendering

Sanitize and validate model output the same way you would treat input from an unknown user.

Defense in depth#

No single layer stops everything. Stacked together, they make the attack surface very small.

Four layers stacked: delimiters, least-privilege tools, human confirmation and untrusted output; an attack that gets through one layer is stopped by the next

Layer	Defense	What it limits
1	Delimiters	How easily injected text is read as instructions
2	Least-privilege tools	What the model can do at all
3	Human confirmation	Whether a risky action actually runs
4	Untrusted output	Whether the model's output can attack your other systems

Short line#

Since injection cannot be fully stopped, limit what a successful one can do. Give the model only the tools and permissions it needs, require human confirmation before anything that sends, deletes, pays or publishes, and treat model output as untrusted input. Together with delimiters, these form defense in depth.

Key takeaways#

Assume injection will sometimes succeed, and design for that.
Least privilege: only the tools the task needs, with minimum permissions. A missing tool cannot be triggered.
Human confirmation before send, delete, pay or publish breaks the attack.
Never pass model output directly into SQL, a shell or HTML. Validate it like user input.
Delimiters, least privilege, confirmation and untrusted output are four layers of defense in depth.

Core idea#

You cannot fully stop injection. Design so a successful one cannot do much.

Four layers of defense in depth#

Layer	Defense	Limits
1	Delimiters	Injected text read as instructions
2	Least-privilege tools	What the model can do at all
3	Human confirmation	Whether a risky action runs
4	Untrusted output	Attacks on your other systems

Key points#

Give only the tools the task needs, with minimum permissions. An email reader gets no send tool.
Confirm before send, delete, pay or publish.
Never pass model output directly into SQL, a shell or HTML.
Use parameterized queries, escape HTML, never build shell commands from output.

Pitfalls#

No single layer stops everything.
Do not trust model output more than user input.

Short line#

Least privilege, human confirmation, untrusted output, plus delimiters: defense in depth.