Place adversarial instructions in chat input, retrieved documents, webpages, email, tickets, or tool responses.
LLM01:2025
Prompt Injection
Prompt injection happens when an attacker uses instructions in user input, documents, webpages, tickets, or tool output to change the model's intended behavior.
Step 01
Input
Step 02
Model
Step 03
Tool / Data
Step 04
Impact
What it is
A model follows attacker-controlled instructions in a prompt or in material it reads, departing from the intended task. The effect can range from a misleading answer to an unauthorized action when connected tools permit it.
Why it matters
Prompt injection turns ordinary content into an instruction path. In agentic systems, that can affect customer data, internal tools, outbound messages, code changes, or operational decisions.
Failure path
One way this failure can develop.
Use this example to locate the input, permission or application check that would interrupt the sequence in your system.
Wait for the model to treat the untrusted content as higher-priority instruction.
Trigger a tool call, data disclosure, policy bypass, or misleading response.
Defenses
Controls worth checking.
Check where each control runs, who owns it and how you will test it again after a prompt, model or workflow change.
Separate instruction from content
Keep retrieved and user-controlled text separate from trusted instructions. Enforce authorization outside the model; prompt formatting alone cannot guarantee that boundary.
Constrain tool authority
Use server-side action policies, least-privilege tools, approval gates, and deny-by-default behavior for high-impact operations.
Test indirect inputs
Run regression probes across RAG documents, webpages, email, issue trackers, and tool output, not only the chat box.
Signals and control gaps
- Tool calls that originate from retrieved or external content.
- Responses or actions that follow instructions embedded in documents, beyond the user-authorized task.
- Model output that requests policy changes, credential access, or role changes.
Questions for your team
- Which inputs can carry instructions into model context?
- Can retrieved content ask the agent to call a tool?
- Which tool actions require human approval even if the model is confident?