Influence the model to emit unsafe HTML, code, command text, SQL-like input, tool arguments, or workflow instructions.
LLM05:2025
Improper Output Handling
Improper output handling occurs when an application passes generated content to a browser, shell, database, tool or other component without the checks required by that destination.
Step 01
Input
Step 02
Model
Step 03
Tool / Data
Step 04
Impact
What it is
Generated content is untrusted input to the next component. A valid JSON object can still contain an unauthorized target or unsafe value; syntax checks, output encoding and authorization address different problems.
Why it matters
Bad output handling can turn a model response into code execution, unsafe content, broken workflow state, customer-facing errors, or unauthorized tool actions.
Failure path
One way this failure can develop.
Use this example to locate the input, permission or application check that would interrupt the sequence in your system.
Rely on the application to pass that output into an interpreter, renderer, database, API, or human workflow.
Trigger downstream effects outside the model boundary.
Defenses
Controls worth checking.
Check where each control runs, who owns it and how you will test it again after a prompt, model or workflow change.
Validate structured output
Use strict schemas, allowlists, type checks, and server-side policy checks before using model output as application input.
Encode output for its destination
Use context-appropriate encoding for displayed content, parameterized database queries and constrained execution for generated code. Scanning alone does not make output safe to execute.
Separate advice from execution
Do not let model text directly become tool arguments for sensitive operations without a deterministic validation layer.
Signals and control gaps
- Model output passed directly to APIs, shells, browsers, databases, or template renderers.
- Generated links, HTML, or files displayed without sanitization.
- Tool arguments derived from free-form output instead of strict objects.
Questions for your team
- Where does model output become input to another system?
- Are schemas enforced after generation or only requested in the prompt?
- Can unsafe generated content reach a browser, shell, database, or API?