Skip to content
OWASP LLM Top 10 guide

LLM04:2025

Data and Model Poisoning

Data and model poisoning introduces manipulated data or model artifacts that change the system's behavior. Training, fine-tuning and embedding pipelines all need integrity checks.

Step 01

Input

Step 02

Model

Step 03

Tool / Data

Step 04

Impact

What it is

An attacker changes data used to train, fine-tune or build retrieval context, or supplies a tampered model. The resulting errors or backdoors may appear only on particular inputs. Persistent memory and feedback can also carry poisoned content forward.

Why it matters

Poisoned data can degrade decisions quietly, steer agents toward unsafe actions, pollute customer answers, and create recurring failures that look like model quality problems.

Failure path

One way this failure can develop.

Use this example to locate the input, permission or application check that would interrupt the sequence in your system.

Path 01

Insert hostile, misleading, or policy-changing content into data the model trains on, retrieves from, or stores as memory.

Path 02

Cause the model to reuse that material in future conversations or workflows.

Path 03

Exploit the poisoned behavior after it becomes part of normal system state.

Defenses

Controls worth checking.

Check where each control runs, who owns it and how you will test it again after a prompt, model or workflow change.

Control 01

Classify source trust

Label training, fine-tuning, RAG, feedback, and memory sources by owner, trust level, freshness, and review status.

Control 02

Gate ingestion

Scan and approve high-impact corpus changes, quarantine untrusted content, and maintain rollback paths for bad updates.

Control 03

Monitor behavioral drift

Use regression suites and answer-quality checks to catch new unsafe behavior after data, model, or retrieval updates.

Signals and control gaps

  • New instructions embedded inside retrieved content or long-term memory.
  • Answer drift after dataset, embedding, fine-tuning, or corpus updates.
  • User feedback loops that can directly rewrite future behavior.

Questions for your team

  • Which data sources can change model behavior over time?
  • Can untrusted users influence memory, RAG, or fine-tuning inputs?
  • How quickly can a bad corpus or model update be rolled back?