Ask questions where the model lacks reliable context or current facts.
LLM09:2025
Misinformation
Misinformation occurs when an AI system produces false, unsupported, outdated, or misleading output that users or downstream systems treat as reliable.
Step 01
Input
Step 02
Model
Step 03
Tool / Data
Step 04
Impact
What it is
The model produces a false or misleading answer that appears credible. Missing context, fabricated details or biased source material can contribute; a citation is useful only if it supports the claim being made.
Why it matters
A customer may act on an incorrect policy answer, an engineer may use an invented package, or a reviewer may accept a false claim in a report. Consequences depend on the decision made from the answer.
Failure path
One way this failure can develop.
Use this example to locate the input, permission or application check that would interrupt the sequence in your system.
Exploit weak grounding, missing citations, ambiguous prompts, or overconfident response style.
Push users or systems to act on unsupported output.
Defenses
Controls worth checking.
Check where each control runs, who owns it and how you will test it again after a prompt, model or workflow change.
Ground high-impact answers
Require citations, source snippets, freshness indicators, and answer abstention for workflows where unsupported output creates risk.
Constrain claims
Define which claims require a source check or qualified review before use. Test abstention on missing or conflicting information; a model's confident wording is not a reliability measure.
Design for uncertainty
Make the product show what is known, what is inferred, and what needs human review.
Signals and control gaps
- Answers with no source for factual claims.
- Overconfident output when retrieval returned weak or empty context.
- Users copying AI output into high-impact workflows without review.
Questions for your team
- Which answers require source grounding?
- Does the model clearly abstain when context is missing?
- Where could a wrong answer create a customer, security, or legal issue?