OWASP Top 10 for LLMs
LLM04
Data and Model Poisoning
Attackers corrupt training, fine-tuning, or retrieval data to plant backdoors or bias.
Poisoning manipulates the data a model learns from or retrieves. In training and fine-tuning it can plant trigger phrases or skew behavior; in RAG it can steer retrieval toward attacker-chosen content. Because the effect is baked into weights or the index, it is hard to detect after the fact.
How it shows up
- Crafted samples in a crowd-sourced dataset create a trigger phrase that flips classifier output.
- Documents engineered to rank highly poison a vector store used for retrieval.
- A fine-tuning run inherits bias from an unvetted data source.
Mitigations
- Validate and curate data sources; track provenance and integrity.
- Monitor for anomalous trigger phrases and behavioral drift.
- Control write access to vector stores and retrieval corpora.
- Use robust training techniques and hold-out evaluations to detect poisoning.