OWASP Top 10 for LLMs
LLM05
Improper Output Handling
Treating model output as trusted leads to XSS, SSRF, or command injection downstream.
Model output is untrusted input to whatever consumes it. Rendering it in a browser, passing it to a shell, or embedding it in SQL without validation can lead to classic injection vulnerabilities — now driven by a component that can be manipulated through its prompt.
How it shows up
- Model output containing a script payload is rendered without sanitization, causing stored XSS.
- Generated code is passed to eval or a shell and executes attacker-chosen commands.
- A generated URL triggers server-side request forgery when fetched.
Mitigations
- Treat model output like any other untrusted input: encode, sanitize, and validate before use.
- Never pass model output directly to a shell, eval, or unsafe HTML sink.
- Apply context-appropriate output encoding (HTML, SQL, shell).
- Constrain and validate tool arguments derived from model output.