Detecting Secrets and PII in LLM Prompts and Logs

The fastest way to leak a credential in 2024 isn't a misconfigured S3 bucket — it's a user pasting OPENAI_API_KEY=sk-... into a chat box. That text then flows to your model provider, into your logs, into your observability tooling, and sometimes into a vector store. Every hop is a place it can leak.

This post covers how to catch secrets and PII before they spread, and how to redact them without breaking the request.

Where sensitive data leaks

• To the model provider. Anything in the prompt is sent to a third party.
• Into logs and traces. Debug logging of prompts is the classic offender.
• Into storage. Chat history, caches, and vector databases persist it.
• Into other users' context. In multi-tenant RAG, one user's leaked data can surface in another's retrieval.

What to detect

A practical detector should cover two families:

Secrets — high-severity, block on sight:

• Provider API keys (OpenAI, AWS, Google, Stripe, Slack, GitHub)
• JWTs and private-key headers
• Credentials embedded in URLs or KEY=value assignments

PII — redact and flag:

• Emails and phone numbers
• Credit-card numbers (validate with the Luhn check to cut false positives)
• National IDs (e.g. US SSNs)
• IP addresses

You can try this live in the PII & Secrets Scanner — paste a sample and watch it classify and redact.

Detect, then redact

Detection alone isn't enough; you usually still want to process the request. Replace matched values with typed placeholders so the model keeps the shape of the input without the sensitive content:

Here is the key OPENAI_API_KEY=[REDACTED:SECRET] — summarize the docs.
Contact the user at [REDACTED:PII] about the renewal.

The Prompt Firewall does exactly this: secrets block the request, PII is redacted and flagged, and you get back a sanitized string safe to forward:

import { evaluateFirewall } from "@opensecureai/firewall";

const { decision, sanitized, reasons } = evaluateFirewall(userInput);
if (decision === "block") return reject(reasons); // e.g. leaked secret
forwardToModel(sanitized); // PII already redacted

Don't log the raw input

A detector is pointless if you log the prompt before running it. Two rules:

1. Run detection/redaction first, then log only the sanitized version.
2. Store metadata (which rule fired, severity, counts) — never the raw match.

That's the approach behind LLM Observability: it records that a secret was detected and blocked, not the secret itself.

Handle the false-positive tradeoff

Pattern matching will occasionally over- or under-fire. Tune for your context:

• Block on secrets aggressively — a false positive is cheap, a leaked key is not.
• Treat PII as flag-and-redact rather than hard block, so legitimate requests still work.
• Validate structured formats (Luhn for cards, key prefixes for tokens) to keep precision high.

Takeaways

• Users will paste secrets and PII into prompts — assume it.
• Detect and redact before the data hits your provider, logs, or storage.
• Block on secrets, redact-and-flag on PII, and never log the raw input.

For a deeper checklist across the whole app, see the LLM Security Checklist.