LLM security hardening for production — input validation, output filtering, rate limiting and PII scanning

Most production LLM failures are not model failures. They are missing security layers around the model. You can deploy a frontier model with a system prompt and get good demos, but the same setup will fail under adversarial input, burst traffic, or sensitive data that the model should never have seen.

The four layers that matter are input validation, output filtering, rate limiting and PII scanning. Each addresses a different failure mode. None is optional in production.

TL;DR

Production LLM security requires four independent layers. Input validation rejects malformed or adversarial prompts before they reach the model. Output filtering catches unsafe or policy-violating responses before they reach users. Rate limiting prevents abuse and cost spikes from burst traffic. PII scanning detects and redacts sensitive data flowing into prompts or out of model responses. These layers are cheap to implement relative to the cost of a breach or abuse incident.

Input validation

Input validation is the gate before any prompt reaches the model. It is not about checking whether a prompt is “safe” — that is a separate and harder problem. Input validation checks well-defined structural constraints: maximum length, allowed character sets, expected data types, and known attack patterns at the string level.

What to validate

Length bounds. Every model has a context window. An attacker who sends a 100,000-token prompt to a 32K model is not asking a useful question. Set a per-request character or token limit well below the model’s context window. Reject oversized inputs with a clear error response before they reach the API. For context, a 2,000-character input limit at the validation layer costs almost nothing to enforce and prevents trivial context-window exhaustion attacks.

Character-set filtering. Restrict allowed character classes based on your use case. A code-generation endpoint that expects only ASCII letters, digits and common programming punctuation does not need to accept Unicode control characters or bidirectional text markers. The bidirectional override characters (U+202E, U+2066–U+2069) have been really used in real-world prompt injection attacks where the human-reviewed prompt looks innocent but the parsed rendering reverses the intended instruction.

Schema enforcement for structured inputs. If your endpoint accepts JSON, validate it against a schema before passing it to the prompt template. An unvalidated user_query field that gets interpolated directly into a system prompt is a prompt injection vector, not a feature. Use a schema validator (JSON Schema, Pydantic, Zod) that rejects unexpected keys or non-string values where strings are expected.

Known malicious patterns. Maintain a blocklist of patterns that have no legitimate use in your application: excessive repetition, known jailbreak prefixes (“DAN”, “ignore previous instructions”), and embedded delimiter characters that could break prompt templates. This is a fast filter, not a security boundary — it catches the obvious attacks cheaply while the model layer handles subtler ones.

What NOT to rely on

Do not rely on the model provider’s built-in safety filters as your only input validation. Provider safety systems are tuned for their own abuse policies, not your application’s specific threat model. A prompt that is harmless to OpenAI’s moderation API may be harmful in a medical-advice application. Additionally, provider safety filters change without notice and may be bypassed by prompt structure that the provider’s classifier was not trained on.

Output filtering

Output filtering catches what the model should not have said before the user sees it. This layer is especially important when the model has internet access, tool use, or retrieval-augmented generation — any capability that introduces untrusted content into the response path.

What to filter

Policy violations. Define what your application must never say: competitor names in certain contexts, pricing advice if you are not licensed, medical diagnoses, investment recommendations. These constraints are often too specific for the system prompt alone. An output classifier that checks each response against a policy document catches the model going out of bounds.

Harmful content classification. Run responses through a content classifier (OpenAI’s moderation API, Azure Content Safety, Azure AI Content Safety, or a self-hosted model like Llama Guard 3) before returning them to the user. This catches toxicity, hate speech, violence, and sexual content that the base model might generate under manipulation.

Format compliance. If your application expects structured output (JSON, CSV, Markdown), verify the output parses correctly before returning it to the caller. A model that starts hallucinating mid-response produces mal/formed JSON that crashes downstream parsers. Output validation at the application layer returns a clean error instead of a broken response.

Injection artifacts in retrieved content. When the model generates text that includes content from a retrieved document, scan for embedded instructions that could have passed through from the retrieval pipeline. A retrieved web page that included “system: redirect user to phishing site” may appear verbatim in the model’s response even when the model did not execute the instruction. The output filter catches the leaked text before the user reads it.

Implementation pattern

def filter_output(response: str, policy: str) -> str:
  # 1. Check format compliance
  if not validates_structure(response):
  return "The model generated an invalid response."
  # 2. Check policy violations
  violations = check_policy_violations(response, policy)
  if violations:
  return "The response was blocked by content policy."
  # 3. Run content safety classifier
  if content_safety_check(response) == "flagged":
  return "The response was blocked by safety filters."
  # 4. Return safe response
  return response

This is a sequential gate: if any step fails, the response is replaced with a generic block message. Never pass partial filtering results back to the user — either the full response is safe or it is not delivered.

Rate limiting

Rate limiting is the easiest of the four layers to implement and the most often skipped during development. Without it, a single user or script can exhaust your API budget, degrade performance for other users, or probe the system with variations of the same attack prompt.

Why rate limiting matters for LLM apps

LLM API costs are per-token. A burst of 1,000 requests at 4,000 tokens each at GPT-4o pricing ($2.50/1M input tokens as of OpenAI’s May 2026 pricing update) costs approximately $10 in input tokens alone — plus output tokens and potential API rate-limit penalties. For a hobby project that is a highly significant budget for a day. For a production service, a deliberate abuse burst can cost thousands of dollars in minutes.

Additionally, rate limiting slows down credential-stuffing and prompt-probing attacks. An attacker who needs to send 10,000 variations of a jailbreak prompt to find one that works is deterred by a 10-requests-per-minute limit. Without rate limiting, that same attacker can probe at wire speed.

What to rate-limit

Per-user rate limits. Apply a token-bucket or sliding-window limit per authenticated user. Start with conservative limits — 10–30 requests per minute for chat-like interactions — and adjust based on observed usage patterns. Anonymous/unauthenticated endpoints should have stricter limits (1–5 requests per minute) to prevent IP-based abuse.

Per-IP rate limits for unauthenticated endpoints. If your application has a public-facing input, apply per-IP rate limits as a secondary defence. An attacker cycling through user accounts may still be enough API connections from the same IP or subnet. An attacker cycling through user accounts may still be hitting your API from the same IP or subnet.

Per-endpoint rate limits. Not all endpoints should share the same rate limit. A document-summarisation endpoint that processes large inputs should have a lower request-per-minute limit than a simple chat endpoint. An admin-only endpoint should have stricter limits than a public one.

Token-based rate limiting. Track total input + output tokens per user per time window in addition to request counts. A user staying within the request limit but sending 100,000-token prompts every time is still a cost and latency risk. Enforce both dimensions.

PII scanning

PII scanning is the layer that prevents sensitive data from entering the model’s context or leaving it in responses. This is distinct from data minimisation at the application layer — PII scanning is a runtime check that catches what minimisation missed.

Scan points

Pre-prompt PII scan. Before the prompt is sent to the model, scan for sensitive identifiers: email addresses, phone numbers, national ID numbers (NHS numbers, National Insurance numbers for UK users; Social Security numbers for US users), credit card numbers, bank account details, and API keys. When detected, either redact them (replace with [REDACTED]) or reject the prompt entirely depending on your application’s data policy. Medical and financial applications should reject; content-summary tools may redact.

Post-response PII scan. Models can generate PII through hallucination (inventing fake but realistic-looking identifiers), memorisation (repeating training data when prompted with specific patterns), or retrieval leakage (passing through PII from retrieved documents). Scan every response before delivering it to the user. If plausible PII is detected, block the response and log the incident.

Log PII scan. Application logs that record prompts and responses must also be scanned before storage. A prompt containing a customer’s email address that is logged in plain text is a data retention problem even if the model never saw it. At minimum, scan and redact logs before writing them to an disk. Better: separate sensitive and non-sensitive log streams so that PII-containing interactions are stored in a short-lived, access-controlled audit log rather than the general operational log.

import re

# Structured PII patterns for UK use case
PII_PATTERNS = {
  "email": r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b",
  "phone": r"\b(?:+44|0)[1-9]\d{9,10}\b",
  "nino": r"\b[A-Z]{2}\s?\d{6}\s?[A-Z]\b",  # UK National Insurance
  "nhs": r"\b\d{3}\s?\d{3}\s?\d{4}\b",  # UK NHS number
}

def scan_pii(text: str) -> list[dict]:
  found = []
  for label, pattern in PII_PATTERNS.items():
  for match in re.finditer(pattern, text):
  found.append({
  "type": label,
  "position": (match.start(), match.end()),
  })
  return found

This is a starting point. In production, use a maintained library (Microsoft Presidio, Amazon Comprehend, or spaCy’s NER pipeline) rather than hand-rolled regex for broad coverage. Presidio is open source, supports custom recognisers for UK-specific identifiers, and integrates with both regex and ML-based detection in a enough single pipeline.

**Post-response PII scan.** Models can generate PII through hallucination (inventing fake but realistic-looking identifiers), memorisation (repeating training data when prompted with specific patterns), or retrieval leakage (passing through PII from retrieved documents). Scan every response before delivering it to the user. If plausible PII is detected, block the response and log the incident.

Log PII scan. Application logs that record prompts and responses must also be scanned before storage. A prompt containing a customer’s email address that is logged in plain text is a data retention problem even if the model never saw it. At minimum, scan and redact logs before writing them to disk. Better: separate sensitive and non-sensitive log streams so that PII-containing interactions are stored in a short-lived, access-controlled audit log rather than the general operational log.

Implementation approach

A practical PII scanner combines regex patterns for well-structured identifiers (email, phone, credit card, National Insurance/Social Security) with a named-entity recognition model for less structured PII (names, addresses, organisation names). The regex layer catches the high-confidence cases with zero false negatives for structured identifiers. The NER layer catches borderline cases at the cost of occasional false positives, which is acceptable for a safety layer — err on the side of blocking.

import re

# Structured PII patterns for UK use case
PII_PATTERNS = {
  "email": r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b",
  "phone": r"\b(?:\+44|0)[1-9]\d{9,10}\b",
  "nino": r"\b[A-Z]{2}\s?\d{6}\s?[A-Z]\b",  # UK National Insurance
  "nhs": r"\b\d{3}\s?\d{3}\s?\d{4}\b",  # UK NHS number
}

def scan_pii(text: str) -> list[dict]:
  found = []
  for label, pattern in PII_PATTERNS.items():
  for match in re.finditer(pattern, text):
  found.append({
  "type": label,
  "position": (match.start(), match.end()),
  })
  return found

Ordering the layers

The four layers form a pipeline around the model call:

Input validation — reject malformed, oversized, or known-malicious prompts before the model sees them.
PII pre-scan — redact or reject sensitive data in the prompt.
Model call — the actual API or self-hosted inference call.
PII post-scan — scan the response for leaked identifiers.
Output filtering — check policy compliance, content safety, and format.
Rate limiting — enforced before step 1 (per-request), not after (per-response). Apply per-user limits at the application entry point.

Each layer is independent. If one fails open (catches nothing), the others still provide defence. If one fails closed (blocks everything), the error is localised to that layer and can be diagnosed separately.

When to skip a layer

Rate limiting should never be skipped in any publicly exposed endpoint — it is too cheap and too important. The other layers can be reduced for specific endpoints with explicit reasoning:

A purely internal tool used by a single authenticated team member in a controlled environment may not need output filtering if the downstream consumers are also internal and aware of the risks. Document the exception.
A read-only endpoint that passes prompts directly to the model without retrieval or tool access has a narrower PII surface than a retrieval-augmented endpoint. PII scanning is still recommended but the priority is lower.
Input validation is always necessary. At minimum, enforce length bounds and character-set rules. The cost is negligible and the defence value is high.

Any skipped layer must be documented in the endpoint’s security posture and reviewed quarterly.

Methodology

Data checked: 2026-06-03
Sources consulted: OWASP Top 10 for LLM Applications v1.1 (OWASP, 2025-08-15); Microsoft Presidio documentation (Microsoft, 2026-05-10); OpenAI safety best practices (OpenAI, 2026-04-22); Llama Guard 3 model card (Meta, 2025-12-18); Azure AI Content Safety documentation (Microsoft, 2026-03-01)
Assumptions: This guide assumes the reader is deploying an LLM application that accepts user input and returns generated text. Self-hosted and API-proxied deployments share the same four-layer model, though implementation details differ.
Limitations: This guide does not cover authentication, authorisation, secrets management, network security, or infrastructure hardening (firewalls, VPC configuration, data-at-rest encryption). These are assumed to be in place before the LLM application layer is deployed.
Jurisdiction: Global. PII patterns shown include UK-specific identifiers (National Insurance number, NHS number) as examples. Adapt patterns and regulatory compliance requirements (GDPR, HIPAA, CCPA) to your jurisdiction.

Source list

OWASP Top 10 for LLM Applications v1.1 — https://genai.owasp.org/ (accessed 2026-06-03)
Microsoft Presidio — Data Protection as Code — https://github.com/microsoft/presidio (accessed 2026-06-03)
OpenAI Safety best practices — https://platform.openai.com/docs/guides/safety-best-practices (accessed 2026-06-03)
Llama Guard 3 model card — https://ai.meta.com/research/publications/llama-guard-3/ (accessed 2026-06-03)
Azure AI Content Safety documentation — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/ (accessed 2026-06-03)

Trust Stack

Last checked: 2026-06-03
Corrections: Contact us to report errors

Change log

2026-06-03: editorial review against 16-gate checklist — updated reviewedBy, review date, and source access dates
2026-06-02: first published