## Where teams misuse it - **Sending the full customer record when only a field is needed.** A billing chatbot that answers "when is my next payment?" does not need the customer's name and address — it only needs the payment schedule. Teams dump the entire customer object into the prompt because it is available, not because it is necessary. - **Building redaction after the fact instead of classification up front.** A team builds a regex-based redaction pipeline that strips names and emails from the prompt before sending it to the API. But the prompt still contains account numbers, transaction IDs, support ticket text, and internal notes. The redaction pipeline was built without classifying what PII types actually exist in the data. - **Treating "anonymised" output as retroactively compliant.** A model generates a response that includes a customer's full name because the name was in the retrieved context and the model used it. Even if the input was minimised, the output leaked it. Output-layer minimisation — telling the model not to include names in its response, or validating the output — is a separate step. - **Skipping PII classification for retrieval content.** Teams that build RAG systems often classify PII in the user prompt (and redact it), but forget that the retrieved documents may also contain PII. A vector database returns whatever chunks were stored, including customer names, account numbers, or health data that was embedded without classification. ### Real scenario A team builds a customer-support chatbot for an e-commerce platform. The product database has fields for: name, shipping address, email, phone, order history, payment method, account tier, support notes. The team writes a system prompt: "You are a helpful support agent. Use the customer profile to answer questions." They pass the full customer object as context. A user asks: "When is my next delivery?" The model replies with the delivery date — correct — but also includes: "I see you live at 123 Oak Street and your Visa card ending in 4242 was charged for this order." The model was not injected or jailbroken. It simply received a customer profile with more information than the question needed, and it generated a helpful response that accidentally disclosed PII. The fix was not a better redaction regex — it was a data-classification schema that only sent shipping-related fields for delivery questions. ## Practical decision check Before connecting a customer-facing LLM feature to production data, ask: - **Which fields does the model genuinely need to answer this specific task?** Not "which fields are available" — which fields are necessary. Define the minimum viable data schema for each workflow. - **What PII categories exist in your data?** Names, emails, addresses, phone numbers, account numbers, payment details, health information, internal notes — classify every field that could reach the model. - **Is PII classification applied to your retrieval content, not just your prompts?** If RAG sources include customer records, support tickets, or product databases, classify what those sources contain. - **Does the output validator check for re-identified PII?** A prompt that was successfully stripped of names may still generate a response that includes the customer's name from context. Validate the output for identifiers that were not in the sanitised input. - **What changes when the data-processing purpose changes?** If a "billing AI" is later used for "personalised recommendations", the minimum viable data changes. Redo the minimisation analysis. ## Related guides - [Data leakage in LLM apps: logs, prompts, files and vendor retention](/run/data-leakage-in-llm-apps-logs-prompts-files-and-vendor-retention/)
- [Provider data retention policies: what API users should compare](/diff/provider-data-retention-policies-what-api-users-should-compare/)
- [AI output monitoring: what to log, sample, and review](/run/ai-output-monitoring-what-to-log-sample-and-review/) ## Methodology - **Data checked:** 2026-05-28
- **Sources consulted:** ICO UK GDPR guidance, NIST AI RMF, OpenAI data usage and retention documentation, CISA and OWASP privacy and security guidance
- **Assumptions:** This guide assumes familiarity with standard API prompt construction and RAG retrieval pipelines. It does not cover legal compliance frameworks or certified privacy certifications.
- **Limitations:** This article is operational guidance, not legal advice. Privacy obligations depend on your jurisdiction, data types, processing purpose, vendor terms, and retention model. Consult your DPO or legal counsel for compliance decisions. The PII classification approach described here is a design pattern, not a certified compliance framework.
- **Jurisdiction:** Global. UK GDPR and ICO guidance are referenced as examples; equivalent frameworks (GDPR, CCPA, PIPADA) apply in other jurisdictions. ## Source list - ICO UK GDPR guidance — https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/ (accessed 2026-05-28)
- NIST AI RMF — https://www.nist.gov/itl/ai-risk-management-framework (accessed 2026-05-28)
- OpenAI data usage and retention documentation — https://platform.openai.com/docs/guides/your-data (accessed 2026-05-28)
- OWASP Top 10 for LLM Applications — https://owasp.org/www-project-top-10-for-large-language-model-applications/ (accessed 2026-05-28) ## Trust Stack - **Last checked:** 2026-05-28
- **Corrections:** [Contact us](/contact/) to report errors ## Change log - **2026-05-28:** Full editorial review against 16-gate checklist. Added 3 Editor's Note aside cards, slugified all H2/H3 IDs, added Trust Stack section with corrections policy and affiliation, standardised Methodology to canonical format, converted Source and evidence notes to proper Source List with access dates, removed workflow leaks (brief references, internal-link suggestions section), fixed frontmatter writtenBy label.
- **2026-05-27:** Added direct source URLs to all named providers and services; added Change Log section.
- **2026-05-24:** First published.
