Data leakage in LLM apps: logs, prompts, files and vendor retention

Most LLM data leakage problems are ordinary product problems with a flashy wrapper. Prompts get logged, files get retained, samples get copied into analytics, and someone assumes “private” is the default because the UI looked calm.

The safest rule is simple: do not send data you do not need to send. Redaction helps, but minimisation beats redaction because it removes whole classes of risk before they reach a vendor, log stream, or support queue.

Retention settings vary by provider, account type, product surface and region. Any article on this topic ages quickly if it does not name the specific service and check date.

Trust stack

AI draft model: gpt-5.4-mini. AI review model: gpt-5.4. Checked against the originating brief and current primary/near-primary sources on 2026-05-24.

Quick answer

What this means

Data leakage in LLM apps is not exotic. It is the same shape as ordinary data leaks: a developer enables debug logging during integration and forgets to disable it before launch; a vendor changes its retention policy in a terms update nobody reads; an analytics library copies full prompt texts into a dashboard that should only get aggregate metadata.

The pattern is almost always the same: data leaves the application through a path the team forgot existed. The model call itself is not the leak — it is the logging wrapper around it, the support ticket that preserves the raw request, the A/B test framework that copies user prompts to a third-party evaluator, or the vendor’s internal training pipeline that claims to be opt-out but defaults to opt-in.

Where teams misuse it

Leaving debug logging in the production model wrapper. Many LLM SDK wrappers default to logging the full request and response during development. Teams forget to disable this before launch, and user prompts — including any PII users accidentally typed — go into application logs that may be retained indefinitely, indexed by log-search tools, or accessible to broader internal teams than the product team.
Assuming “privacy mode” is the default account setting. OpenAI API accounts default to retaining prompts and responses for 30 days for abuse monitoring (as of the API docs checked May 2026). Anthropic retains for a shorter period by default, with opt-out available. Neither is “no retention” until the team explicitly adjusts the account settings. A surprising number of deployed applications run on the default retention period without anyone checking.
Copying full prompts into analytics and A/B frameworks. A product-analytics library that tracks “user sent message” may copy the full prompt text into a session replay or event log. The team sees aggregate feature usage and never inspects what the analytics provider holds.
Ignoring what the vendor’s fineprint says about training. Several providers reserve the right to use API inputs to improve their models unless the account opts out. The opt-out checkbox is not the default. If a team sends customer data through the API without verifying the data-processing addendum, they have effectively shared that data with the model provider’s training pipeline.
Support-ticket tooling that preserves raw inputs. When a user reports an AI error, the support agent copies the raw prompt and response into a ticket. That data is now in the CRM, the support-knowledge base, and any AI-summarisation tool the support team uses — well outside the application’s data boundary.

Retention details (May 2026)

As of May 2026:

OpenAI API (default): Prompts and responses retained for 30 days for abuse and misuse monitoring. Zero-data-retention (ZDR) available on request for API users through the data-processing agreement, but not the default for all accounts. For ChatGPT and consumer surfaces, retention terms differ.
Anthropic API (default): Prompts and responses retained for a shorter evaluation window (typically less than 30 days) with documented opt-out. The Claude API does not use data for training by default.
Google Cloud Vertex AI: Model input/output data is customer-controlled and not used for model improvement unless explicitly opted in.
AWS Bedrock: Model inputs and outputs are not used for service improvement or training. However, logging and monitoring features (CloudTrail, CloudWatch) may capture them.

These details age. The more durable behaviour is: check the provider’s data-processing terms at the time of integration and set the retention/opt-out flags before sending real data.

Practical decision check

Before launching an LLM feature, ask:

Which systems will store the full prompt and response? Application logs, analytics, support tickets, A/B frameworks, model-monitoring dashboards — enumerate every sink.
What is the retention period for each sink? 30 days, indefinite, or “until someone cleans it up”? Define and audit each.
Is the provider’s data-processing agreement configured for zero retention? If the account is running on default retention terms, fix that before launch.
What data in the prompt could you strip before it reaches the vendor? Name, email, account ID, location — if the prompt does not need it, remove it client-side.
If a vendor changes its retention policy next month, will you know? Do you track terms changes, or will you only find out when a customer asks?

Evidence and caveats

Originating brief: 061-data-leakage-in-llm-apps-logs-prompts-files-and-vendor-retention.md
Check date: 2026-05-24
This draft uses current primary or near-primary sources only for the gap-fill citations requested by the brief.
No hands-on product claim is made unless the source path is explicit in the text.
If provider policy, retention, tool-use or citation docs change, this page should be re-checked before promotion.

Source and evidence notes

ICO UK GDPR guidance and data minimisation principles — https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/
NIST AI Risk Management Framework — https://www.nist.gov/itl/ai-risk-management-framework
OpenAI API data usage / retention docs — https://platform.openai.com/docs/guides/your-data
Anthropic privacy documentation — https://support.anthropic.com/

Internal-link suggestions

/cache/chat-history-is-not-memory-how-llm-apps-remember-users/
/cache/what-is-a-token-and-why-it-affects-ai-cost/
/run/pii-handling-for-llm-apps-minimisation-before-redaction/

Methodology

What was checked: originating brief plus current provider/standards documentation relevant to the topic.

What the sources were used for:

to keep the claims cautious and specific;
to date the guidance where policy or operational details can move;
to avoid turning source notes into marketing copy.

Assumptions and limits:

This is an evergreen concept page, not a benchmark report.
No launch, outreach, affiliate, payment or tracking changes are implied.
The draft is public-clean and omits internal ticket IDs by design.

Change Log

2026-05-27: Added direct source URLs to all named providers and services; added Change Log section. Content unchanged.