Access control for RAG: why retrieval permissions matter before generation

Short answer: If your RAG system retrieves documents the user should not see, it does not matter what instructions you put in the generation prompt. The model will see the content and may expose it — via direct quoting, inference, or accidental leakage in reasoning traces. Access control must be enforced at indexing and retrieval time, not delegated to the prompt.

What it means

In a typical RAG pipeline, the user sends a query, the system retrieves relevant documents from a vector database, and the retrieved context is inserted into the LLM prompt for generation. The problem is obvious once you state it: if the retrieval step returns a confidential document, the prompt — and therefore the model — sees that document.

Prompt-level mitigations (“do not reveal internal documents”, “only answer from authorised sources”) are fragile because:

Models can accidentally quote or paraphrase retrieved content even when instructed not to
Safety refusals or reasoning traces may reveal that a confidential document was retrieved
The model has no way to distinguish document-level permissions — it sees everything in the prompt as context
Prompt injection or prompt manipulation can bypass instruction-level restrictions

Access control in RAG must be a retrieval-layer concern, not a generation-layer concern.

Where teams misuse it

“We use a system prompt that says ‘only answer from authorised documents’.” This is the most common and most dangerous approach. The model has already seen the unauthorised document. Whether it exposes the content depends on the model’s instruction-following, the user’s prompt, and luck — not on a secure access boundary.

“We store all documents in one index and filter by metadata at query time.” Metadata filtering is better than nothing, but it depends on the filter being correctly applied in every query path. A bug in the filter logic, a missing filter parameter, or a vector search that returns results before the filter is applied can all leak documents.

“We just give each user their own collection.” This works if the mapping between users and collections is correct and maintained. It breaks when users change roles, when documents are shared across teams, or when collections are administratively misconfigured.

Practical decision check

Before deploying a RAG system to production, verify:

Does retrieval enforce document-level permissions before returning results to the prompt? Not just in the embedding query, but as a separate authorisation step.
Is access control tested for every role and document sensitivity level? Create test queries that should return no results for each role/document combination.
Is there a “needle test” for leakage? Plant a clearly identifiable “secret” document that only an admin should retrieve, and verify no other role can get it via direct query, paraphrasing, or reasoning-trail exposure.
Is access control logged and auditable? Can you identify which user retrieved which documents, and whether any unauthorised access attempt occurred?
Does the system handle document access changes retroactively? When a user loses access to a document, do existing RAG sessions stop retrieving it?

Architectural patterns that work

Pre-filter at retrieval time — apply document-level permissions before the vector search, not after. Use a separate authorisation service or index traversal that checks each candidate document’s access control list.
Post-filter with separation guarantee — if pre-filtering is not possible (e.g., because the vector store does not support it), retrieve a large candidate set, apply authorisation filtering, then use only the authorised subset in the prompt. Log any filtered-out documents as a security event.
Per-user or per-role partitioned indexes — create separate vector indexes for each access group. Query routing goes to the correct index based on user role. This is operationally heavier but provides the strongest access boundary.
Audit-trail-aware chunking — attach document-level permissions to each chunk at indexing time so the permission follows the content wherever it is stored.
Never rely on the prompt alone — every unauthorised document that reaches the prompt is a security incident waiting to happen.

Evidence and caveats

Sources:
Date checked: 2026-05-25. Vector database access control features evolve rapidly; check current documentation for filtering capabilities.
Caveats: Pre-filtering at retrieval time adds latency to the search step. Per-user partitioned indexes multiply storage and maintenance overhead. No access control approach is perfect — layered defence (retrieval filtering + prompt instruction + output monitoring) is the appropriate posture for sensitive document collections.
What would update this: Standardised access control interfaces across vector databases, or regulatory requirements specifically addressing RAG data access boundaries.

Change log

2026-05-25 — Initial audit revision. Added direct source URLs to evidence section; changed source listing from named-only references to linked citations. No material changes to claims or guidance.