RAG permissions: why your assistant leaks and how to stop it

Because a vector index has no idea who is asking, and the filter is the easy half. The hard half is keeping permissions synchronised with a source of truth that changes during the working day, which is where the leak window opens and why this never shows up in testing.

JarvisBitz

An internal assistant gets built, it works, and it goes to a security review where somebody asks what happens if a contractor asks it about executive compensation. The honest answer, in most first builds, is that it tells them, in a well written paragraph, with a citation.

This is the most common blocker we see on otherwise finished retrieval projects, and it is almost never caught earlier because of how these systems are tested. The people building it query it with their own accounts, and they can see everything, so every answer looks correct. The bug is invisible until someone with narrower access tries it, which is usually after the demo that got it funded.

Your index does not know who is asking

The root cause is structural rather than a missing feature. As Truto put it in May 2026, "your vector database has no concept of who is asking". It holds vectors and metadata and returns whatever is nearest. Permissions that live in SharePoint, Confluence, your CRM or your file store do not travel into that index unless somebody carries them there deliberately.

Two responses are common and both fail.

Telling the model not to. A system prompt instructing the assistant to only discuss documents the user is entitled to see. This is the same category error we covered in prompt injection: it asks the component being attacked to police itself. Truto is blunter, and correct, that "relying on an LLM's system prompt to not reveal sensitive information is naive and will immediately fail security reviews".

Filtering afterwards. Retrieve first, then drop the passages the user should not have before showing the answer. This feels safer and is not, because by then the content has already been in the model's context and has already shaped the answer. You are removing the citation while keeping the leak.

The filter is the easy half

Most published guidance stops at pre-filtering, and if that were the whole problem this would be a solved feature rather than a recurring blocker. The part that actually takes engineering is that permissions are not static and your index is a copy.

Truto's illustration is the clearest we have seen: "an employee leaves the finance team on Monday morning. The nightly sync runs at 2 AM. From 9 AM Monday to 2 AM Tuesday, that user can still pull highly sensitive finance documents from your RAG pipeline."

Seventeen hours, from a system that is working exactly as designed. Nothing errored, no alert fired, and the access log in the source system shows nothing because the source system was never queried. That window is the actual product of your sync interval, and most teams have never calculated it.

It gets worse in a specific way. Many pipelines re-sync a document when the document changes, which is the wrong trigger, because permission changes usually happen without touching the document at all. Someone changes team, a group membership is edited, a folder is re-permissioned. The file is untouched, so nothing re-syncs, and the stale ACL can persist indefinitely rather than until the next nightly run.

What to build instead

• Capture permissions at ingestion, as metadata on the vector. Whatever the source system knows about who may read a document travels with it into the index. This is the part people mean when they say access control, and it is necessary rather than sufficient.
• Filter before the search, as a hard constraint. Not a re-rank, not a post-pass. The user's identity narrows the candidate set before nearest neighbours are computed.
• Sync on permission events, not document events. Subscribe to permission change notifications where the source system offers them. Where it does not, run a reconciliation job whose only purpose is comparing ACLs, on a schedule set by how long a leak window you can defend.
• Reconcile periodically regardless. Event streams drop messages. A job that walks the corpus and re-reads permissions is unglamorous and is the thing that catches what the events missed.
• Log the identity used for every retrieval, alongside the documents returned. Without it you cannot answer the question a security review will ask, which is not whether it is secure now but what it returned last March.

The cases that break the simple model

Two situations defeat document level permissions entirely, and it is better to find them now than during the review.

Mixed sensitivity inside one document. A board pack where most is circulable and three pages are not. Document level ACLs are the wrong granularity, and the options are splitting at ingestion or excluding the document. Both are decisions for the business rather than the pipeline.

Answers that aggregate. A user permitted to see each of five documents individually may not be permitted to see the conclusion drawn across all five, which is precisely what a retrieval system is good at producing. This has no technical fix. It is a scope decision about what the assistant is allowed to be asked.

The test worth running before the review

Create an account with genuinely narrow access, narrower than anyone on the project has, and ask it the twenty most sensitive questions you can think of. Then change that account's permissions, wait five minutes, and ask again.

The second half is the one that finds the real problem. Most systems pass the first test and fail the second, and the gap between them is your leak window measured in practice rather than assumed from a cron schedule.

We cover the related failure, permissions applied after retrieval quietly changing what the model saw, in why RAG gives wrong answers, and the questions a buyer should put to any vendor building this in AI vendor security questions.

If you are heading into a security review with a retrieval system and want the leak window measured before somebody else measures it, our free AI audit covers exactly this, and how we architect secure AI systems covers building it properly the first time.

Related reading

Is RAG still needed now that context windows are huge?

Yes, for most business systems. Long context solved the cases where all your data fits and you can afford to resend it every call. Retrieval still wins on cost, freshness, permissions, and being able to show where an answer came from.

Why your RAG gives wrong answers that sound right

Because retrieval and faithfulness are different failure modes and most teams only measure the first. A better retriever can make wrong answers more convincing, not less, so faithfulness has to be scored separately and low scoring answers routed to abstention.

Should you fine-tune a model on your company documents?

Almost certainly not. Fine-tuning shapes behaviour and format, it does not reliably install facts. Most requests to train a model on internal documents are a retrieval or prompting problem wearing a more expensive hat.