There is a specific comment you hear at every enterprise AI
conference lately, and it usually starts as a question
whispered in a security meeting.
An architect leans over to the person next to them
and asks if they are actually carrying each file's permissions
into the index to filter per user at query time.
Most people in the room go silent when they hear that.
They stay quiet because the honest answer is no,
and they know they aren't doing it.
That silence is the entire problem.
This episode is about the massive gap between where your data
sits and who is actually allowed to see it.
We are talking about why the term private rag does not mean
what you think it means and how this missing layer
transforms a knowledge system into the biggest insider data
leak your company will ever build.
We are going to explore why permission models matter more
than your choice of vector database or your infrastructure
decisions.
To do that, we have to answer one deceptively simple question.
If you extract documents from SharePoint
to build a perfect retrieval system,
but lose the permission model that SharePoint was enforcing,
what have you actually built?
Why everyone talks about location, not access?
The entire narrative around private rag has a structural floor
that has been hiding in plain sight for a long time.
If you go to any vendor conference and listen to how
Rags Security gets discussed, you will hear a lot
about infrastructure choices.
They talk about self-hosting versus the cloud,
or they brag about German data centers and EU data boundary
certifications.
The conversation centers on one specific obsession,
which is where the data physically sits.
Companies invest heavily in this single question
by building VPNs and deploying on-premise infrastructure.
They sign complex compliance agreements for data residency
because the assumption is simple.
If the data stays in Germany and never leaves the firewall,
then it must be secure.
This is the sovereignty illusion.
The moment you start indexing documents for Rags,
that illusion collapses completely.
Here is what actually happens in a real organization.
You have a SharePoint instance with millions of documents,
and every single one is protected by a permission
model that took your team years to build.
You have site-level access controls and library-level
permissions, plus folder hierarchies
with inherited access rules and item-level ACLs.
All of these layers from groups to external sharing work
together to ensure that only the right people
see the right documents.
Then you decide to build a Rags system
because you want to make all that knowledge searchable.
You want employees to ask questions and get answers grounded
in your internal documents so you start the extraction process.
Document by document, you pull content out of SharePoint
to chunk it and embed it.
You store it in a vector database that might be self-hosted
or sitting on a VPS in Berlin
with premium compliance certifications.
But the moment that document leaves SharePoint,
the permission model dies.
You capture the words and the metadata,
and you even captured the context,
but you did not capture the guardians.
You left behind the access control lists
that actually decided who was allowed to open those files
in the first place.
This is not just a hypothesis.
Gardner is very direct about what comes next,
and their prediction for 2027 is stark.
They expect 40% of AI data breaches to come from cross-border
Gen-AI misuse.
But if you read carefully, the real issue isn't geography.
It is not about the data moving across borders,
but rather about the data moving without its access controls.
The false equation everyone has been using
is that data in Germany equals security.
But that is not security.
That is sovereignty.
Real security requires a different question entirely,
and you have to ask who can see what and under what conditions.
You have to know what happens when permissions change
or what prevents a junior employee
from accessing executive strategy documents.
Most rag projects never ask these questions
because they are too busy optimizing
for retrieval speed and accuracy.
They spend their time tuning embeddings
and testing the ranking to make sure the system is fast and private.
The result is a system where your most sensitive information
is now searchable by anyone with a log into the rag interface.
A single compromised account or a well-crafted prompt
can surface documents that should have stayed locked away
from view.
You have built something that looks like a knowledge system,
but in reality, it functions like an insider data leak
waiting to happen.
The organization invested months building that permission
model in SharePoint for a reason,
and it was about governance rather than bureaucracy.
It was a way of saying that a financial report is for the board,
and a legal document is only for the general counsel,
or rag strips all of that away.
So what do you do about it?
That is where we go next.
The three misconceptions about private.
Before we solve the problem,
we need to dismantle the thinking that created it.
There are three beliefs that teams have about private rag
that sound reasonable on the surface,
but they are just wrong,
and each one is dangerous in a different way.
The first misconception is about infrastructure.
Self-hosted equals secure.
It feels intuitive to run your own vector database
on your own servers where nobody else can reach it,
and because your data never touches a third party cloud,
this makes sense from a sovereignty standpoint.
However, from a security standpoint, it is incomplete.
A self-hosted vector database with no role-based access control
is fundamentally less secure than a managed cloud database
that enforces per user filtering.
When you optimize for location, you solve where the data sits,
but you haven't solved who can actually access it.
A contractor with a laptop on your VPN
can query the same vector index as your CFO,
and because the database has no way to distinguish between them,
both queries return the same documents.
The only difference is which physical network they are coming from.
This is why a German hosted database
with all of SharePoint indexed but zero permission metadata
is not a security achievement.
It is a compliance nightmare waiting to be discovered.
The second misconception is about access.
VPN access equals authorization.
This one creeps in quietly because the thinking assumes
that since only employees are on the VPN,
only employees can reach the RAC system.
The logic follows that only employees will see employee data
and the problem is solved,
but here is the problem that isn't authorization.
It is just network topology.
Network access and data authorization are completely different layers,
which means you can restrict who reaches the system
without restricting what they see once they are inside.
A junior sales representative and the VP of business development
both have VPN access and can both query the RAC system,
but they should not see the same documents.
In most RAC deployments, they do.
The real question authorization answers is this,
given that you can reach the system,
what subset of documents are you allowed to see?
Network perimeter security doesn't answer that question,
but permission metadata does.
The third misconception is about the LLM itself.
The model will handle it.
This is the most insidious one because it pushes the problem
inside the model where you can't see it.
The reasoning usually sounds like this.
We trust the LLM to be intelligent enough
to figure out which documents it should cite
and which it shouldn't.
We will rely on the model to remember
that certain information is confidential
and will use a system prompt to instruct it
not to expose sensitive data.
This is a structural failure.
The LLM has no way to know which documents are confidential
because it only knows what is in its context window.
If you have retrieved a document and passed it to the model,
the model sees it as grounded information
and treats it as evidence.
It will reason over it and if that reasoning serves
the user's query, it will surface that information
in the answer.
A system prompt that says,
"Don't expose confidential data" is a hope rather than a guarantee.
It can be bypassed, it can be misunderstood
and it can fail silently.
The only reliable way to ensure sensitive documents
don't influence the answer is to ensure
they never reach the model in the first place.
And that requires knowing which documents the user
is actually authorized to see
before retrieval happens.
The stakes of these misconceptions are concrete.
HR documents become visible to contractors,
executive strategy leaks to junior employees
and legal files are accessible to sales teams.
This doesn't happen because the teams were malicious
but because the system never asked if the user had
permission to see the data.
That is where the permission problem becomes concrete.
What happens when documents leave SharePoint?
Let's walk through the actual mechanics of the problem
because understanding the extraction process
is where the vulnerability becomes unavoidable.
SharePoint's permission model is built in layers
that start at the site collection level
where you can restrict who accesses an entire site.
Then it flows down to the library level
to limit access to specific document libraries
and folders within those libraries can have their own rules.
Individual items like specific files
or documents can have their own access control lists
that either inherit from above
or break that inheritance to define their own rules.
On top of that, you have security groups
and Microsoft 365 groups that change constantly
as people join projects or leave teams.
That ripple effect flows through the entire permission model
which means a user's access to a document
is the product of every layer they inherit from
and every group they belong to.
This is complex but it is also why it works.
It is why a financial report can be visible
to the finance team and the executive board
while remaining hidden from the sales team and contractors.
Now you extract that financial report for RAC.
The extraction process looks simple on the surface
because you pull the document and break it into chunks
by section or paragraph.
You generate embeddings for each chunk
and store those embeddings in your vector database.
But here is what you lose at each step.
When you extract the document,
you get the content and the metadata
like the title and author,
but you don't get the complete access control list.
You don't get the list of every user and group
who can open this file and you don't get the inheritance chain.
You lose the information that says section one is visible
to everyone while section three is visible only to the board.
Then you chunk it.
Now you have broken a document with a single permission model
into a hundred smaller pieces
that are semantically independent but share an origin.
In your RAC system, most setups default
to having all chunks inherited from the parent document.
But if the parent had 20 different permissions scopes,
you have now lost that granularity.
The chunk that contained board-only strategy
is now tied to the same access rules
as the chunk that contained public information.
Then you embed it.
The embedding captures semantic meaning
but it does not capture permissions.
Your embedding engine has no concept
of this vector represents information
that only five people should see
and the vector exists in a space
where access control is invisible.
Then you store it in your vector database
and this is where the real problem materializes.
For your database to enforce permissions at query time,
it needs to carry the actual list of users
and groups allowed to see each specific chunk.
If that metadata isn't there
because you lost it during extraction,
then the database has nothing to filter on.
Consider a real scenario where a 50 page financial report
has 20 different permissions scopes for different audiences.
When it lands in your vector database,
it becomes a single record with a single permission field
or perhaps no permission metadata at all.
Now that same report is retrievable by anyone
who can query your Ragsystem.
This is the permission drift problem
that Gartner identifies as a recurring failure mode
in multi-tenant Ragsystems.
The source of truth has a permission model
but your retrieval system has a different model
or no model at all.
They drift apart and that gap is where breaches happen.
The false assumption most teams make
is that they will handle permissions at query time.
But if the index has no metadata encoding
who can see what, there is nothing to filter.
You cannot enforce access rules that were never captured.
The technical solution requires something different.
Materializing permissions at ingestion time.
ACL metadata extraction, the missing layer,
giving up on Rags isn't the answer.
The real solution is to rebuild the very thing
that extraction usually destroys.
ACL metadata extraction means you capture
the entire SharePoint permission model,
the exact second you pull a document.
You aren't just grabbing the text.
You are grabbing the authorization context that lives with it.
You then encode that context directly into your vector database
so that when a user asks a question,
the system knows exactly who is allowed to see what.
This fundamentally changes how your data is structured
instead of just storing a block of text
and its mathematical embedding every single record
in your database carries explicit permission metadata.
You'll see fields like allowed user id,
which list the specific people who can see that chunk.
You'll have allowed group ids for team access
and sensitivity label to mark the classification level.
If you're working in a multi-tenant environment,
you'll even include a tenant ID.
Every chunk becomes self-describing
and it tells the system exactly who it's visible to
before the search even starts.
Microsoft provides specific guidance
on how to handle this in a real-world setup.
When you use Azure AI Search,
you set indexer permission options on your SharePoint data source
to give the indexer a specific job.
You're telling it to pull the user ids and group ids
from SharePoint and turn them into metadata fields
in your index.
The indexer does the heavy lifting
of reading those complex SharePoint ACLs
and translating them into a clean, normalized format
that your search layer can actually understand.
But here is the critical detail that most teams miss.
You cannot use user names or email addresses for this.
People get married and change their last names.
Companies rename their domains, display names,
shift all the time.
Instead, you must use EntraID Goides,
which are the unique strings of numbers and letters
that represent every user and group
in the Microsoft ecosystem.
A Go-Ide is stable and never changes
when a person updates their profile
and it won't break if an old email address
gets reassigned to someone else.
This ensures the identity matching
between your rack system and your source of truth
stays perfectly consistent.
User A in SharePoint will always be recognized
as user A in your vector database.
The extraction happens during the ingestion phase
rather than at query time.
This timing is vital for both your security
and your system performance.
You aren't waiting for a user to ask a question
and then scrambling to figure out permissions on the fly.
You've already resolved every single permission
the moment you index the document.
The metadata is already there, it's encoded
and it's ready to go.
When the query finally arrives,
the filtering is incredibly fast
because the database just applies a simple expression
before it ever returns a result.
A real implementation looks like this.
You start by setting up a SharePoint connector
through Microsoft Graph.
But instead of using the broadsides,
read or permission that would give the connector access
to every file in the company you use sites.
Select it.
This allows you to scope the access
to only the specific sites you actually need to index.
As the connector pulls each document,
it calls the Microsoft Graph API to ask
who can see the file and what the inheritance chain looks like.
It resolves all of those complex rules
into a simple list of guides.
Finally, it packages the document chunk
with that permission data
and sends it off to the vector database.
The real challenge is that the SharePoint permission model
is incredibly messy to translate.
You have inheritance rules and broken inheritance
where a specific folder tells the system
to ignore the parent rules.
You have external guests with limited access
and sharing links that grant permission
to people without names.
Direct assignments can override group memberships at any time.
Getting this right requires serious engineering
because you have to turn all that chaos
into a metadata model your database can actually use.
Most teams underestimate
how hard this is.
They assume pulling permissions is a simple checkbox
but it's actually a complex mapping exercise
even though it's difficult, it is completely necessary.
Once that metadata lives in your index
and every chunk carries its own security badge,
your retrieval layer finally has something
it can enforce and that is where the actual security begins.
Authorization before retrieval not after.
This is the exact moment
where most rag deployments fail the test.
It happens right when it matters most at query time.
The principle here is simple
but most people get it wrong.
You should never retrieve a document
and then decide if the user is allowed to see it.
That decision has to happen before the retrieval not after.
This distinction is much more important than it sounds.
If you retrieve the data first and filter it later,
you've already allowed sensitive information
to influence the entire process.
The vector database has already ranked it.
The re-ranking model has already scored it.
The document might have even made it into the context window
where the LLM started thinking about it
before you realized the user shouldn't have access.
By the time your filter kicks in,
the damage isn't being prevented because it's already happening.
Post filtering is a structural failure
because sensitive data leaves a trail.
It changes ranking scores and influences semantic caches.
It shapes how the model reasons.
Even if the user doesn't see the final text,
a document that should have been invisible
still changed the final answer.
A filter that shows up too late cannot erase that influence.
Real security requires a completely different sequence of events.
When a user asks a question,
the system has to follow a very specific order.
First, it must authenticate the user
and verify their session to confirm they are
who they say they are.
Second, it has to resolve their group memberships
in EntryD.
This is a huge step because permissions usually flow
through groups rather than individual names.
If the system doesn't know which groups a user belongs to,
it has no way of knowing what they are allowed to see.
Third, the system builds an authorization filter.
It creates a query that says,
"Only show documents where the tenant matches this user,
and either the user's ID or one of their group IDs
is in the allowed list."
Fourth, it applies that filter to the vector search
before the search even runs.
The database narrows the field
to only the authorized documents.
Fifth, it retrieves only from that safe subset,
only those documents come back,
and only those documents can influence the final answer.
The filter expression is simple in theory, but complex to build.
You are essentially telling the database
to find documents where the tenant ID is correct
and the user's identity intersects with the allowed list.
Notice what this does to the workflow.
You aren't pulling everything and then sorting through the pile.
You are only pulling the documents
that meet the security criteria from the very start.
The filtering is inseparable from the search itself.
The database never even looks at unauthorized documents
because they are outside the search area from the beginning.
But there is a practical side to this
that most architects miss.
Permissions are not static.
People leave projects, group memberships change,
and files get reshared with new teams every day.
The SharePoint model is alive and constantly shifting.
Your system might have indexed everything on Tuesday,
but if permissions changed on Wednesday,
your Thursday query might be using all data.
This is why you need a plan for continuous authorization.
You can't just assume the metadata you saved
during ingestion is still 100% accurate.
You generally have two ways to handle this.
You can refresh your permission metadata on a tight schedule,
which is thorough, but can get expensive,
or you can use a fallback check.
The database filter is based on the saved metadata,
but before the results go to the LLM,
the system makes a quick live call to SharePoint
to confirm nothing has changed.
This catches those rare cases where someone's access
was revoked right after the document was indexed.
There is a performance cost to doing things the right way.
One financial firm indexed half a million documents
with this metadata and saw their latency go from 200 milliseconds
to 250, that is a 25% slowdown.
For most companies, that is a perfectly fine tradeoff.
If you have a massive, high volume system,
you'll need to optimize by caching group lookups
or using database indexes designed for these specific checks.
The tradeoff is a conscious choice.
A slower search that is actually secure will always
be the fast search that leaks private data.
And this is where the work gets even harder.
A single retriever following these rules is one thing,
but when you have multiple agents passing user context
back and forth, you run into a completely different set
of problems.
Why one agent can't do everything?
This is where the architecture question becomes unavoidable.
Most organizations start with a single agent,
which is usually just one LLM process.
It receives the user's question, decides what to retrieve,
gets the documents back, and then reasons over them
to generate an answer.
Everything happens in one place, one actor,
one point of responsibility.
The appeal is obvious because it's simpler and easier to build.
You don't need to orchestrate multiple systems,
and you don't need to think about how identity flows
from one agent to another.
You just point the agent at your index documents
and let it work.
But it will fail under pressure in specific predictable ways.
The term for this problem in security architecture
is the confused deputy.
It's an agent that holds all permissions simultaneously,
meaning it can read anything and do anything
without any built-in constraint on itself.
When a user asks it a question, the agent
doesn't think about what the user is allowed to see.
Instead, it only thinks about what information it has access to
that answers the question.
Those are two completely different questions.
The first is about authorization, while the second is about capability.
A single agent conflates those two.
It has the capability to retrieve everything in the index
because it has broad permissions to the database,
and it treats that capability as a guide for what to do.
If the information exists and answers the query,
the agent retrieves it.
Now compound that with prompt injection.
A malicious user, or even a malicious document inside your index,
crafts a prompt design to make the agent ignore its intended purpose.
It might be something subtle that manipulates the agent
into exfiltrating data through a seemingly innocent query.
The single agent has no checkpoint to stop it.
There's no second system verifying if the request is legitimate,
and there's no gatekeeper saying, "You're not supposed to access that."
The agent just executes.
It retrieves, it reasons, it outputs.
The failure mode is worse than simple data leakage
because it follows the principle of confused delegation.
The user never explicitly asked for the sensitive data,
but the agent was manipulated into surfacing it anyway
while operating under one set of instructions.
And because the agent is the final authority
with nobody validating what it retrieved,
the damage happens silently.
Here's what happens in practice.
A junior employee asks the system a question about a client project,
and while the question is legitimate,
there is instruction text embedded in the index documents.
A well-meaning senior employee put a note there
saying this section contains pricing strategy and margin analysis
and should only be shown to accounts over $5 million in ARR.
The single agent retrieves that section,
but it doesn't understand the note as a boundary.
It understands it as context.
It includes the pricing and margin information in its answer
because the information is relevant to the query.
The junior employee now has access to pricing data
they weren't supposed to see.
This didn't happen because they hacked the system
or manipulated the agent directly.
It happened because the single agent was the only decision maker
and had nobody checking its work.
The individual agent also doesn't know what it doesn't know.
It can't distinguish between a document marked confidential
for executives and a document that is public.
To the agent, those are just different documents
in its retrieval results.
It treats them the same.
A confidence score, a relevance ranking.
There is no categorical difference and no permission boundary.
The document either answers the question or it doesn't.
This is why gardeners' data matters here.
They predict that 40% of agente AI projects will be canceled by 2027.
The reason isn't model quality or retrieval accuracy.
It's governance.
It's the realization, usually late in deployment,
that a single agent making all the decisions
creates liability that the organization isn't prepared to accept.
The false economy of building one agent is that you save time upfront
but you trade that time for risk downstream.
When that risk materializes and data leaks or prompt injection succeeds,
the cost of fixing it is exponentially higher
than the cost of building it right initially.
The solution isn't to give up on Rags.
It's to distribute responsibility.
You need to add checkpoints and create agents
with specific roles, specific permissions and specific constraints.
That's the multi-agent model.
The five-agent retrieval model.
The alternative is to think about the retrieval problem
as a sequence of distinct responsibilities.
Each one is owned by an agent
with a specific mandate and specific constraints.
Not all agents need broad access
and not all agents need to see the full permission model.
Some agents don't touch data at all because they handle routing
while others validate or generate.
The idea is that by breaking the monolith into specialized components,
you distribute authority in a way that prevents any single actor
from making all the decisions.
Here's how it works in practice.
Agent one is the router.
Its job is to understand what the user is actually asking for
and determine if the request is legitimate.
It doesn't touch documents and it doesn't retrieve anything.
It receives the natural language question
and evaluates whether it is a reasonable request.
It asks if this person is asking for something
that makes sense within their role.
It checks if the question triggers any red flags
like patterns that look like prompt injection
or data exfiltration attempts.
The router makes a simple decision to either proceed or block
if something looks wrong.
The request stops here.
No retrieval happens.
No data is at risk.
Agent two is the query translator.
Once the router has approved the request,
the translator's job is to convert natural language
into a structured retrieval query.
The user asks the question in English
and the translator transforms that into the exact parameters
the retrieval system needs.
It determines what keywords to search for,
what semantic meaning to match,
and what metadata filters to apply.
But here's the critical part.
The translator never decides which documents the user
is allowed to see.
It only shapes the query.
It has no access to permission models
and no authorization logic.
It's a pure transformation function.
Agent three is the retriever with authorization.
This is where the security boundary lives.
This agent takes the translated query from Agent two
and executes it against the vector database.
But before it returns anything,
it applies the user's permission filter.
It constructs the authorization expression
we talked about earlier and verifies
if this user has access to this document.
It acts as a bouncer.
Documents that fail the permission check
never make it out of this agent.
The retriever has a simple rule.
If the user can't access it, they don't get to see it.
Period.
The retriever is narrow in scope,
but absolute in its enforcement.
Agent four is the validator.
The retriever sent back a set of authorized documents,
but the validator asks if they are the right documents.
It checks if this collection of sources
actually makes sense for answering the user's question
and if it fits this person's role.
It looks for inconsistencies or red flags in the retrieved set.
The validator is a quality gate that can refuse to proceed
if the retrieved documents don't pass validation.
If a financial analyst queries about production logistics
and the retriever comes back with internal memos
from the finance department,
the validator flags that mismatch.
It prevents nonsensical or suspicious retrievals
from reaching the next step.
Agent five is the generator.
This agent receives only the documents
that have passed authorization and validation.
Its job is straightforward.
It creates a coherent, well-sourced answer
using only the context provided.
Here's what's crucial about the generator's position.
It never sees the full permission model.
It doesn't know which groups could access which documents
and it doesn't have the authorization logic.
It only sees documents that have already been vetted
as appropriate for this user.
The generator operates under a fundamental constraint
where it can only work with what it was given.
It cannot fetch additional context on its own
and it cannot bypass the authorization layer.
It has no way to know that sensitive documents exist
because they were filtered out before it ever saw them.
Each agent has a role.
Each agent has specific permissions.
Each agent has specific responsibility.
The router guards the gate.
The translator shapes the request.
The retriever enforces access.
The validator ensures coherence.
The generator creates the answer.
This separation of duties changes everything.
No single agent holds all permissions
and no single agent makes all decisions.
The permission model itself doesn't live in the LLM.
It lives in the retriever,
which is a deterministic component outside the model.
Microsoft's multi-agent reference architecture
uses this pattern for co-pilot deployments
because it works at scale.
And it requires something that basic RAC doesn't demand,
which is strong identity and access control
at every single hop.
From user token to agent action,
we need to look at how identity actually moves
through a multi-agent system
because having five specialized agents
is only useful if every single one knows exactly
who they are serving.
When a user logs in through EntraID,
the system validates them and issues a digital token.
This token carries specific details like the user's unique ID.
Their group memberships, their role, and their tenant.
Because it is cryptographically signed,
nobody can forge or change the data.
So the token stands as proof that says this is user A
and here is what they are allowed to do.
The obvious temptation is to just pass
that raw token to every agent in the chain.
You might think it makes sense to let each agent use it
to prove the user's identity as they talk to one another.
But in reality, that is a massive mistake.
You should never pass raw user tokens directly to an LLM
or share them freely between agents.
The problem isn't that your agents are untrustworthy
since these are your own systems,
but rather that the token itself becomes a massive security
liability.
If an agent gets compromised or a prompt injection trick
forces an agent to reveal what it knows,
that raw token becomes the crown jewel for an attacker.
It can be replayed in other systems or even sold,
acting as a secret key that grants access to places
the user never authorized.
Instead, you should use OAuth 2.0 token exchange, which
acts like a translation layer for identity.
The gateway receives the user's original token
and confirms that the person is real,
but then it does something different.
Instead of passing that original credential forward,
the gateway creates a new, scoped token.
This is a derivative token that tells agent 3
it is authorized to act for user A for exactly 15 minutes
and only for the purpose of retrieving documents.
Every agent in your chain receives its own limited token.
The router gets a token that only allows it
to evaluate requests while the translator gets one
that lets it shape queries without ever seeing a document.
The retriever receives a token that
says, "Act for user A with user A's specific access level."
And the validator gets read only access to check the results.
Finally, the generator gets no special access at all,
since its only job is to process the text it receives.
This is the core principle of delegation.
Agent 3 isn't acting as a powerful service account
with broad access, but is instead acting as a deputy
with the exact same authority as the user.
If user A is allowed to see the sales folder
but is blocked from the finance folder,
then Agent 3 will be blocked from finance as well.
Let's look at a real scenario where user A works in sales
and belongs to the North America and product feedback groups.
When they ask the system a question,
the retriever receives a scoped token
that encodes those specific group memberships.
The retriever then builds a filter
that only shows documents where the ID matches user A
or belongs to one of those three specific groups.
The retriever happens under that strict constraint,
so the user sees their sales and feedback documents
but never catches a glimpse of HR or executive files.
Handling group memberships is where the technical complexity
usually kicks in.
You generally have two choices for how to handle this.
First, you can resolve groups at the moment
over authentication by fetching everything from EntraID
and putting it in the token.
This is fast, but if a user joins a new group,
the token won't show that change
until they log out and log back in.
The second option is to cache group memberships
with a short lifespan,
meaning the retriever refreshes the list every few minutes
so changes propagate quickly.
There is also a technical hurdle called group overage
that you need to watch out for.
If a user belongs to more than 150 groups,
EntraID cannot fit all those IDs into a standard token.
Once you hit that ceiling,
the system has to choose between dropping group info
or making extra API calls.
This matters because if the group information is missing,
your authorization filter won't know which documents
to show the user.
This identity model is the foundation of the whole system.
It ensures that every agent acts with the right authority
and it keeps the user's actual credentials safe.
Why policy can't live in the prompt?
This is where the architecture has to make a hard choice
about where authority actually lives.
When teams start building access controls for Rags,
they usually have the same intuitive idea.
They know the LLM understands language and context
so they try to put the authorization rules
right in the system prompt.
They tell the model to only show public documents
and to never reveal anything tagged as confidential.
It sounds like a reasonable plan
because the model is smart and follows instructions.
But the truth is, it won't work reliably.
The reason is simple.
LLMs do not enforce rules, they follow instructions.
Instructions can be overridden or bypassed
through prompt injection and a user can easily craft a prompt
that conflicts with your original setup.
If a retrieved document contains text
that contradicts your rules,
the model doesn't have a way to prioritize one over the other.
It just looks at all the information
and makes a probabilistic guess,
which means it will eventually violate your security rules.
This isn't a problem with the quality of the model.
Even the most advanced models fail at this
because you are asking them to do something
that should be handled entirely outside of the AI.
The real security requires a policy decision point
or a PDP, which sits outside the model.
This is a deterministic engine
that looks at an authorization request
and gives a simple yes or no answer.
It isn't making a suggestion or a guess,
but is instead making a rule-based decision
about whether a specific user is allowed
to see a specific document right now.
The PDP takes in structured data like the user's ID and roles,
along with the document's classification and sensitivity level.
It also looks at the context
such as whether the user is a contractor
or if they are logging in from a high-risk location.
It evaluates all these factors against your rules
and returns a final decision.
These rules are very direct like finance employees
can see budgets, but contractors cannot.
You might have a rule that denies access
if a user tries to view a confidential file
from a VPN outside of office hours.
The PDP doesn't care about your prompt
or which LLM you are using
because it only cares about the rules and the requests.
Crucially, the PDP itself is not an LLM.
It is a dedicated rule engine like openFGA or turbos
that is designed to be auditable and consistent.
You can see exactly which rule was triggered,
you can trace every decision
and you can update your policies
without ever touching your code.
In a real workflow, the retriever agent calls the PDP
before it ever returns a document.
The retriever might find 10 documents
that match the user's query,
but it asks the PDP for permission before moving forward.
If the PDP says no to five of those documents,
they are dropped immediately.
Only the approved documents make it into the context window
for the LLM to see.
If you use Azure AI search,
this logic is often built directly into the query filter.
You aren't pulling a bunch of data and filtering it later,
but are instead ensuring the database only scores documents
that the user is actually allowed to see.
The separation is how you move
from probabilistic security to deterministic security.
The LLM never has to make a judgment call on access
and it never sees data that shouldn't be there.
Everything happens in a component designed for rules and auditing.
When you need to scale this across different business units
or countries with different regulations,
the PDP architecture is the only way to make it work.
You can't rely on prompt engineering
or hope the model remembers the rules.
You need a separate system that knows the law
and enforces it every single time.
Why one leak can compromise everything?
The problem we have been discussing involves
enforcing permissions correctly
and this challenge becomes catastrophic
when you multiply it across dozens of customers or business units.
In a multi-tenant system,
a single architectural failure does not just leak one person's data
but instead it leaks everyone's data at once.
Imagine you are building a rag-powered
knowledge management system for enterprise customers.
Each client indexes their own documents into your platform
and your user base includes healthcare providers,
financial firms and manufacturers
with sensitive supply chain data.
These companies trust you with their most confidential information
because they believe their data is isolated from their competitors.
But here is the problem.
Isolation is often an illusion.
One of the first real cases of this breaking
involved a semantic cash collision.
Customer A ran a query about their annual strategy planning
so the system retrieved the documents,
generated an answer and cashed the response for speed.
Two minutes later, customer B ran a query
that was semantically similar
and it matched the cash results so perfectly
that the system returned customer A's private answer to customer B.
This was not a breach of the raw documents themselves
but it was a breach of intelligence
that gave a competitor a window into a private strategy.
It was an accident caused by a configuration mistake
and it was also completely unrecoverable.
The root cause was architectural
because the vector database used a single shared index
where all documents from every customer
lived in the same namespace.
Since the cash operated at the query level
without accounting for tenant identity,
the system only asked if anyone had run the query before
rather than asking if this specific customer had run it.
One missing constraint in how the cash key was constructed
meant that confidential information started leaking between rivals.
The blast radius of that mistake was absolute
because one bug affected every tenant simultaneously.
Every single query running through the cash system
was potentially exposed to other users.
You could not fix the issue for one tenant
without fixing it for all of them
and the worst part was that nobody knew
how many times it had already happened.
Gartner identifies this exact failure mode
as one of the five recurring problems
in multi-agent rag systems
and they call it cross-tenant leakage.
This is not a hypothetical theory.
It is happening right now and organizations are discovering the floor
only after a customer finds evidence of a leak.
The technical fixes are clear
and they start with the principle of using
per-tenant encryption keys or separate indexes
if your scale allows for it.
A more common approach involves strong metadata filters
enforced at the database layer with a hard constraint
which means every single query must include a tenant ID check.
This cannot be a suggestion or a soft filter
that a developer can disable
but must be a mandatory constraint
that the database enforces before it even looks for documents.
The constraint works because it is enforceable
outside the application logic.
If your database configuration is set to always filter by tenant
there is no code path where a developer can forget to include it.
It becomes architectural.
But here is why this is urgent.
The failure mode is not rare, it is systemic.
If a healthcare SaaS company indexes patient records
from 50 different hospitals
and the permission metadata is lost during extraction
the system treats those records as one giant pile of data.
A query from hospital A could return patient data from hospital B
not by accident but by design
because the system never knew which hospital owned which record.
The regulatory exposure here is crushing.
HIPAA does not care if the leak was a simple configuration error
it only cares that protected health information
was accessed by the wrong person.
GDPR will find you for data transfers between customers
without authorization which makes your organization
liable both technically and legally.
Security trimming is not an optional feature
you add when you have extra time.
It is a compliance requirement
that marks the difference between a system you can defend in court
and one that is completely indefensible.
The ungoverned archae problem
the multi agent architecture identity propagation
and policy engines we have talked about all assume
you actually know where your rag systems are running.
The assumption is that you have visibility
and can enforce controls
but for most organizations that assumption is completely false.
Shadow AI happens when employees build their own AI systems
without any formal approval, IT review or security assessment.
A team member might decide they want to make
the internal wiki searchable
and instead of waiting 18 months for an enterprise project to start
they spend a weekend with an open source framework.
They grab a free pinecon account, index the entire company wiki
and share the link with their team.
Gartner predicts that 75% of employees will use unapproved AI tools
by 2026 that is not just a few tech enthusiasts
it is the vast majority of your workforce
and every one of them represents a potential data leak.
These shadow systems follow a predictable and dangerous pattern.
You find hard coded API keys in Python scripts
no access controls on the vector database
and absolutely no audit logging of who is asking what.
Because the person building it has broad internal access
they index everything from design docs to financial forecasts
into a flat corpus with no permission model.
The system has no idea what should be public
and what is a confidential strategy.
The problem gets worse when that system is shared.
It starts with the immediate team then moves to adjacent departments
and eventually someone mentions it in a Slack channel
where contractors see it, suddenly those contractors have query access
to your entire internal knowledge base
and they can get answers based on documents
that should never have left the building.
The system cannot enforce the obvious rule
that a contractor should only see public project materials
rather than internal financial data.
Because the system was a weekend project
there is no identity propagation or policy engine sitting
between the query and the results.
The vector database does not care that a contractor is different
from an executive so it just returns whatever documents match the search.
Since this is a shadow system flying under the radar
nobody in security even knows it exists.
There is no audit log to check
and no way to detect that sensitive information is being exposed
to the wrong people.
The breach happens silently and repeatedly
until a customer eventually notices their private data
being quoted back to them by a third party.
The solution is not to ban AI tools because that is impossible
and only encourages employees to build more hidden systems.
Instead you have to inventory what is running
and classify each system by its risk level.
A shadow rag that only indexes public documentation is low risk
so you can just implement basic logging and move on.
However, if a shadow system has access to customer data
or proprietary IP it is a high risk threat
that you must shut down or migrate into a governed architecture immediately.
This is why zero trust thinking is operationally urgent.
It is not just about building the right architecture for new projects
but about finding the systems that are already running without your permission.
Never trust always verify at every hop.
Zero trust isn't a new buzzword in the security world.
The military, financial institutions and critical infrastructure providers
have used this model for years because their stakes are too high
to assume anyone is safe.
They operate on the reality that you can't trust a user or a device
just because they manage to get onto your internal network.
But here's the problem.
Most people think a firewall is security but in reality a perimeter is just a shell.
The moment you assume a component is safe because it sits behind that shell
you've created a massive vulnerability for an attacker to exploit.
Zero trust rejects that assumption entirely by requiring you to verify
every single interaction, every request and every piece of data moving between systems.
There are no shortcuts and there is no implicit trust.
Now apply that same diagnostic thinking to your multi agent rag system.
Most teams don't do this.
They build a specialized architecture we've been talking about with five different agents
and then assume those agents are safe because they own the whole stack.
Since everything is running on the same infrastructure and part of the same project
they skip the hard security work.
They pass tokens freely and trust that if an agent has a capability
it should be allowed to use it.
That's the exact assumption zero trust is designed to break.
In a zero trust agent system the rule is absolute.
Every agent, every retrieval and every tool call must be authenticated and authorised independently.
This doesn't just happen at the front door.
It happens at every single hop in the chain.
Here is how that looks when the system is actually running.
It starts with agent one, the router.
When a user sends a request the router doesn't just pass it along.
It verifies the session, checks if the token is valid
and looks at the user's risk posture.
It asks if they are logging in from a weird location
or if they've recently changed their password and only after gathering all that evidence
does it decide if the request is legitimate.
If the verification fails the process stops right there.
If it passes agent two receives a scope token.
This is a critical distinction because that token doesn't give agent two the keys to the kingdom
or the ability to call external APIs.
It grants the authority to do exactly one thing.
Transform natural language into a structured query.
When agent two is done it passes that query and a fresh token to agent three
but it never passes its own credentials or raw user data.
It only passes a token representing the user's delegated authority
for that specific next step.
Then agent three takes over the token it receives says it can act for a specific user
to retrieve documents that the user is permitted to see and nothing more.
It can't modify files or write logs because its scope is strictly bounded by the token's signature.
Agent three checks that signature for cryptographic proof of its source,
pulls the documents that pass the filters
and then hands those documents and a new token to agent four.
Next agent four validates the work.
It's token only allows read only access to document metadata
so it can check if the results actually make sense.
It can't retrieve more documents or override previous authorizations
and if it rejects the results the flow stops immediately
no documents ever reach the model of the validator says no.
Finally agent five generates the response.
By this point the token it holds only allows it to work with context
that has already been pre-validated and pre-authorized.
It can't go off script to call APIs or fetch new data
because the token is intentionally narrow
every single one of these agents logs exactly what happened.
They record who called them, what tokens were exchanged
and whether a request was approved or blocked.
This creates a full audit trail so that if something breaks
you can trace the chain and see exactly where the failure happened.
To make this work you need specific technology like mutual TLS.
This ensures that both the calling agent
and the receiving agent verify each other's identity cryptographically
before a single byte of data is exchanged.
Every message is signed with a private key to prove it hasn't been tampered with
and every agent has its own unique identity in the certificate system.
This does add a layer of complexity.
You have to maintain the certificate infrastructure
and validating tokens at every hop adds latency to the system.
In fact network overhead from MTLS handshakes could increase your latency
by 30% compared to a basic rag setup.
That is a real cost you have to weigh.
But the security gain is absolute.
No agent can pretend to be another
and no request can bypass the verification chain.
If one agent is compromised the damage is trapped within that agent's narrow scope.
The architecture is built to assume a breach will happen
and contain it before it spreads.
Detecting when something goes wrong.
Even the most perfect architecture has to assume that something will eventually fail.
Whether it's a compromised agent, a malicious prompt that slips past validation
or a simple configuration error, the goal isn't just to prevent failures.
The goal is to detect them fast enough to stop the bleeding.
Behavioral monitoring is your early warning system.
You start by establishing a baseline for what normal looks like in your system.
You track how many documents agent three usually pulls,
how long a retrieval takes, and how often agent four rejects a result.
By looking at historical data, you build a profile of how the agents should behave during a typical day.
Then you watch for the moment things deviate.
If agent three suddenly pulls a thousand documents instead of the usual five,
that's a massive red flag.
It could be a broad query, a malfunction, or an attacker trying to scrape your data,
but the reason doesn't matter yet.
What matters is that the pattern is broken and the system needs to react.
The same applies if agent four's rejection rate jumps from 1% to 30%.
This tells you that something the retriever is sending back no longer makes sense.
Either the retrieval logic is broken, or the documents themselves have become problematic,
but either way, the anomaly needs an answer.
You also watch for agents calling tools they've never used,
or making requests from unknown IP addresses.
If an agent's latency spikes from 200 milliseconds to five seconds,
that's a behavioral shift worth investigating.
Your response framework should have layers.
Low-level issues might just trigger an alert for the ops team,
but high-savarity anomalies should trigger an automated kill switch.
The system can disable the agent, root traffic around it,
and save all the evidence for a forensic investigation.
Let's look at a real scenario where agent three retrieves a thousand documents.
Because this is outside the baseline,
the system flags it immediately without waiting for a human to check a dashboard.
It logs the details and escalates the issue to a security engineer
who can see the user, the query, and exactly which documents were pulled.
If it looks like a real threat, the engineer can roll back the results,
clear the caches, and notify the right people.
The logging requirements for this are absolute.
Every retrieval must record the user ID, the agent ID,
and the specific document IDs that were returned.
You need the timestamp and the authorization decision to see if the user passed or failed the check.
If an agent had to try multiple times,
every single attempt needs to be in that forensic trail.
But here is the catch.
Logging itself is a risk.
A log file shows exactly which users are looking at which sensitive documents,
and if those logs are stolen, they become a map of your organization's most private data.
That metadata is just as sensitive as the documents themselves.
Because of that, you have to protect your logs as carefully as your primary data.
You should use Redaction or Encryption, so the log shows the decision but hides the full document title.
You can log a document ID and keep the actual title in a separate encrypted mapping.
Logs should be stored in a tamper-evident system where any attempt to delete them triggers an alarm.
In a real implementation, you'd use tools like Splunk or DataDog to ingest this telemetry.
You define the rules, such as firing an alert if retrieval volume hits a certain threshold
or if a user starts querying at odd hours from a new location.
The monitoring system correlates these signals to find this suspicious activity.
This is where your zero-trust architecture meets the reality of daily operations.
Technical controls will stop most failures, but for the ones that slip through,
detection and response are your last line of defense.
And if this shift in how you think about AI security was helpful, follow me,
Mirko Peters, on LinkedIn for more.
If you want to see more content like this, leave a review so other people can find it
and definitely share this with your team if you're building these systems right now.
Building this is hard, not building it is harder.
Let's be honest about what we're asking you to build here.
Everything we've covered, the multi-agent architecture, identity propagation,
permission-aware retrieval and policy point sitting outside the model, is not a weekend project.
This isn't a basic RAC system with some access control bolted on at the end.
It is a security first information system that just happens to use LLMs as the interface.
The time investment is real.
If you're in a mature organization with solid identity infrastructure and clear governance,
you can probably design and deploy this in three to six months.
But if you're dealing with legacy systems, fragmented data,
or security teams new to fine-grained authorization, you should probably add another six months to that estimate.
That timeline also assumes you have the right people available from start to finish.
Most organizations don't actually do it right the first time because they follow the path of least resistance.
They start with a basic, single-agent RAC to show value and get buy-in from stakeholders quickly.
Then, around month four or five, the security team asks the obvious question about how permissions are being enforced.
And the honest answer is usually that they aren't being enforced in any systematic way.
At that point, you're stuck with a difficult choice.
You can shut the system down and redesign it properly or you can try to retrofit security trimming onto what you've already built.
Both options are expensive.
Shutting down kills your credibility and momentum with stakeholders who wanted this work in yesterday.
Retrofitting creates massive technical debt because you're trying to force permission layers into a system that was never designed to carry them.
A financial services firm went through this exact nightmare recently.
They spent $200,000 building a basic RAC system for their trading desk that indexed equity research and market data.
It was fast and accurate, but then compliance asked what happens if a junior trader accesses strategy documents meant only for portfolio managers.
That was the moment they realized their fast solution was actually a massive liability.
Fixing it took $500,000.
They had to build a new metadata extraction pipeline, redesigned the retrieval logic and create an entirely new index structure.
If they had spent $600,000 upfront to do it right, they would have saved a fortune.
Instead, they spent $200,000 on the wrong thing and then $500,000 more just to fix their mistakes.
That isn't an unusual story.
In fact, it's the typical path for most companies.
The staffing problem is another hurdle you have to clear.
You can't build a system like this with just machine learning engineers and database experts.
You need security architects who understand identity systems and engineers who are comfortable with OAuth, Sammell and Authorization frameworks.
You also need policy experts who can translate complex business rules into machine readable controls.
These are not junior roles and these people are not in unlimited supply.
If you're trying to hire these specialists while you're already building the system,
you're adding at least six months to your project timeline.
Organizational alignment is where most of these efforts actually fall apart.
Building this requires teams to work together that usually stay in their own silos.
Security has to sign off on the architecture.
Compliance has to verify the regulatory requirements.
And IT has to maintain the identity tokens.
Even the business teams have to accept that some requests will be slower
because Authorization adds latency.
If any of those groups treats this as someone else's problem, the system will fail immediately.
The business case for doing this ride sounds a bit painful at first.
It costs more upfront.
It takes longer to ship and it requires hiring expensive people you don't currently have on staff.
But that investment prevents a $10 million breach and avoids regulatory fines
that could easily reach into eight figures.
It keeps your company from becoming the primary example in the next big compliance case study.
That is the only argument that actually works with leadership.
The alternative is deploying a system that silently leaks data for years
until a customer notices their confidential info being quoted by a competitor.
That scenario costs exponentially more when you factor in the lawsuits,
the investigations, and the permanent hit to your reputation.
You aren't saving money by cutting corners today.
You're just deferring those costs until they become catastrophic.
The honest assessment is that getting this ride requires a massive investment.
But that investment is always proportional to the risk you're trying to manage.
The ecosystem for permission aware rag.
Now we can move from the abstract architecture to the actual tools you'll use to build this.
The honest truth is that no single vendor has solved this end-to-end yet.
You're going to have to stitch together multiple pieces to make it work.
If you're already in the Microsoft 365 ecosystem, you should probably start with Azure AI search.
It has native support for pulling SharePoint permissions during the ingestion process.
There is a specific parameter called indexer permission options
that tells the system to grab user and group IDs directly from the documents.
Those IDs become metadata fields in your index,
which allows you to add filters to your queries.
This ensures the results only show documents the current user is actually allowed to see.
It's a clean approach because the tool was built for this specific permission model.
Microsoft Graph is the engine that makes that whole process possible.
Graph is where you pull the permission metadata from in the first place,
and it's the primary way to see which users have access to which files.
It's also the system you call to check group memberships when a user submits a query.
You authenticate your agent through Graph and it returns the group info in the token.
The whole architecture hinges on Graph working correctly and your organization using it consistently.
When it comes to the policy decision point,
the part that decides if a user can see a file,
open FGA and service are the industry standards.
These are fine-grained authorization services where you write rules in a native language.
You ask the service if user A can see document B,
and it gives you a simple yes or no.
These tools aren't tied to any specific cloud provider,
so they're great if you're working in a multi-cloud environment.
For the actual agent orchestration,
Lang Graph is the framework that's currently leading the way.
It supports multi-agent workflows where identity is passed explicitly from one step to the next.
You can build that five agent model we talked about
and attach permission checks to every single hop.
Lang Graph handles the plumbing like token exchange and credential generation
which lets you build the system systematically.
If you need your index to stay in sync with permission changes in real time,
you should look at pathway.
Most systems rely on scheduled re-indexes,
but pathway handles continuous synchronization.
This is vital in environments where permissions change every few minutes
and you can't afford to show someone a document they just lost access to.
Vector databases themselves have very different levels of support for this kind of work.
Weeveate, Pinecone and QDrant all allow for metadata filtering,
but they differ in how they enforce those rules.
Some let you configure access at the collection level
while others force you to handle the entire authorization model in your application code.
You need to know exactly how your database handles these controls before you commit to it.
The standard patent for most organizations is using Azure AI Search and Microsoft Graph
for their internal documents.
The integration is clean because you're staying within one ecosystem,
but for companies that use multiple clouds or non-Microsoft data sources,
the patent shifts toward using OpenFGA for policies
and a custom pipeline to pull permissions from various APIs.
Think about a healthcare organization that needs to index patient records
from several different hospitals.
They use Azure AI Search with HIPAA compliant settings where every record is encrypted at the document level.
Their access policies are written in OpenFGA to ensure doctors in hospital A
can only see their own patients.
When a query comes in, the system checks the user's role,
retrieves only the authorized documents,
and then passes them back to the agent.
That isn't one tool doing everything.
It's a collection of specialized tools integrated into a single coherent system.
The big gap that nobody really talks about is that no vendor offers a complete out-of-the-box solution for this.
You have to understand the architecture well enough to assemble these pieces yourself.
Picking the tools is actually the easy part.
Making them work together securely is where the real work happens.
The 40% prediction and what comes after.
Gartner recently released a prediction for 2027 that carries a lot of weight
because of how specific it is.
They project that 40% of AI data breaches will stem from cross-border Gen-AI misuse.
This isn't about a single system being configured incorrectly
and it isn't a one-off accident.
When 40% of your risk comes from one place, it's no longer a rare event.
It becomes the primary way your company will likely experience a breach by the end of this decade.
But what does cross-border Gen-AI misuse actually look like in practice?
It happens when organizations move sensitive data into AI services
without knowing where that data is being processed.
They fail to maintain the permission models that protected the data at the source
and they completely ignore the legal frameworks that apply
once that data crosses a national border.
If we translate that into plain English, the scenario is much simpler.
Imagine a company in the EU has data governed by strict GDPR rules.
That data has a clear permission model where only specific groups of people can see certain documents.
The company decides to use a third-party AI service for customer support
so the team starts uploading customer records to that system.
Now that data is sitting in the US or Singapore or wherever that vendor happens to keep it servers.
The permission model from the original system didn't travel with the data
and the EU's legal protections didn't follow it either.
You have moved sensitive information across a legal boundary
and lost all continuity of control.
Let's look at a concrete case to see how this breaks.
A European bank starts using a US-hosted version of chat GPT
to help their support agents draft responses to customers.
The agents begin pasting customer queries into the chat box
which often include account numbers, transaction histories, and total balances.
The vendor's terms of service clearly state they will use your inputs to train their models
so your private data is now training data for a foreign company.
Because it's being processed on US infrastructure, it's now subject to US legal discovery
and intelligence requests.
Customer data that was supposed to be protected by European law
is now sitting in a system the EU has no way to monitor.
That isn't just a small mistake. It is a compliance nightmare that could cost millions of dollars.
The root cause here is structural because Genai adoption is moving much faster than governance can keep up.
Teams are desperate for productivity gains
so they deploy these systems as fast as possible.
While security and compliance teams try to build new frameworks,
three more use cases have already launched before the first policy is even finished.
The people in charge of safety are essentially chasing shadows.
Gartner has another prediction that makes this even more dangerous.
They expect 75% of employees will be using unapproved AI tools by 2026.
This shadow AI isn't just a niche problem anymore.
It has become the standard way people work.
Most of these shadow systems were never reviewed by a security team or checked for compliance.
They were built by employees who just wanted to solve a problem and get their work done faster.
This reality cuts both ways for a business.
You can't just ban AI because the productivity gains and competitive pressures are too high to ignore.
If you tell your employees they aren't allowed to use these tools,
they will just do it anyway, without any oversight.
The game has changed from prevention to governance.
Instead of saying no, you have to learn how to say yes under very specific conditions.
Permission aware rag is a major part of that governance strategy.
It gives you a way to let people use AI to improve their work
while the system itself enforces access controls.
Your data won't leak outside of its authorized scope because the permission model from your source systems is preserved.
Every query is logged, every result is filtered and the boundaries stay intact.
The global regulatory response is already hardening.
By 2027 AI governance will be a mandatory part of sovereign laws all over the world.
The EU's AI Act already has strict rules for high-risk systems
while China and India are building their own residency frameworks.
Your organization is going to face these requirements in multiple countries at the same time.
This isn't a problem for the distant future, it is happening right now.
The real impact is that companies ignoring this will face massive fines,
lawsuits and mandatory breach notifications.
A bank that loses data through an unsecured AI system
might be hit by both data protection laws and financial regulations.
Healthcare providers could face HIPAA penalties
and public agencies might see their contracts cancelled entirely.
But there is an opportunity here if you flip the perspective.
Organizations that set up permission aware rag correctly
will have a massive advantage in regulated industries like finance and healthcare.
These sectors need AI but they have to use it safely.
The companies that solve the safety problem first will become the only trusted partners in the room
while everyone else will just look reckless.
Have you built a system or a vulnerability?
There is one specific question that will haunt your organization if you get this wrong.
If you take every document from your share point and build a technically perfect rag system
that is fast and accurate, but you lose the permission model along the way,
what have you actually built? Have you created a powerful knowledge system?
Or have you just constructed the biggest insider data leak in the history of your company?
For most organizations, the uncomfortable answer is the second one.
Let's look at how this plays out in a real world scenario.
Imagine a company with 100,000 employees and 10 million documents stored in SharePoint.
Every single one of those documents has a permission model attached to it.
Summer for everyone, summer for specific departments and summer locked away for executives or auditors.
SharePoint enforces these rules every single day.
And it works perfectly. Nobody ever sees a file they aren't supposed to see.
Then the organization decides to build a rag system to unlock the value of all that data.
The UNDEX all 10 million documents, turn them into embeddings and store them in a vector database.
The system is incredibly fast.
A junior employee can ask a complex question and get a perfect answer in two seconds.
The retrieval is flawless, but the permissions have become completely irrelevant.
The trouble starts when that junior employee asks the system about the future strategy of the company.
The rag system finds the most relevant chunks of information, which happened to come from private executive strategy documents.
Because the permission model was stripped away during the ingestion process, the vector database has no idea that this information is confidential.
It serves the answer up anyway.
Now that junior employee is reading about unreleased products, acquisition targets and upcoming layoffs.
This was information that was never supposed to cross that boundary.
The legal reality here is very clear. This isn't just a minor technical flaw.
It is a form of negligence. You had a working permission model and you chose to throw it away.
You built a system that you knew would leak secrets if the wrong person asked the right question.
When this eventually comes to light, regulators are going to ask what you did to prevent it.
Telling them you prioritized speedover security will not be an acceptable answer.
The fallout usually follows a very predictable pattern.
Someone discovers the leak when a contractor mentions a secret they shouldn't know, or a compliance ordered flags an unusual query.
Once the leak is found, the response is immediate and painful.
The system gets shut down instantly. Every single query has to be audited to see who saw what.
And external legal counsel has to get involved. The damage to your reputation is often permanent.
We built an AI that leaked our secrets, becomes the only thing people remember about your company.
Investors and customers will start to wonder if you are competent enough to handle their data at all.
This damage goes far beyond one system. It destroys the trust people have in your entire approach to technology.
The alternative is to just build it the right way from the start.
You have to materialize permissions at the moment you ingest data.
You need to capture the access list from SharePoint and encode them directly into your metadata.
This allows you to filter results at the moment of the query and monitor the system for any strange behavior.
It will cost more money, it will take more time, and it will require expertise you might not have yet.
But the cost of doing it wrong is much higher. A CISO at a major bank told us that they spend $600,000 to build their permission-aware system correctly.
Their competitor tried to save money by spending only $200,000 on a basic rag setup.
When that competitor suffered a major breach, they ended up spending $10 million on fines and incident response.
Investing early wasn't just a security choice. It was the only logical financial move.
That isn't a rare story. It is the pattern we see everywhere.
For CISOs, architects, and teams.
Let's bring this down to what it actually means for the people who have to build and operate this.
For a CISO, permission-aware rag stops being optional.
The moment you deploy any agentic system that touches internal data.
This isn't a recommendation or a best practice you can defer, but a hard compliance requirement that your organization must meet.
Your data governance framework exists specifically to enforce who can see what, and when you build a system that bypasses that framework,
you're creating a massive control failure. Regulatory frameworks like GDPR, HIPAA, and SOX all assume your organization maintains a coherent access control model across every system, touching regulated data.
A rag system that ignores permissions violates that core assumption, which puts the entire company at risk.
The governance model has to shift because you can't review AI systems one at a time in isolation anymore.
You need an AI governance committee that includes security, privacy, legal, and IT leadership to oversee these projects.
Permission models must be reviewed before any rag system goes into production.
And this has to happen as a requirement rather than an afterthought.
That committee needs to ask the hard questions about where permissions come from and how they stay in sync with the index.
They need to know if a policy engine is enforcing access at query time, and if you can audit every single retrieval to prove who saw what.
If the answer to any of those questions is that you'll figure it out later, the system simply does not go live.
A SISO at a major bank put it directly when he said they treat rag permission trimming the same way they treat database access control.
It's non-negotiable, and that is the bar every serious organization needs to hit.
For architects, this changes what a rag architecture actually is at its core.
It's no longer just a retrieval system with a language model bolted on, but a security architecture that happens to use retrieval and language generation.
Your design must answer three specific questions to be viable.
First, you have to show how identity propagates from the user through every single agent in the chain.
Second, you must define where authorization is enforced and ensure it stays deterministic and outside the model itself.
Third, you need to separate responsibility between agents so that a compromise in one doesn't automatically grant access to all your data.
Every rag system must answer those questions before a design review even begins.
The architecture diagram shouldn't just show how data flows, but it needs to show exactly how identity and authorization boundaries work.
It has to identify which components are trusted to make access decisions and which are not, which is fundamentally different from how most rag systems are designed today.
For engineers, the skill set changes entirely because you can't build permission-aware rag with just ML engineers and database experts.
You need engineers who deeply understand identity systems like OAuth, SAML, and token exchange to manage credentials properly.
You need people who can work with complex authorization frameworks and find great access control without breaking the system.
This isn't something you learn from a quick tutorial, but something you pick up by working in high stakes environments like banking or health care where failures have real consequences.
For product teams, permission-aware retrieval stops being a security concern and actually becomes a competitive feature.
It's not a bug fix or a technical debt item, but a specific capability that customers will start asking for during the sales process.
Questions about whether contractors can see sensitive documents or if junior employees are being overexposed will become standard during customer due diligence.
Teams that build this capability as a first-class feature will have a massive market advantage, while teams that treat it as an afterthought will lose deals to competitors who got it right.
For organizations, this is a strategic decision about whether you want to deploy a rag safely or become a cautionary tale for the rest of the industry.
Organizations that implement permission-aware rag correctly will have a huge advantage in regulated sectors like finance, health care, and government.
These sectors need the productivity gains that AI offers, but they absolutely must have them safely.
The vendors who solve this problem first will own those markets while everyone else will spend the next three years trying to retrofit security onto systems that will never build for it.
That's the reality.
Why this matters beyond security?
Permission models aren't just a security constraint you have to enforce.
There are business assets you can actually use to make your systems better.
Most organizations think about access control as friction or a compliance requirement that slows down productivity to reduce risk.
Permission-aware rag inverts that thinking entirely.
The permission model becomes the foundation for serving different audiences from the same knowledge base, which makes the system more valuable rather than more frustrating.
Start with a simple observation that different roles in your company need different types of data to do their jobs.
A sales team needs customer information and transaction history, while a legal team needs contracts and regulatory letters to stay compliant.
An executive team needs strategy and financial forecasts, and in a traditional setup you'd solve this by building three separate systems.
Sales would have their CRM, legal would have their document manager, and executives would have their dashboards.
That means three systems, three permission models, and three sets of redundant data to maintain.
Rage with permission enforcement changes this equation by allowing you to build one system with one index and one knowledge base.
The system is designed so that a sales rep queering it gets customer focused results, while a lawyer gets contract precedents, and an executive gets strategy.
It's the same underlying index, but the system provides different views and permissions scopes for each person.
You end up with one system serving multiple audiences without ever overexposing sensitive information to the wrong people.
This unlocks a level of operational efficiency that basic rag can't touch, you aren't maintaining three separate systems or duplicating data across multiple platforms, which makes the whole thing cheaper to operate.
You're managing one permission model instead of three, and that is much simpler to maintain and far more reliable as a single source of truth.
A financial services firm implemented this pattern by building a single rag system to serve advisors, compliance officers, and executives from one base.
The advisors can query customer portfolios and transaction history, while the compliance officers look at regulatory files and policy documents.
The executives can access strategy documents and performance metrics, and because the system handles permissions scopes, everyone sees exactly what they need.
Nobody sees what they don't, and the firm doesn't have to manage a dozen different AI silos.
Now zoom out for a moment, organizations that understand their permission models deeply can answer questions they currently can't even ask.
When you ask what data only five people can see, it sounds like a security question, but it's actually a vital business question.
Maybe you have a critical process that only five people know about, which means you have a massive bottleneck in your operations.
Maybe your institutional knowledge is too concentrated in one spot, and permission metadata is the only thing that reveals these dangerous patterns.
The same logic applies when you look for documents that are overshared across the company.
You might think a document is confidential, but if your permission model says it's visible to 200 people, you have a problem.
Maybe the permissions are stale or were set incorrectly, but the question itself forces an audit of what's actually happening versus what you think is happening.
Permission models also reveal your true organizational structure, and where decision-making authority actually sits.
The pattern of who can access what exposes how information flows through your teams, and where the real power is concentrated.
This isn't about surveillance, but about pattern recognition that makes your organization silos and collaboration points visible and actionable.
The innovation opportunities where this gets really interesting, once you have permission aware retrieval, you can build features that a basic RAC system could never handle.
You could show employees documents they don't have access to yet, but should request based on their specific role.
This turns the AI into a career development tool where junior employees can see what senior staff know and ask for access to learn.
You can even use it to find experts in the organization by looking at who has access to specific high level information.
The system doesn't just retrieve documents anymore, it connects you to people and builds relationships based on how information is used.
Permission aware RAC stops being a compliance checkbox at that point, it becomes the actual infrastructure for how your organization works.
Where this is heading. The future of this space isn't a mystery, and the direction is actually very clear.
Zero trust is going to become the baseline architecture for every agentic RAC system you build.
This isn't just a specialized pattern for high-risk environments or an option you implement when you finally have the budget.
It is the default, it is the minimum bar for entry. If we look at the evolution trajectory, most organizations are currently sitting at step one.
They are building single agent RAC with almost no access control because they want to move fast and deliver value quickly.
In this stage, permission models aren't even on the radar.
But by 2026, we are going to see a massive migration wave as these organizations realize they have a serious problem.
They will start upgrading to multi-agent RAC with basic access control where permissions are tracked and identity begins to propagate.
It won't be perfect, but it will be a significant step up from where they are now.
By 2028, the leaders who got this right will be running fully zero trust agentic RAC with continuous verification.
Every single hop will be authenticated, every access decision will be logged, and every anomaly will be detected.
That is the final endpoint. The organizations that refuse to make this transition are going to hit regulatory pressure first.
Compliance frameworks will eventually mandate these security measures and privacy regulators will look at your agentic systems with very specific questions.
They will demand that you prove your agents aren't accessing data they shouldn't be touching.
If you can't provide that proof, you are non-compliant.
This isn't just a friendly recommendation from an auditor. It is a legal requirement that carries real weight.
Once the regulators move in, market disadvantage follows right behind them.
Customers in regulated industries like finance, healthcare and government will start asking their vendors if their agentic systems enforce strict access control.
They will want to see the proof.
The vendors who can answer yes will win those deals, while the vendors who say they are still working on it will simply lose.
The market has a way of sorting itself out like that. On the technical side, identity systems are moving fast.
Workload identity is becoming standard infrastructure, which means tokens are getting shorter life spans and much narrower scopes.
Find great authorization is moving out of specialized services and straight into core platforms.
By 2028, you will expect your cloud provider to natively support per agent identity and per request authorization.
This won't be a custom integration you have to build yourself. It will be a built in primitive that just works inside the company.
AI governance committees will become a standard part of the enterprise.
They won't be optional anymore. They will be just as common as cybersecurity governance is today.
Security reviews for AI systems will become as rigorous as any infrastructure deployment review you've ever been through.
You won't be allowed to ship an agent without showing that it has the proper authorization controls in place.
These standards will start with the most security conscious organizations and then they will become standard industry practice.
Eventually, the regulators will just bake them into the official requirements. The market itself is shifting its focus.
Right now, vendors are competing on speed and accuracy by claiming their rag system is faster or their agent is more capable.
By 2028, that differentiation shifts towards security and governance.
Companies will win by proving their system prevents cross-tenant leakage or by showing they enforce fine-grained authorization.
Building on zero-trust principles won't be a secondary feature. It will be the primary value proposition.
This timeline matters because you need to move right now. If you start building permission-aware rag today,
you will have a mature system ready by 2026. But if you wait until the regulations force your hand, you are already two years behind the curve.
You'll be stuck retrofitting security onto legacy systems and fighting technical debt while trying to hit a compliance deadline.
The organizations that understand this shift and invest properly will be the leaders by 20, 27 and 2028.
Everyone else will just be trying to catch up. None of this is theoretical because it is already happening.
Early adopters are reporting success, mid-market organizations are planning their migrations and vendors are building much better tooling.
The momentum is visible to anyone looking. By the time the regulations officially mandate these changes in 2027, this will already be the industry standard.
The one question that changes everything.
Let's go back to that listener comment that started this entire conversation.
Are you carrying each file's permissions into the index and filtering per user at query time?
That one question is the divider. It separates the systems that are actually secure from the ones that just look secure on the surface.
It marks the difference between a system that survives a regulatory audit and one that fails it completely.
It is the line between protecting your data and exposing it to the world.
This entire episode has been about finding the answer to that question.
If your answer is no and you're building rag systems without permission-aware retrieval, then you have a lot of work to do.
Data sovereignty without security is just an illusion. Where your data actually sits matters much less than who is allowed to access it.
Permission-aware rag isn't an optional add-on. It is the foundation of the whole system.
Start asking this question about your own systems today.
If this changed how you think, follow me, mycopieters, or linked info more.
Subscribe to M365FM for more deep dives on systems that actually work and leave a review to help others find this.
Share this with your team, especially if you're dealing with these architecture flaws right now.