RAG Security Assessment: Tenant Isolation, Data Leakage, and Prompt Injection Risks
How to review RAG systems for authorization failures, tenant-isolation gaps, prompt injection, vector-store leakage, document poisoning, and audit logging.
Start a RAG assessment with the documents selected for the user, before looking at the generated answer. A plausible answer can conceal an unauthorized retrieval, and an empty answer does not undo content already sent to a model provider.
The boundary includes source permissions, index metadata, retrieval filters, context assembly, citations, caches and logs. A user should receive only material they are currently authorized to access. That policy needs to work when the search is broad, the index is stale, or an uploaded document contains hostile instructions.
OWASP's 2025 vector and embedding guidance describes access-control and poisoning concerns. The practical review below turns those concerns into document-level checks and a runnable synthetic walkthrough.
Follow the document into the answer
The visible answer is the last part of a longer data path:
User query
-> embedding / search
-> vector database or document index
-> retrieved chunks
-> LLM context window
-> generated answer
Inspect what crosses each connection. Retrieval can select another tenant's document; context assembly can release it to a provider; a citation or trace viewer can expose its title even when the answer hides the text. Review the retrieval service as a data-access API, then follow generation and derived copies.
Distinguish the failure you observed
An authorization finding means content reached someone or something outside the permitted access path. A poisoning finding means source text influenced behavior. A traceability gap means the team cannot establish which records or operations were involved. One test can reveal more than one failure.
State those observations separately. A hostile document retrieved with permission is different from a cross-tenant disclosure, and a model's attempted send is different from a completed transfer.
1. Tenant Isolation Must Happen Before Model Context
Authorize retrieved content before it enters model context.
Bad pattern:
Retrieve broadly -> put chunks in prompt -> tell model not to reveal unauthorized data
Better pattern:
Authenticate user -> resolve tenant/role/resource permissions -> retrieve only authorized chunks -> preserve source metadata -> generate answer
The model should never be responsible for deciding whether the user is allowed to see retrieved content. That decision belongs in deterministic authorization logic.
| Control | Review Question |
|---|---|
| Tenant filtering | Is every retrieval constrained to the user's tenant or organization? |
| Role filtering | Does retrieval respect the user's role and privileges? |
| Resource-level ACLs | Are document permissions enforced before content or metadata leaves the protected retrieval service? |
| Metadata integrity | Can metadata be spoofed, omitted, or overwritten? |
| Server-side enforcement | Are filters enforced by backend logic, not just prompt text? |
| Cross-tenant testing | Has the system been tested for leakage across tenants or workspaces? |
2. Vector Stores Need Security Boundaries
Vector search is optimized for relevance, not security. The most semantically relevant chunk may not be the chunk the user is authorized to see.
A RAG assessment should review:
| Area | Security Concern |
|---|---|
| Index design | Are tenants separated physically, logically, or only by metadata? |
| Embedding metadata | Is authorization metadata preserved with every chunk? |
| Chunk lineage | Can each chunk be traced back to source document, owner, version, and access policy? |
| Query filters | Are filters mandatory and server-side enforced? |
| Similarity thresholds | Can broad queries pull unexpected sensitive context? |
| Re-indexing | Are permissions re-evaluated when documents or users change? |
| Deletion | Does removing source access remove retrieval access? |
A vector database should be treated as a security-sensitive data store. It may not store raw documents, but embeddings, metadata, source identifiers, and chunk text can still reveal sensitive information.
3. Test hostile content through storage
Put an inert instruction in a source the application normally ingests, then follow parsing, indexing and retrieval. The instruction may be in a PDF, ticket, wiki page or uploaded file. Record who controls that source and which parts reach model context.
An approved internal location does not grant every sentence authority. Source text remains data; it cannot authorize a tool call. Preserve origin and version information, test the model's behavior separately, and enforce any resulting action at the tool boundary.
Because the text is stored, repeat after a new session and after removing the source. Check whether chunks, summaries or caches keep the instruction available.
4. Preserve the source and permission decision
Retain the source document and chunk IDs, tenant, source version, permission version and retrieval time. Include classification and retrieval scores where they explain the decision. Check that the mapping survives reranking and context assembly.
A citation needs its own access check when opened. Otherwise a denied body can still leak a confidential title or URL. Source references also let an operator find stale or derived content after a document is corrected or revoked.
5. Inspect summaries and citations for disclosure
Ask for a broad account summary, a comparison with similar customers and the exact source language. Check which records contributed to each answer. A summary, generated table or paraphrase can disclose protected content without returning the original document.
Include citation previews, error messages and answer caches in the observations. An empty final answer cannot undo content already sent to an external processor.
6. RAG Systems Need Negative Tests
Test permitted retrieval and requests that must be denied.
| Test | Expected Result |
|---|---|
| User from Tenant A asks about Tenant B | No retrieval and no summary. |
| Low-privilege user asks about privileged document | No retrieval and no metadata leak. |
| User asks for "similar customers" | Aggregation respects authorization. |
| Retrieved document contains hostile instructions | Record model behavior separately; source text cannot grant permission and unauthorized tool requests are rejected. |
| Deleted document is queried | No retrieval after access revocation. |
| User asks for hidden source text | No unauthorized reconstruction. |
| Search query omits tenant filter | Backend rejects or constrains request. |
| Conflicting documents are retrieved | System surfaces uncertainty and source conflict. |
For every case, record the exact context and metadata returned, the policy result, and the tested configuration. A refusal sentence alone is insufficient if unauthorized chunks already entered context.
A synthetic two-tenant walkthrough
The RAG boundary reference program, version 1.0 dated September 9, 2026, uses three embedded documents. All names, identifiers, domains and marker strings are synthetic. The program uses Python's standard library; it makes no API calls and does not load a model or create embeddings.
| Fixture | Tenant and reader | Content purpose |
|---|---|---|
A-1 | Amber, Maya | Maintenance text with AMBER-CANARY-314 |
B-1 | Birch, Ben | A different maintenance interval with BIRCH-CANARY-927 |
A-2 | Amber, Maya | An uploaded note containing an inert instruction to send documents to collector@example.net |
Maya and Ben are authenticated fixture subjects. A query is literal search text. It cannot select a tenant. The retriever returns only permitted candidate IDs and versions, and the context builder reloads current ACLs before releasing document text. The citation renderer uses the same authorization path for titles and source URLs.
Download the program and run:
python3 rag_boundaries.py
The September 9 local run passed 8 tests. The sequence is useful even if the real application uses a completely different search backend:
- Establish positive controls. Maya can retrieve Amber's marker and Ben can retrieve Birch's. A test that returns nothing for everyone would conceal a broken search implementation.
- Try cross-tenant retrieval in both directions. Searching for the other tenant's exact marker returns no candidate. Broader shared-word searches still stay inside the subject's permissions.
- Bypass the first filter in the test harness. Hand a Birch candidate ID to Maya's context builder. The independent ACL check returns no text. This represents a stale cache or defective upstream retriever.
- Retrieve the hostile note. Its instruction remains literal text labelled
untrusted_source. A syntheticsend_documentsproposal receivestool_not_available; this read-only workflow exposes no sending capability. The test does not ask a model to resist the instruction. - Revoke access after search. Clear Maya's ACL on
A-1, then reuse the earlier candidate. Both context and citations are empty. Repeating only the initial search would miss this stale-result path. - Check citation leakage independently. Submit mixed Amber and Birch candidates. Maya receives Amber's citation only; Birch's title and URL are absent. A hidden document body with a visible confidential title would still be a disclosure.
- Change and delete source records. Reject a stale version and a deleted document even when the candidate still exists in memory.
The marker strings make accidental cross-tenant content easier to spot. They do not measure all forms of disclosure. A production evaluation must also inspect paraphrases, summaries, answer caches, error messages, reranker inputs and user-visible source metadata.
The reference verifies its own deterministic policy and output construction. It cannot establish vector-store isolation, live ACL synchronization, provider retention, model resistance to injection, or safe behavior under concurrency. Labelling hostile text is demonstrated; a model obeying that label is not. These distinctions matter when reporting a passing run.
Every fixture is recreated in memory on each run. The scripts write no data files and contact no reserved example address. There is no test data to delete from a provider. You can remove the downloaded script when finished; see the reference README for the complete local setup and limits.
MSP document search needs a client boundary at every hop
For an MSP, substitute “client organization” for tenant. An operator who legitimately supports Amber and Birch should still choose an explicit client workspace for a customer-facing answer. A broad operator entitlement must not become permission to send Birch's notes to an Amber recipient.
Check intake routing, connector identity, chunk metadata, search filters, answer caches and citations using the same client identifier. An intentionally cross-client administrative search needs its own authorized audience and workflow. Do not silently reuse that broader path for a client portal. The local example covers isolated retrieval; it does not model multi-client operator delegation.
7. RAG Audit Logs Should Preserve Retrieval Decisions
If there is a RAG failure, the organization must reconstruct what happened.
| Field | Purpose |
|---|---|
| User ID | Who asked. |
| Tenant ID | Which boundary applied. |
| Role/permissions | Why access was allowed. |
| Query | What was requested. |
| Retrieval filters | What constraints were applied. |
| Retrieved chunk IDs | What entered model context. |
| Source document IDs | Where chunks came from. |
| Classification labels | Sensitivity of retrieved material. |
| Model response | What was shown. |
| Refusal/allow decision | Whether policy intervened. |
| Trace ID | Links all events together. |
Apply access controls to the audit store too. A cross-tenant document ID or sensitive query in a broadly accessible trace viewer can create a second disclosure path.
Tie each finding to the retrieval path
Link the architecture and data-flow map to the tested policy, identities and source versions. A finding should identify the failed boundary, returned material, affected path, correction and retest. Record gaps where the supplied environment could not exercise a connector, model or cache.
Keep a result from the local reference separate from an observation of the deployed application. Both are useful, but they support different claims.
Carry the checks into the real retrieval path
The local walkthrough is a starting point for test design. Run equivalent cases against the application's actual source connectors, index, reranker, context builder, model and citation renderer in an authorized test environment. Record versions and distinguish unauthorized retrieval, generated disclosure and blocked tool attempts.
Keep those cases when changing chunking, models, metadata or caches. A review of one retrieval endpoint does not cover another endpoint that uses different filters. The RAG boundary worksheet can hold the deployment-specific questions and owners.