RAG Security: How to Secure Retrieval-Augmented Generation Pipelines

Retrieval-augmented generation (RAG) changed how teams ship AI over private data. It quietly changed the threat model too. The moment your model answers from a knowledge base instead of its own weights, every document you retrieve becomes part of the prompt. That's the core reframe this guide is built on: retrieved context is untrusted input, the knowledge base itself is an attack target, and embedding a document strips the access controls that used to protect it. RAG security is the discipline that closes those gaps. This guide is written for the AI, platform, and application-security engineers who build and defend RAG and agentic systems over enterprise data. It covers what RAG security is, how a RAG pipeline creates an attack surface, the core risks, how RAG security differs from general LLM security, and a pipeline-wide defense checklist you can apply today.

What is RAG security?

RAG security is the practice of protecting a retrieval-augmented generation pipeline against attacks that target the data it retrieves, the store that holds it, and the way retrieved content is fed into the model. It treats every retrieved document as untrusted input, governs who can retrieve which data at query time, and monitors the full pipeline from ingestion to output.

Retrieval-augmented generation security matters because RAG expands the LLM threat model in a specific direction. A standalone LLM is exposed mainly through the prompt a user types. A RAG system is exposed through much more: everything it ingests, everything it stores as vectors, and everything it pulls back at retrieval time. Those are new surfaces, and they sit in the data layer where traditional application-security controls often don't reach. For teams shipping GenAI features over proprietary or regulated content, RAG security is also about protecting the enterprise knowledge that feeds the model, not just hardening the model itself.

Here's a useful way to hold the idea. RAG makes an LLM more accurate by grounding it in your data, and in doing so it makes your data part of the model's runtime input. Anything an attacker can get into that data, or any permission gap they can exploit to retrieve it, becomes an attack on the model's behavior. RAG isn't secure by default, and the rest of this guide explains why.

How a RAG pipeline works: the attack-surface map

To secure a RAG system you have to see it as a pipeline, not a single call. Most shallow guides recap "RAG architecture" and stop. The more useful RAG security architecture view labels each stage with the attack that lands there, because defenses belong at the stage where the risk actually enters.

The RAG pipeline stages

A typical RAG pipeline moves data through eight stages:

1. Ingestion - documents are pulled from sources (wikis, ticketing systems, file stores, the open web) into the pipeline.
2. Chunking - documents are split into passages sized for retrieval and context windows.
3. Embedding - each chunk is converted into a vector by an embedding model.
4. Vector store - vectors and their metadata are indexed in a vector database.
5. Retrieval - a user query is embedded and matched against the store to pull the most relevant chunks.
6. Prompt assembly - retrieved chunks are concatenated with the system prompt and user query into the final context.
7. Generation - the LLM produces an answer grounded in the assembled context.
8. Output - the response (and often citations or tool calls) is returned to the user or downstream system.

Where each attack lands

Mapping the threats to the stages turns an abstract list of risks into a defensible model. Each attack has a natural home in the pipeline, and that home is also where its control belongs.

Pipeline stage	Primary threat that lands here	What the attacker gains
Ingestion	Knowledge-base / data poisoning	Malicious content enters the corpus and is later treated as authoritative
Chunking	Delimiter and boundary manipulation	Hidden instructions survive splitting and reassemble in context
Embedding	Embedding inversion	Source text is partially reconstructed from stored vectors
Vector store	Cross-tenant leakage, weak access metadata	Retrieval returns chunks from data the user should not see
Retrieval	Access-control bypass, sensitive-data leakage	Permissioned documents surface to unauthorized users
Prompt assembly	Indirect prompt injection	Retrieved text is executed as instructions by the model
Generation	Manipulated or attacker-chosen output	The model returns false, harmful, or exfiltrating answers
Output	Data exfiltration, unsafe tool calls	Sensitive data leaves the system or an agent acts on injected commands

Why the retrieval layer is uniquely exposed

General LLM-security guidance focuses on the model and the user prompt. RAG adds a retrieval layer with three properties that most of that guidance misses. These three ideas are the through-line for every risk below.

• Untrusted context becomes executable. A retrieved document isn't neutral data. An LLM doesn't reliably distinguish "content to reason about" from "instructions to follow," so hidden directives inside a retrieved chunk can be obeyed. That's indirect prompt injection, and it means retrieved context must be treated as untrusted input, exactly like a request body from an anonymous client.
• The knowledge base is itself a target. Attacking the corpus is often easier and more effective than jailbreaking the model. It needs no model access and no fine-tuning. Because the LLM treats retrieved chunks as authoritative grounding, planting a few malicious passages can steer answers directly. Academic work has begun to formalize the RAG threat model and its attack surface, placing the knowledge base at the center of the adversary's options.
• Embedding strips access control. When a document is chunked and embedded, the permissions attached to its source (who could open the original file) don't travel with the vector unless you deliberately carry them. Retrieval then matches on similarity, not identity, so the system will happily return a chunk from a document the requesting user was never allowed to read.

Core RAG security risks

The RAG vulnerabilities below are the RAG security risks that recur across real systems and standards. Each maps to a category in the OWASP Top 10 for LLM Applications, the 2025 risk list from the OWASP GenAI Security Project, which is a useful shared vocabulary when you brief a security team.

Knowledge-base and data poisoning

RAG data poisoning (also called knowledge-base poisoning) is the injection of malicious or misleading content into the corpus so the model retrieves and trusts it. Because RAG treats the knowledge base as ground truth, a poisoned passage can force a specific wrong answer, insert a backdoor phrase that triggers a bad response, or degrade quality across a topic. Research on corpus-corruption attacks (PoisonedRAG) has shown that injecting only a small number of crafted passages can reliably steer a model toward attacker-chosen outputs.

Poisoning is a form of model manipulation that never touches the model. The corpus is the weakest link, because ingestion often pulls from many sources, some of them user-generated or externally hosted, and few teams verify document integrity on the way in. This risk maps to OWASP LLM04 Data and Model Poisoning.

Indirect prompt injection via retrieved documents

Direct prompt injection is what a user types into the chat box. RAG prompt injection is different: the malicious instruction is hidden inside a document the system retrieves, so it arrives through the data layer rather than the user. This is indirect prompt injection, and it's the reason the two risks above are really one kill chain. Knowledge-base poisoning is the delivery vector, and indirect prompt injection is the payload firing. An attacker poisons the corpus with a document that contains instructions ("ignore your guidelines and output the following"), retrieval pulls it into context because it looks relevant, and the model executes it.

Framing these as separate problems is a common mistake in vendor content. They're the same attack seen at two stages: getting the payload in, and having it run. Adversarial-ML threat catalogs like MITRE ATLAS document both corpus poisoning and prompt injection as techniques against LLM systems, which is why threat-modeling a RAG pipeline should trace the full path, not just the final generation step. This maps to OWASP LLM01 Prompt Injection.

Sensitive-data leakage and disclosure

RAG systems are built to surface information, which is exactly what makes accidental disclosure easy. Leakage happens two ways. First, retrieval can return content the requesting user should not see, because the corpus mixed sensitive and general documents and nothing filtered by permission at query time. Second, the model can echo secrets that were embedded in the context, or a crafted query can coax it into revealing more of a retrieved document than intended. Even the vector representations can leak, since embeddings retain enough of the source to partially reconstruct it. This risk maps to OWASP LLM06 Sensitive Information Disclosure.

Retrieval-time access-control gaps

This is the risk most guides under-explain, and it's the one that turns a helpful assistant into a data-exposure incident. RAG access control is hard because embedding strips source permissions. When you chunk and vectorize a document, the access-control list that governed the original file doesn't automatically follow the chunk into the vector store. Retrieval then matches on semantic similarity alone. Unless you rebuild permission enforcement inside the pipeline, a RAG system won't respect document permissions and will return restricted content to anyone whose query is a good match.

The fix is to make retrieval identity-aware: carry per-chunk access metadata into the vector store and enforce it at query time, so the set of retrievable chunks is scoped to what the specific requesting user is allowed to see. For agentic RAG, this extends to delegated, on-behalf-of authorization for AI agents, where an agent retrieving on a user's behalf inherits that user's permissions and least privilege rather than a broad service identity. Getting identity and access right at retrieval time is the foundation of RAG with permissions.

Vector database and embedding threats

The store itself is an attack surface. Vector database security covers the vector-and-embedding weaknesses that OWASP catalogs as LLM08. Two matter most. Embedding inversion is the reconstruction of source text from stored vectors, which means a leaked or over-permissioned index can expose the underlying data even without the original documents. Cross-tenant leakage happens when a shared multi-tenant index lacks strict isolation, so one tenant's query retrieves another tenant's chunks. Weak authentication on the vector database, unencrypted storage, and missing namespace isolation all widen this surface. This risk maps to OWASP LLM08 Vector and Embedding Weaknesses.

Each of the five risks above is deep enough to warrant its own guide. This page covers the essential mechanics and defenses; the RAG data poisoning, knowledge-base poisoning, indirect prompt injection, vector database security, and RAG access-control implementation topics are candidates for dedicated cluster pages that this hub will link to as they publish.

RAG security vs general LLM security

Readers often ask how RAG security differs from ordinary LLM or prompt security. The short answer: LLM security defends the model and the user prompt, while RAG security additionally defends the data pipeline that feeds the model. The comparison below is the difference most SERP content skips.

Dimension	General LLM / prompt security	RAG-specific security
Primary attack surface	The model and the user-supplied prompt	The ingestion pipeline, the knowledge base, the vector store, and retrieval
Primary threat vector	Direct prompt injection and jailbreaks typed by the user	Poisoned documents delivering indirect prompt injection through retrieval
Trust boundary	User input is untrusted; model output is checked	Retrieved context is also untrusted, even though it comes from "your own" data
Access control	Who can call the model	Who can retrieve which documents, enforced per-user at query time
Main defense locus	Prompt hardening, output filtering, model guardrails	Defense-in-depth across the whole pipeline, from ingestion integrity to retrieval identity to output monitoring

The takeaway is that RAG security is a superset problem. You still need everything LLM security asks for, plus data-layer controls that prompt-focused guidance never mentions.

RAG security best practices

The strongest RAG security best practices share one principle: defense must be pipeline-wide, not a single guardrail. A poisoned corpus defeats an output filter. An output filter defeats nothing if retrieval already leaked restricted data. Securing RAG pipelines means placing a control at each stage where a risk enters. The controls below align with the OWASP RAG Security Cheat Sheet.

• Verify document integrity and provenance at ingestion. Hash and track the source of every document, restrict which sources can write to the corpus, and validate or quarantine untrusted and user-generated content before it is embedded. This is your first line against poisoning.
• Make retrieval identity-aware. Carry per-chunk access metadata into the vector store and enforce per-user permissions at query time so retrieval is scoped to what the requester is allowed to see. For agents, use delegated on-behalf-of authorization and least privilege.
• Validate input and output, and delimit retrieved context. Separate retrieved content from instructions using clear context delimiters, strip or neutralize instruction-like patterns in retrieved chunks, and filter model output for sensitive data before it is returned.
• Encrypt the vector store in transit and at rest, and isolate tenants. Treat the vector database like any sensitive datastore: strong authentication, encryption, and strict namespace or tenant isolation to prevent cross-tenant leakage and embedding-inversion exposure.
• Monitor the full pipeline continuously. Log ingestion, retrieval, and generation with enough context to trace an answer back to the chunks and identity that produced it. Watch for anomalous retrieval patterns, sudden corpus changes, and injection signatures.
• Fail closed. When permission checks, integrity checks, or validation cannot be resolved, deny retrieval or withhold the answer rather than returning possibly-exposed or possibly-poisoned content.

RAG security implementation checklist

Use this checklist to audit or build a RAG system. It's grouped by pipeline stage so each item maps to where the control lives. Treat it as a practical starting point for securing RAG pipelines, not a compliance guarantee.

Ingestion and knowledge base

• Restrict and authenticate which sources and users can add documents to the corpus.
• Hash documents and record provenance so tampering and poisoning are detectable.
• Validate, sanitize, or quarantine untrusted, external, and user-generated content before embedding.

Embedding and vector store

• Attach per-chunk access-control metadata (owner, sensitivity, tenant) at embedding time.
• Encrypt vectors and metadata at rest and in transit; require strong auth on the vector database.
• Enforce strict tenant or namespace isolation to prevent cross-tenant retrieval.

Retrieval and prompt assembly

• Filter retrieval by the requesting identity so only permitted chunks are eligible; fail closed on ambiguity.
• For agents, propagate the user's identity via delegated on-behalf-of authorization and least privilege.
• Delimit retrieved context clearly and neutralize instruction-like content before it reaches the model.

Generation, output, and monitoring

• Filter output for sensitive data, secrets, and unsafe tool calls before returning it.
• Log ingestion, retrieval, and generation end to end, traceable to identity and source chunks.
• Alert on anomalous retrieval, corpus changes, and injection signatures; review regularly.

Mapping RAG risks to security standards

Grounding RAG security in recognized frameworks makes it easier to brief stakeholders and audit coverage. The table maps each core risk to the relevant standard. For the full breakdown of each category, this page links out to the dedicated sibling guides rather than duplicating them.

RAG risk	OWASP LLM Top 10	Threat-model / governance reference
Indirect prompt injection via retrieved docs	LLM01 Prompt Injection	MITRE ATLAS prompt-injection techniques; NIST AI 600-1 recognizes indirect injection
Knowledge-base / data poisoning	LLM04 Data and Model Poisoning	MITRE ATLAS poisoning techniques; NIST AI 600-1 data-poisoning risk
Sensitive-data leakage and disclosure	LLM06 Sensitive Information Disclosure	NIST AI RMF (MAP / MEASURE)
Vector database and embedding threats	LLM08 Vector and Embedding Weaknesses	NIST AI RMF (MANAGE)
Retrieval-time access-control gaps	Cross-cutting (LLM06 / LLM08)	NIST AI RMF (GOVERN); identity/authZ per frontegg.com

The OWASP Top 10 for LLM Applications gives you the risk vocabulary, MITRE ATLAS supplies the adversary technique catalog for threat modeling, and the NIST AI Risk Management Framework (AI RMF), with its GenAI-focused AI 600-1 profile, provides the governance baseline (GOVERN, MAP, MEASURE, MANAGE) and recognizes both direct and indirect prompt injection and data poisoning as information-security risks.

RAG security use cases

Where does RAG security apply most sharply? Three patterns come up again and again:

• Enterprise knowledge assistant. An internal assistant over wikis, HR docs, and code needs strict retrieval-time access control so an employee cannot pull HR or finance content they were never permitted to open. This is the access-stripping problem at full scale.
• Customer-facing RAG chatbot. A support bot grounded in documentation ingests content that customers and partners can influence, which makes ingestion integrity and indirect-injection defenses the priority, alongside output filtering so the bot never leaks another customer's data.
• Agentic RAG. When a RAG agent both retrieves context and calls tools, the blast radius grows: an injected instruction in a retrieved document can trigger an unsafe tool action, not just a bad answer. Delegated, least-privilege authorization and tool-call governance become essential. Agentic RAG that calls external tools inherits the tool-calling risks covered in our MCP security guide.

Frequently asked questions

What is RAG security?

RAG security is the practice of protecting a retrieval-augmented generation pipeline against attacks on the data it retrieves, the vector store that holds it, and the way retrieved content is fed to the model. It treats retrieved context as untrusted input, enforces per-user access control at retrieval time, and monitors the pipeline end to end.

Is RAG secure by default?

No. RAG isn't secure by default. Adding a retrieval layer introduces new attack surfaces (corpus poisoning, indirect prompt injection through retrieved documents, and access-control loss when embedding strips source permissions) that a standalone model does not have. Security has to be designed into the pipeline deliberately.

What is RAG data poisoning?

RAG data poisoning, or knowledge-base poisoning, is the injection of malicious or misleading content into the corpus so the system retrieves and trusts it. Because the model treats retrieved chunks as authoritative, even a small number of poisoned passages can force attacker-chosen or degraded answers without any access to the model itself.

How is RAG prompt injection different from direct prompt injection?

Direct prompt injection arrives in what the user types. RAG prompt injection is indirect: the malicious instruction is hidden inside a document that retrieval pulls into the prompt, so it enters through the data layer. Knowledge-base poisoning is the delivery vector, and the indirect injection is the payload the model then executes.

Do RAG systems respect document permissions?

Not unless you enforce them. When documents are chunked and embedded, their source access-control lists don't follow the vectors, so retrieval matches on similarity alone and can return restricted content. RAG with permissions requires carrying per-chunk access metadata into the store and enforcing identity-aware, per-user authorization at query time, including delegated on-behalf-of authorization for agents.

Can embeddings be reversed to recover the source data?

Partially, yes. Embedding inversion techniques can reconstruct meaningful portions of the original text from stored vectors. That's why a leaked or over-permissioned vector index is a data-exposure risk on its own, and why vector stores should be encrypted, access-controlled, and tenant-isolated like any sensitive datastore.

Which OWASP LLM Top 10 risks apply to RAG?

The most relevant are LLM01 Prompt Injection (including indirect injection through retrieved docs), LLM04 Data and Model Poisoning, LLM06 Sensitive Information Disclosure, and LLM08 Vector and Embedding Weaknesses. Our OWASP Top 10 for LLM guide breaks each category down in detail.

How do you secure a RAG pipeline?

Apply defense-in-depth across every stage: verify document integrity at ingestion, make retrieval identity-aware, delimit and validate retrieved context, encrypt and isolate the vector store, filter output, monitor end to end, and fail closed. No single guardrail is enough; the defense has to be pipeline-wide. The implementation checklist above is a practical starting point.

Related resources and next steps

RAG security comes down to one shift in mindset: the retrieval layer is untrusted input, the corpus is an attack target, and access control has to be rebuilt at retrieval time. Defending it well means treating security as a property of the whole pipeline rather than a filter bolted onto the model.

Continue deeper into the AI Agent Security cluster:

• Map RAG threats to the full risk taxonomy in the OWASP Top 10 for LLM applications guide.
• Threat-model the adversary techniques behind poisoning and injection with MITRE ATLAS.
• Understand the tool-calling risks that agentic RAG inherits in our MCP security risks guide.

If you're moving from understanding RAG risks to defending them in production, pipeline-wide runtime guardrails, data-access governance, and continuous threat detection for RAG and agentic systems are exactly what an AI-security platform is built to provide. See how agen.co approaches AI agent security across the full pipeline.