What Is AI Security? Threats, Standards, and Controls

Security has spent thirty years protecting systems that store and move information. AI systems do something different. They decide, and increasingly they act on the decision without waiting for anyone.

That single change is what makes AI security its own discipline. A model that drafts an email is a content problem. A model that sends the email, pulls the customer record it referenced, and files a refund is an access problem. Most AI security programs today are still built for the first one.

What is AI security?

AI security is the practice of protecting AI systems and the organization that runs them: the models, the data those models consume and produce, and the actions the system is permitted to take. It spans the whole lifecycle, from training data and prompts through deployment to the moment a production agent calls a tool. The short AI security definition worth memorizing: it is access control for software that makes its own decisions.

That framing matters because it tells you which instincts to reuse. You already know how to handle a principal that can act: give it a name, scope what it is allowed to touch, log everything it does, and be able to revoke it in seconds. AI security is those same instincts applied to a new kind of principal, one that improvises.

Securing AI, using AI for security, and AI safety are three different jobs

The phrase gets used for at least three unrelated things, and the confusion is expensive. Teams buy a tool for one and assume they have covered the others.

Term	What it means	The question it answers
Securing AI (AI security)	Protecting AI models, data, and agent actions from attack and misuse	Can this system be manipulated, and what can it reach?
AI for security	Using machine learning inside security products for detection, triage, and response	Can AI make my SOC faster?
AI safety	Preventing harmful, biased, or unintended model behavior, including when no attacker exists	Does this system behave acceptably on its own?

Safety and security overlap but are not interchangeable. Security assumes an adversary. Safety does not require one: a well-intentioned agent that misreads an instruction and deletes a production table has caused a safety failure with a security-sized bill. The research community has needed entire papers to pin down the boundary, which tells you how blurred it is in practice, as recent research demarcating AI safety from AI security sets out.

This page is about securing AI.

Why AI security changed the day AI started acting

Retrieval-augmented chatbots were a data problem. Agentic AI is an authority problem. An agent plans a multi-step task, picks its own tools, and executes without a human approving each step, which means the interesting security question is no longer what the model said but what it was allowed to do.

Three things changed at once:

• Autonomy. Autonomous AI agents run loops, not single turns. One bad instruction can become forty actions before anyone notices.
• Reach. Tool protocols connected models to real systems. Understanding what AI agents are and how they work starts with the fact that an agent without tools is a chatbot, and an agent with tools has your CRM, your repo, and your payment rails.
• Volume. Agents get spun up per team, per workflow, sometimes per ticket. The population of things holding credentials in your environment grew faster than the process for issuing them.

Incident counts have followed the adoption curve, while the practice of evaluating these systems rigorously has not. Stanford's 2025 AI Index report puts it plainly: AI-related incidents are rising sharply, yet standardized responsible-AI evaluations remain rare among major industrial model developers. The gap between how fast teams ship agents and how fast they govern them is the working definition of the current risk.

The three layers of AI security

Almost every vendor glossary splits AI security into models and data. That split is incomplete. A third layer sits underneath, and it is the one that turns a bad output into an incident.

Layer	What lives here	Primary controls	Failure looks like
Model	The model, its system prompt, its context window, its inference endpoint	Input and output filtering, jailbreak detection, model provenance, rate limits	The model is talked into ignoring its instructions
Data	Training data, fine-tuning sets, retrieval corpora, generated outputs, logs	Classification, provenance, access scoping on sources, output redaction	Sensitive data ends up somewhere it was never approved to go
Action	Tool definitions, connected systems, credentials, the agent's identity	Named agent identity, per-action authorization, human approval gates, revocation, audit trail	A legitimate-looking request executes a change nobody authorized

The model layer

This is the layer the industry understands best. The dominant risk is instruction hijacking: text that reaches the model and is treated as a command rather than as content. Prompt injection has no complete fix, only layered mitigation, and OWASP ranks it LLM01, the number one risk for LLM applications.

Model-layer controls are mostly filters and detectors. AI guardrails sit on the input and output path and block what should not pass in either direction. They are necessary and they are not sufficient, because a filter that is 99 percent effective still lets one instruction through, and one instruction is enough when the model can act.

The data layer

Every AI system is a data pipeline wearing a conversational interface. Training sets can be poisoned, retrieval corpora can be seeded with attacker-controlled text, and outputs can carry data the requester was never entitled to see. RAG security is the specific discipline of treating retrieved context as untrusted input rather than as ground truth, which is the correct default.

The subtle failure here is entitlement drift. An agent inherits a broad service account, queries a knowledge base on a user's behalf, and returns a passage that user could never have opened directly. Nothing was breached. The data still leaked.

The action layer

Here is the part the glossaries skip. When an agent calls a tool, something real happens: a row updates, a ticket opens, money moves, a repository changes. The call arrives at the target system carrying a credential, and that credential usually belongs to an application, not to the agent and not to the person the agent is working for.

So when the audit question comes, and it always comes, the logs say a service account did it. That is the accountability gap at the center of AI security today. Closing it means the agent has an identity of its own, the human behind it stays attached to the request, and every action is authorized on its own merits rather than inherited from a long-lived token. The layered view of AI agent architecture is a useful companion here, since the action layer maps directly onto the tool-use components of an agent.

How AI security works across the lifecycle

AI security is not a gate at the end. It attaches at four points, and skipping any one of them pushes the cost to the next.

1. Design. Decide what the agent is for and, more usefully, what it must never do. Write the blast radius down before anyone writes a tool definition. AI AppSec gives you a threat-surface model to run this against.
2. Build. Scope every tool and data source to the minimum the task needs. Prefer narrow tools over general ones: an agent with a refund_order tool capped at a dollar amount is safer than the same agent with database write access.
3. Test. Attack it before production does. AI red teaming is adversarial testing built for non-deterministic systems, where a single passing run proves nothing and you need distributions rather than pass or fail.
4. Run. Authorize each action, watch behavior continuously, and keep an audit trail that survives the incident review. AI threat detection covers the runtime half: spotting a compromised or drifting agent while it is still acting.

The main AI security issues, layer by layer

The catalog below is the honest version. Some of these have good mitigations, some have partial ones, and one of them has none.

Issue	Layer	What goes wrong	Maturity of the fix
Prompt injection	Model	Untrusted text is executed as instruction	Partial. Layered mitigation only
Jailbreaking	Model	Guardrails are argued away by the user	Partial. Detection improves, never closes
Model and supply-chain poisoning	Model	A tampered model or dependency ships with a backdoor	Good. Provenance and signing work
Sensitive data disclosure	Data	Output includes data the requester cannot access	Good, if entitlements are enforced per request
Retrieval poisoning	Data	Attacker text is planted in the corpus the agent trusts	Moderate. Source hygiene plus output checks
Data exfiltration through tools	Action	An agent is steered into moving data out through a legitimate tool	Moderate. Needs egress-aware authorization
Tool poisoning	Action	Malicious instructions hide in a tool description the agent trusts	Moderate. Registry vetting and pinning
Excessive agency	Action	The agent can do far more than its task requires	Good, and almost always under-applied
Credential inheritance	Action	The agent acts as a shared service account, so nobody is accountable	Solvable. This is an identity problem
Shadow AI	Organizational	Agents and AI tools run outside any inventory	Solvable. Discovery first, policy second

Model-layer issues

Injection and jailbreaking are the same family: the model cannot reliably tell instruction from content because both arrive as text. Treat every mitigation as probabilistic and design the layers below to survive the one that gets through.

Data-layer issues

Exfiltration rarely looks like theft. It looks like an agent summarizing a document into a channel where the wrong people read it, or writing a record to a system with looser permissions than the source. MCP data exfiltration walks the specific vectors that open up once an agent has read access on one side and write access on the other.

Action-layer issues

This is where injection becomes incident. The attacker's goal is rarely a rude answer. It is a tool call. AI agent attacks catalogs the vectors that target an agent's tools, memory, and credentials rather than its prose.

Two are worth naming on their own. MCP tool poisoning hides instructions inside a tool description, so the agent reads the attack as part of its own configuration. And the broader set of MCP security risks shows how quickly a connected-tool ecosystem inherits every weakness of its least-vetted server. Multi-agent systems compound all of it, because one agent's output becomes another agent's trusted input.

Organizational issues

Shadow AI is the unsanctioned AI tool, agent, or connected server running in your environment without anyone's approval. It is not primarily a technology failure. It is what happens when the sanctioned path is slower than the unsanctioned one, and you fix it by making the governed path the easy path.

AI security standards and regulations

There is no single AI security standard, and anyone who tells you otherwise is selling one. What exists is a useful set of overlapping frameworks, each strong in one place and silent somewhere else. The silences are the part nobody maps, so here they are.

Framework	What it governs	Layers covered	Where it is silent
NIST AI Risk Management Framework (AI RMF 1.0)	Voluntary organizational practice, structured around four functions: Govern, Map, Measure, Manage	All three, at the level of process	Technical enforcement. It tells you to manage risk, not how to authorize a tool call
NIST Generative AI Profile (AI 600-1)	Generative-AI-specific risks layered onto the AI RMF	Model and data	Agent autonomy and tool use are thin
OWASP Top 10 for LLM Applications	The ten most critical application risks, with attack scenarios and mitigations	Model and data, plus excessive agency	Enterprise identity and delegated authority
OWASP Top 10 for MCP	Risks specific to Model Context Protocol deployments	Action	Model behavior and training-data concerns
MITRE ATLAS	Adversary tactics and techniques against AI systems, in the ATT&CK format	Model and data	Prescriptive controls. It is a threat catalog, not a control set
EU AI Act (Regulation 2024/1689)	Legal obligations by risk tier for providers and deployers	Governance and documentation across all layers	Implementation detail. Compliance is proven, not designed, by the text

The NIST AI Risk Management Framework organizes an entire program around four functions and is the most common backbone for one. Its generative-AI companion profile extends it to the risks that only show up once a model is generating rather than classifying, published as NIST AI 600-1, the Generative AI Profile. The OWASP catalog is the most operationally useful of the set because it maps its risks onto NIST, MITRE ATLAS, and CWE, so a finding in one vocabulary translates into the others, which the 2025 edition documents explicitly. And Regulation (EU) 2024/1689, the AI Act, turns a subset of all of this into law with dated obligations by risk tier.

Deeper treatments of each: the NIST AI Risk Management Framework broken down function by function, the OWASP Top 10 for LLM risk by risk, the OWASP Top 10 for MCP for connected-tool deployments, and EU AI Act compliance with its risk tiers and deadlines.

What an AI security program actually contains

Strip away the vendor categories and a working program has six moving parts. Most organizations have two of them.

Component	What it does	Typical owner
Inventory	Knows every model, agent, and connected tool in the environment, including the ones nobody registered	Security engineering
Posture management	Finds misconfiguration and drift across models, pipelines, and agents before an attacker does	Security engineering
Guardrails	Filters input and output at runtime against policy	Platform or AI engineering
Identity and authorization	Names every agent, binds it to the human or service it acts for, and authorizes each action	IAM
Detection and response	Watches agent behavior in production and contains what goes wrong	SOC
Governance and audit	Sets policy, evidences compliance, and answers who did what	GRC, with security

Each has a deeper guide. AI security posture management covers the inventory-and-drift half. AI risk management is the framework layer that decides which risks you accept. AI governance sets the policies those decisions enforce, and AI agent governance narrows it to autonomous systems specifically. AI observability and AI audit supply the evidence trail, and an AI compliance platform is where the evidence gets assembled for a regulator.

Securing the action layer: how an agent's tool call gets authorized

Ask the question that ends most AI security conversations: an agent just changed a record in production. Who is accountable?

If the answer is a service account name, the program is incomplete. The fix is structural. Every tool call passes through a point that knows three things before it lets the call through: which agent this is, which human or system it is acting for, and whether this specific action is permitted right now.

Four checks belong at that point, in order:

1. Authenticate the agent. It has its own identity, not a shared key. Non-human identity is the established discipline for this, and agents are the newest and least governed population inside it.
2. Attach the human. The request carries the person or service the agent is acting for, so delegated authority is explicit rather than implied. Frontegg's write-up on governing agents through identity guardrails is a good reference for how delegation, step-up, and session control work in practice.
3. Authorize the action. Not the session, the action. Scope, parameters, and limits are evaluated per call, and anything above a threshold escalates to a human.
4. Record it. One immutable line per action: which agent, on whose behalf, what it called, what came back.

Since most enterprise agents reach their tools through the Model Context Protocol, this is also where MCP security and MCP identity live. The Model Context Protocol specification defines how clients and servers exchange tools and context. It does not decide whether a given call should be allowed.

AI security best practices

Specific enough to act on this quarter.

1. Inventory before you police. You cannot scope an agent you do not know about. Discovery comes first, every time.
2. Give every agent its own name. No shared service accounts, no borrowed API keys. One identity per agent, issued and revocable like any other.
3. Keep the human attached. Delegated authority means the request knows who it is acting for. Anonymous autonomy is the thing to design out.
4. Authorize per action, not per session. A token that was valid at login says nothing about whether this specific call should run now.
5. Make tools narrow. Ten purpose-built tools with tight parameters beat one general tool with broad access, every time.
6. Treat all retrieved content as untrusted. Documents, web pages, tool descriptions, and other agents' output are inputs, not instructions.
7. Put a human in the loop at the threshold, not everywhere. Pick the actions that are irreversible or expensive and gate those. Gating everything trains people to click approve.
8. Red team before launch and on a schedule. These systems change behavior when the model version changes, so a one-time assessment expires.
9. Log for the incident review you will have. If the log cannot answer who was accountable, it is telemetry, not an audit trail.
10. Rehearse revocation. Time how long it takes to cut one agent off from one system. If the answer is measured in days, that is the real exposure window.

AI security vs traditional cybersecurity

Most of what you know still applies. Four things genuinely differ, and they are the four that break existing tooling.

Dimension	Traditional cybersecurity	AI security
The principal	Humans and applications with fixed permissions	Agents that improvise which permissions they use
Behavior	Deterministic. The same input produces the same path	Non-deterministic. The same prompt can produce a different tool call
The attack surface	Code, configuration, credentials, network	All of that, plus natural language reaching the model as instruction
Testing	A passing test proves the behavior	A passing run proves one sample. You need distributions
Blast radius	Bounded by what the account can reach	Bounded by what the agent's tools can reach, which is usually wider
Accountability	A user id in a log	Contested, unless agent identity was designed in

The practical consequence: your SIEM, your IAM, and your DLP are not obsolete. They are under-informed. They see a service account doing normal-looking API calls and have no way to know a model chose them.

Your first 90 days: an AI security checklist

Three phases. Do them in order, because each one makes the next cheaper.

Days 1 to 30: see it.

• Inventory every model, agent, assistant, and connected tool, sanctioned or not.
• For each agent, list the tools it can call and the systems those tools reach.
• Identify which credentials each agent uses and who else shares them.
• Score the inventory by blast radius. The agentic risk map gives you a scoring method rather than a gut feel.

Days 31 to 60: bound it.

• Replace the two or three widest-scoped agent credentials with dedicated identities.
• Narrow the highest-risk tool definitions: add parameter limits and remove unused capabilities.
• Add approval gates on irreversible and high-value actions.
• Turn on guardrails for the input and output paths that touch regulated data.

Days 61 to 90: prove it.

• Red team the two highest-risk agents, including indirect injection through documents and tools.
• Wire agent actions into detection so anomalous tool-call patterns raise an alert.
• Run a revocation drill and record the time to cut off a single agent.
• Map your controls to one framework and note the gaps honestly. The AI governance maturity model grades depth of attribution rather than depth of paperwork, which is the more useful scale.

Frequently asked questions

What is AI security in simple terms?

AI security is protecting AI systems from attack and misuse, and protecting your organization from what those systems can do. It covers three things: the model and its prompts, the data flowing in and out, and the actions the system is allowed to take in real systems. Think of it as access control for software that decides for itself.

What is the difference between AI security and AI safety?

AI security assumes an adversary is trying to manipulate the system. AI safety asks whether the system behaves acceptably even when nobody is attacking it, covering harmful, biased, or unintended output. They overlap in practice: an agent that misreads an instruction and deletes production data is a safety failure with security consequences. Mature programs fund both.

Is AI security the same as using AI for cybersecurity?

No, and conflating them leaves a gap. Using AI for cybersecurity means machine learning inside detection and response tools. AI security means securing the AI systems your company builds and runs. A security product with AI features does not secure your agents, and an agent security program does not make your SOC faster.

What are the biggest AI security issues?

Prompt injection leads every serious list, because a model cannot reliably separate instruction from content. Close behind: sensitive data disclosure, retrieval and tool poisoning, excessive agency, and credential inheritance, where agents share service accounts so no action traces back to a responsible party. Shadow AI makes all of them harder by hiding the systems involved.

What AI security standards should we follow?

Start with the NIST AI Risk Management Framework as your program backbone and the OWASP Top 10 for LLM Applications as your technical checklist. Add the OWASP Top 10 for MCP if agents call external tools, and MITRE ATLAS for threat modeling. If you operate in the EU, the AI Act sets legal obligations on top.

How do you secure an AI agent?

Give it its own identity rather than a shared key, keep the human it acts for attached to every request, scope its tools to the minimum the task needs, authorize each action rather than the session, gate irreversible actions behind human approval, log every call immutably, and rehearse revoking its access until that takes seconds.

Where to go deeper

This pillar is the map. The cluster below is the territory.

• Prompt injection - how direct and indirect attacks work, and the layered mitigations that actually reduce them.
• AI guardrails - the runtime filters on what an agent can take in, output, and do.
• AI red teaming - adversarial testing methods built for non-deterministic systems.
• AI threat detection - catching a rogue or compromised agent while it is still acting.
• AI security posture management - finding misconfiguration and drift across models, pipelines, and agents.
• AI risk management - the framework layer for deciding which risks you accept.
• MCP security - securing the protocol most enterprise agents use to reach their tools.
• Non-human identity - the identity discipline that agent accountability extends.

The layer that decides whether any of this holds is the one where an agent's action gets authorized. If you want to see what that looks like when every tool call passes a single accountable checkpoint, start with the MCP gateway.