AI Security Platforms Explained: How Companies Are Defending Their Own AI Systems in 2026

Gartner now forecasts that spending on securing AI systems — not AI used for cybersecurity, but security built specifically to protect AI models, agents, and pipelines — will reach $2.8 billion in 2026 and jump 68.7% to $4.8 billion in 2027, according to an August 26, 2026 forecast from analyst Shailendra Upadhyay. That growth isn't speculative: it follows a string of 2025–2026 incidents, most notably a March 2026 breach in which an autonomous AI agent gained full read-write access to McKinsey's internal AI platform in under two hours, that turned "AI security" from a compliance talking point into a budget line with a specific, demonstrated failure mode behind it.

A Distinct Category, Not a Rebrand of "AI for Cybersecurity"

It's easy to conflate two things that sound similar but point in opposite directions. "AI for cybersecurity" means using AI models to detect intrusions, triage alerts, or automate incident response — AI as the defender. "AI security platforms" mean the reverse: tools built to protect the AI systems themselves, because a model, an agent, or the pipeline connecting them to enterprise data is now a thing that can be attacked, tricked, or hijacked in its own right. Gartner's forecast splits this market into four segments, each covering a different point in an AI system's lifecycle: AI application security ($508 million in 2026), AI usage control ($433 million), AI governance platforms ($275 million), and AI gateways ($251 million). AI usage control is the fastest-growing segment, projected to expand 73% into 2027, which Gartner attributes to enterprises needing to govern how employees and agents actually use AI tools day to day, not just whether the underlying model is technically sound.

The Breach That Made the Threat Model Concrete

For most of 2025, arguments for AI security platforms rested on hypotheticals: prompt injection could leak data, an over-permissioned agent could be manipulated into taking unauthorized actions. In March 2026, that changed. Security firm CodeWall disclosed that its autonomous offensive AI agent had been pointed at McKinsey's internal generative AI platform, called Lilli, with no insider access or credentials, and within two hours had gained full read-write access to its production database. The exposure was substantial: roughly 46.5 million internal chat messages, 728,000 confidential files, and 57,000 user accounts were reachable, and the agent could have rewritten the system prompts that shape how Lilli responds to the roughly 40,000 McKinsey staff who use it. McKinsey patched the exposed endpoints within 24 hours of responsible disclosure. What made the incident notable wasn't the sophistication of the exploit — CodeWall found a SQL injection flaw, one of the oldest bug classes in web security, sitting on 22 unauthenticated API endpoints. It was the speed: a task that would take a skilled human attacker days or weeks took an autonomous agent about two hours, because the agent doesn't get tired, doesn't second-guess itself, and can chain reconnaissance, exploitation, and privilege escalation without a break.

The Other Attack Surface: A Malicious Server Hiding in Plain Sight

If the McKinsey incident showed what happens when an attacker uses an AI agent, a separate 2025 incident showed what happens when the attack targets the infrastructure connecting agents to the rest of the internet. In September 2025, security researchers at Koi Security discovered what's widely described as the first documented malicious Model Context Protocol (MCP) server in the wild: a fake npm package called "postmark-mcp" that impersonated a legitimate tool for letting AI assistants send email through the Postmark service. The package behaved normally through 15 published versions, building trust and downloads, before a September 17, 2025 update quietly added a single line of code that blind-copied every outgoing email to an external address. The package had been downloaded roughly 1,500 times a week and was reportedly built into hundreds of developer workflows before it was pulled from npm eight days later. Postmark itself was uninvolved and issued a statement clarifying the fake package had no connection to its actual service. MCP servers typically run with broad, high-trust permissions inside an agent's toolchain, which is exactly why a single planted line of code could expose password resets, invoices, and internal correspondence at scale.

Why Established Cybersecurity Vendors Are Buying Rather Than Building

The clearest signal of how seriously incumbent cybersecurity vendors are treating this shift isn't product roadmaps — it's acquisitions. Within a roughly two-month window in mid-to-late 2025, three specialized AI-security startups were absorbed into three different established cybersecurity platforms. Palo Alto Networks completed its acquisition of Protect AI on July 22, 2025. SentinelOne agreed to acquire Prompt Security on August 5, 2025, for a deal reported at roughly $180 million in cash and stock, folding it into its Singularity platform. Check Point announced its acquisition of Lakera in mid-September 2025, expected to close in Q4 2025, to build what it called an end-to-end AI security stack. The pattern suggests incumbents concluded that AI security requires specialized capability they didn't already have — model-file scanning, prompt-injection detection, runtime behavioral monitoring — rather than something that could be bolted onto existing endpoint or network tools. Gartner's own market note observes that AI application security and AI usage control, the two segments seeing the most startup activity, are likely to see continued consolidation and acquisitions as larger cybersecurity companies buy the specialists to broaden their offerings. Not every player has been absorbed: HiddenLayer, focused specifically on ML model and runtime security, remains independent and backed by investors including Microsoft's M12 fund, continuing to compete alongside the now-larger, newly AI-capable incumbents.

What the Cost Data Shows

The financial case behind this spending isn't abstract either. IBM's 2026 Cost of a Data Breach Report, based on a Ponemon Institute study of 602 breached organizations across 17 industries surveyed between March 2025 and February 2026, found that AI-enabled malicious breaches cost an average of $6 million, roughly $1 million more than the $4.99 million global average, and that one in four malicious breaches were now AI-enabled, up 56% year over year. Separately, the report found that among organizations that experienced an AI-related breach, 92% lacked adequate AI access controls, and shadow AI incidents — unsanctioned AI tools deployed without security oversight — more than doubled from 20% to 43% of breached organizations in a single year. That combination is the real argument for AI security platforms: not a hypothetical future risk, but a documented pattern where the AI systems already in production are the specific point of failure, and access control (not exotic model manipulation) is the recurring root cause.

Where the Evidence Is Still Thin

It would be its own kind of hype to treat this as a fully mature, settled category. Gartner's own framing is forward-looking rather than descriptive of today: the firm predicts that over half of successful cyberattacks on AI agents will exploit access control weaknesses and prompt injection by 2029 — a projection, not a current measurement. And the two headline incidents behind this piece both point toward an uncomfortable nuance: the McKinsey breach exploited a SQL injection flaw and unauthenticated API endpoints, both long-standing, non-exotic application security failures that predate generative AI entirely, and the postmark-mcp incident was, in Koi Security's own characterization, not sophisticated — one developer, one line of code. Neither breach required a genuinely novel AI attack technique; both required an AI system that had been deployed without applying security fundamentals the industry has understood for years, just executed or exposed faster because an agent was involved. That's a real distinction for a buyer to sit with: a meaningful share of what "AI security platforms" sell is genuinely new (model-file scanning, prompt-injection detection, MCP-specific supply chain vetting), but a meaningful share of what's actually causing losses right now is old-fashioned access control and authentication discipline, applied to systems that were built and shipped faster than the standard review process could keep up with.

What This Means for a Team Right Now

Map what you actually have before buying anything. AI application security and usage control tools are only useful against systems you know exist — the McKinsey and postmark-mcp incidents both trace back to visibility gaps (unauthenticated endpoints, an unofficial MCP package) rather than a missing product category. Treat MCP connections as a software supply chain problem. Any MCP server your agents call runs with broad permissions by default; vet the source the same way you'd vet a dependency with write access to production data, not as a convenience integration. Don't assume the fix is a new AI-specific control. Gartner's own vendor data and both 2025–2026 incidents point toward access control and authentication gaps as the dominant failure mode, meaning existing security discipline, applied consistently to AI systems, closes a large share of the exposure before a specialized tool is even purchased. Expect vendor consolidation to continue. With three major acquisitions in a two-month span in 2025 and Gartner flagging more consolidation ahead, a startup's roadmap and independence are both live variables in any multi-year AI security purchase.

FAQ

Q: What's the difference between "AI security platforms" and "AI for cybersecurity"? A: AI for cybersecurity uses AI models as the tool — detecting anomalies, triaging alerts, automating response. AI security platforms protect AI systems themselves, treating the model, the agent, its tool connections, and its data access as things that need dedicated defense against attacks like prompt injection, model theft, or malicious MCP servers. Q: What is a Model Context Protocol (MCP) server, and why does it matter for security? A: MCP is a standard that lets AI assistants and agents connect to external tools and services, like sending email or querying a database, in a structured way. Because MCP servers typically run with broad, high-trust permissions inside an agent's workflow, a compromised or malicious one, like the postmark-mcp package discovered in September 2025, can expose whatever sensitive data flows through it. Q: Was the McKinsey Lilli breach a real attack or a test? A: It was a security research exercise conducted by CodeWall, a firm that uses autonomous AI agents to probe client infrastructure for weaknesses, not a malicious attack by a criminal actor. McKinsey worked with the researchers under responsible disclosure and patched the exposed endpoints within 24 hours. The significance is what it demonstrated about how fast an autonomous agent can exploit access control gaps, not that data was actually stolen for criminal use. Q: Do I need a dedicated AI security platform, or can existing security tools cover this? A: It depends on what you're protecting against. Standard application security and identity tools still catch fundamentals like unauthenticated endpoints and access control gaps, which are behind the highest-profile incidents so far. Dedicated AI security tools add capabilities standard tools generally don't have, such as model-file scanning for embedded threats, prompt-injection detection, and MCP-specific vetting, which matter more as agentic AI deployment expands. Q: Why are established cybersecurity companies acquiring AI security startups instead of building the capability internally? A: Based on the pace of 2025 acquisitions, incumbents appear to have concluded that AI-specific security techniques, like adversarial model testing and runtime prompt-injection detection, require specialized expertise built from the ground up rather than incremental additions to existing endpoint or network security products. Gartner's market analysis also points to continued consolidation as larger vendors look to broaden their AI security offerings by acquiring the startups building these capabilities.