Daviyon DanielsYour organization just completed its annual penetration test. Clean report, a few medium findings, nothing critical. Your vulnerability management program is current. You have checked the compliance boxes.
Now ask yourself: does any of that tell you whether your AI systems are secure?
It does not. Not because your security team is doing poor work, but because the methodologies underlying traditional assessments were designed before AI systems were part of the threat surface. They are evaluating the right things — for a pre-AI world.
The numbers bear this out. Cisco's 2025 Cybersecurity Readiness Index, surveying 8,000 business leaders across 30 markets, found that 86% of organizations experienced at least one AI-related security incident in the prior 12 months — yet only 4% achieved a "mature" cybersecurity readiness posture. Organizations are running AI. Their security programs have not caught up.
If your organization has deployed AI systems, you have security risks that your existing assessment program is not finding.
What Traditional Assessments Were Built to Find
Penetration testing, vulnerability scanning, and traditional security assessments are mature disciplines built around a well-understood threat model. They look for:
- Unpatched software vulnerabilities with known CVEs
- Misconfigurations in network infrastructure, cloud services, and applications
- Authentication weaknesses — weak credentials, missing MFA, session management flaws
- Injection vulnerabilities — SQL injection, command injection, XSS
- Access control failures — privilege escalation, insecure direct object references
- Exposed sensitive data in transit or at rest
These are real risks. They matter. Continuing to assess for them is correct.
But they share a fundamental assumption: the attack surface is deterministic infrastructure. Given the same input, the system produces the same output. You can scan it, map it, and test it exhaustively.
AI systems do not work that way. Their behavior is probabilistic. Their attack surface includes the model itself — its training data, its weights, its inference behavior — not just the infrastructure it runs on. And the attacks that target them exploit the model's reasoning, not its code.
Traditional assessment tools and methodologies have no visibility into that layer.
The Specific Gaps
Prompt injection is invisible to traditional scanners. Prompt injection is the AI-era equivalent of SQL injection. An attacker crafts malicious input designed to manipulate the AI system's behavior — overriding instructions, extracting sensitive information, or causing the system to take unintended actions.
This is not theoretical. In June 2025, researchers at Aim Security disclosed EchoLeak (CVE-2025-32711, CVSS 9.3) — a zero-click prompt injection exploit against Microsoft 365 Copilot. An attacker sends a crafted email with hidden instructions. Copilot ingests them automatically, no user interaction required. The attack chain bypassed Microsoft's prompt injection classifier, circumvented link redaction, and could exfiltrate chat logs, OneDrive files, SharePoint content, and Teams messages — all without the victim clicking anything.
A vulnerability scanner scanning your LLM-based application will find infrastructure vulnerabilities. It will not attempt thousands of prompt injection variations against your system prompt. It does not know what a system prompt is.
Assessing prompt injection resistance requires deliberate adversarial testing of the AI system's input handling — a discipline that requires understanding of how LLMs process instructions and what manipulation techniques are effective. This is not in scope for traditional penetration testing engagements unless specifically commissioned and the testers have AI-specific expertise. Prompt injection remains the #1 threat on the OWASP LLM Top 10, appearing in 73% of production deployments.
Model behavior is not evaluated. A penetration test validates that your authentication is working, your APIs are secured, and your application handles inputs correctly. It says nothing about what the model actually does.
Is your AI system producing outputs that violate your policies? Is it disclosing information from other users' sessions? Is it susceptible to jailbreaking in ways that could create legal or reputational liability? Is it producing biased outputs in high-stakes decisions?
None of those questions are answerable through traditional security assessment. They require behavioral evaluation of the model itself — test inputs designed to probe specific failure modes, output monitoring across a range of conditions, and analysis against documented acceptable use parameters.
Data pipeline risks are out of scope. AI systems are only as trustworthy as their training data and the pipelines that feed them. Data poisoning — injecting malicious or manipulated data into training pipelines to corrupt model behavior — is a real attack vector, particularly relevant for organizations that fine-tune models on their own data or train custom models.
Traditional assessments do not evaluate your ML pipelines. They do not assess whether your training data sources are trustworthy, whether your data labeling processes have been compromised, or whether your model update procedures prevent supply chain attacks.
Third-party AI supply chain risk is ignored. Most organizations using AI are using third-party models — foundation models via API, embedded AI features in SaaS products, open-source models from public repositories. Each represents a supply chain risk that traditional vendor risk management programs were not designed to evaluate.
The OpenClaw saga in early 2026 demonstrated how quickly AI supply chain risks materialize at scale. OpenClaw (originally Clawdbot, then Moltbot) — an open-source AI personal assistant connecting LLMs to messaging platforms, file systems, and shell commands — amassed over 250,000 GitHub stars within weeks of going viral. Internet scans found 42,000+ publicly accessible instances, with 93.4% exhibiting authentication bypass vulnerabilities. A critical 1-click RCE flaw (CVE-2026-25253, CVSS 8.8) allowed attackers to steal auth tokens and gain full administrative control. Meanwhile, threat actors uploaded 341 malicious skills to the project's ClawHub marketplace, bundling the AMOS infostealer targeting cryptocurrency wallets.
Separately, Moltbook — a social network built for OpenClaw agents — suffered a database misconfiguration that exposed 1.5 million API keys, 35,000 email addresses, and 4.75 million records total.
A standard vendor assessment will review your AI vendor's SOC 2 report and call it done. It will not assess whether the model's training data is trustworthy, whether the vendor's model update process could introduce backdoors, or whether the model produces outputs that would be unacceptable under your organization's policies. It will not catch malicious plugins in an AI marketplace or exposed agent infrastructure. Those are AI-specific risks that require AI-specific evaluation.
The same pattern is emerging across the Model Context Protocol (MCP) ecosystem. In September 2025, a counterfeit npm package called "postmark-mcp" silently BCC'd all outgoing emails to an attacker-controlled address — a near-exact replica of a legitimate library with a single added line of code, downloaded 1,643 times before removal. By early 2026, Trend Micro had identified 492 MCP servers running on the public internet with zero authentication and no traffic encryption, collectively providing access to over 1,400 tools — with over 90% offering direct read access to underlying data sources.
Model drift and degradation are not monitored. Traditional security monitoring looks for anomalous events: unexpected authentication attempts, unusual data access patterns, known malware signatures. It was not designed to detect the gradual degradation of AI model performance over time.
Model drift — where a model's real-world operating environment diverges from its training distribution, causing degraded performance — can create security and compliance risks without triggering any traditional security alert. Without AI-specific monitoring and evaluation, organizations often discover model drift through downstream consequences rather than proactive detection.
What AI-Specific Assessment Looks Like
An AI security assessment starts where traditional assessment ends. It assumes your infrastructure security is being handled by your existing program and focuses on the AI-specific layer.
The foundational component is an AI asset inventory. Before you can assess AI security risk, you need a complete picture of what AI systems your organization uses — built, bought, or embedded in other products. This frequently surfaces AI deployments that central security teams did not know about.
From there, AI risk assessment applies frameworks developed specifically for this domain. NIST's AI Risk Management Framework (AI RMF) provides a comprehensive structure for evaluating AI system risk across the AI lifecycle — from design through deployment through monitoring. OWASP's LLM Top 10 provides a specific enumeration of LLM application vulnerabilities that parallels the role the OWASP Top 10 plays for web applications.
CIS Controls v8.1 provides a complementary foundation for governance and operational controls. ISO 27001 provides the broader information security management context. A thorough AI assessment draws on all of these.
Adversarial testing of AI systems — the practical equivalent of penetration testing for AI — requires deliberately probing AI systems with inputs designed to elicit unintended behavior. This includes prompt injection testing, jailbreak attempts, data extraction probes, and evaluation of boundary conditions. The goal is to understand what the AI system actually does under adversarial conditions, not just what it is supposed to do.
Data governance assessment evaluates whether the data used to build and operate AI systems meets appropriate quality and security standards. This includes training data provenance, data labeling integrity, input validation at inference time, and output logging and monitoring.
Third-party model evaluation applies due diligence to the AI components your organization depends on but did not build. What are the model's documented limitations? What does the vendor's security program look like specifically for AI? What are the behavioral constraints of the model and how are they enforced?
The Framework Gap Is Real
Standard compliance frameworks are catching up to AI, but they are not there yet. If you are using CIS Controls v8.1, NIST 800-53, or ISO 27001 as your primary compliance frameworks, they provide important foundations for AI security — access control, risk management, vendor management, incident response — but they were not written with AI-specific attack vectors in mind.
NIST AI RMF (published 2023) and its companion profiles are specifically designed for AI governance and fill part of this gap. The EU AI Act is now actively enforceable — prohibited AI practices (manipulative AI, social scoring, unauthorized biometric identification) have carried penalties up to 35 million EUR or 7% of global annual turnover since February 2025, with full high-risk AI system requirements taking effect in August 2026. OWASP LLM Top 10 addresses LLM-specific application security. An adequate AI security program draws on all of these.
Organizations that assume their existing compliance posture addresses AI risk are making a category error. Compliance with traditional frameworks is necessary but not sufficient. The controls those frameworks require were designed for a different technical environment.
The Shadow AI Problem Compounds Everything
Even organizations with good security intentions are struggling with visibility. According to Cisco's research, 60% of IT teams cannot see specific prompts or requests made by employees using generative AI tools, and 60% of organizations lack confidence in detecting unsanctioned AI deployments. Only 51% require employees to use approved GenAI tools through a security service — 22% have completely unrestricted access.
Cisco's State of AI Security 2026 report, published in February 2026, adds another dimension: 83% of organizations plan to deploy agentic AI capabilities, but only 29% feel prepared for secure rollout. Agentic AI — systems that autonomously take actions, call tools, and interact with external services — amplifies every gap described in this article. An agent with access to your email, calendar, and file system is not just a chatbot. It is an autonomous actor operating within your security perimeter.
You cannot assess what you cannot see. Traditional security programs were not built to discover AI deployments, let alone evaluate the unique risks they introduce.
The Case for Acting Now
AI adoption is moving faster than security programs are adapting. That gap represents growing risk — not theoretical future risk, but active exposure in systems you are running today.
The organizations that will have meaningful AI security programs are the ones that recognize the gap exists and make deliberate decisions to address it, rather than waiting for an incident to force the issue.
At Ayliea, AI security assessment is our core practice. Our assessments evaluate AI systems specifically for the risks that traditional security assessments miss — using frameworks built for the AI era, not adapted from methodologies designed for a pre-AI world. If you are not sure what your AI security posture actually is, that is the question we help answer.