AI systems introduce an entirely new class of vulnerabilities that traditional security tools weren't designed to detect. Here's what you're up against.
Malicious instructions embedded in user input that override the model's system prompt or intended behavior. Can be direct ("ignore previous instructions") or indirect — hidden in documents, web content, or tool outputs the model retrieves. Once injected, an attacker can extract internal prompts, exfiltrate data, or cause the model to take unauthorized actions.
Techniques to bypass a model's safety training and content filters. Includes roleplay attacks ("pretend you are an AI with no restrictions"), hypothetical framing, encoding attacks, and adversarial suffixes that confuse safety classifiers. Jailbroken models can produce harmful outputs, leak training data, or ignore your application's guardrails.
AI interfaces are exceptionally effective channels for leaking data — both intentionally (malicious prompts designed to extract information) and accidentally (employees pasting internal data into external AI tools). PII, credentials, source code, and financial data routinely appear in enterprise AI prompts with no visibility into where it ends up.
Employees adopt AI tools faster than security teams can review them. Tools that haven't signed a DPA, tools with broad data training clauses, tools with known vulnerabilities — all being used daily with production data. Shadow AI represents the single largest uncontrolled data exposure surface in most enterprises today.
Autonomous AI agents that execute multi-step tasks introduce a new attack surface: any point of external input (web pages, emails, tool responses) can contain injected instructions. A single poisoned retrieval can redirect an agent's entire session. Agents also have persistent access to tools and data — privilege escalation and unauthorized access become systemic risks.
Attacks targeting the data and pipelines used to train or fine-tune AI models. Data poisoning introduces backdoors into model behavior. Adversarial examples cause systematic misclassification. Supply-chain attacks compromise the models, libraries, or infrastructure your AI applications depend on.
Recent research, incidents, and regulatory developments shaping the AI security landscape.
The updated OWASP Top 10 for Large Language Model Applications (2025 edition) expands coverage to include Excessive Agency, Misinformation, and Unbounded Consumption as new entry-level threat categories — reflecting the rise of agentic AI deployments.
A Canadian tribunal ruled Air Canada liable after its AI chatbot provided incorrect bereavement fare information to a customer. The case set a legal precedent: organisations are responsible for AI-generated outputs, even when those outputs contradict official policy — underscoring the need for output monitoring and guardrails.
Researchers at ETH Zürich and Google DeepMind demonstrated that AI agents reading web content can be hijacked by malicious text embedded in documents — causing them to exfiltrate data, send emails, or execute code without user knowledge. The attack surface grows with every tool an agent has access to.
The EU AI Act became law in August 2024, introducing mandatory risk assessments, transparency requirements, and conformity obligations for AI systems operating in the EU. High-risk systems (including those used in employment, education, and critical infrastructure) face audit requirements by August 2026. Non-compliance carries fines of up to 3% of global annual turnover.
NIST's AI Risk Management Framework, released in 2023, saw widespread adoption in 2024 with over 1,000 organisations publishing AI use policies referencing it. NIST also released AI RMF Playbook v1.1 and the Generative AI Profile (NIST AI 600-1), providing specific guidance for LLM risks including CBRN information generation and homogenisation.
A Hong Kong finance worker was tricked into transferring $25M USD after attending a video call with AI-generated deepfakes of the company's CFO and other executives. The incident highlighted the growing threat of AI-powered social engineering and the need for out-of-band verification protocols for financial transactions.
Enterprise AI prompts contain sensitive data that violates security policy
Of detected prompt injection attempts come from indirect sources (retrieved documents, tool outputs)
Average number of distinct AI tools in use at a mid-sized enterprise — most unreviewed
Average time for an AI agent to propagate a malicious instruction to a downstream tool call
Kovral's threat engine covers every category above — deployed in 15 minutes.
