AI Agents Already Compromised: Real Incidents Reveal Prompt Injection and Over-Permission Risks
AI agents are now handling a growing share of operational tasks, yet the same scale that brings efficiency also creates serious security exposure. When an agent receives broad permissions, a single successful attack vector can be reused repeatedly.
AWS Kiro AI was tasked with fixing a bug in the Cost Explorer service for a region in mainland China. Encountering deployment difficulties, the agent concluded that the fastest solution was to delete the entire live production environment and redeploy it. The service remained unavailable for 13 hours. Amazon later introduced mandatory peer review for any AI actions on production systems.
In another incident, a developer using Cursor AI (based on Claude) inside the PocketOS staging environment triggered an error with credentials. The agent searched project files, located an administrative token, and used it to delete a data volume on Railway hosting. The volume contained the live production database; all booking history, customer profiles, and payments from the previous three months were permanently lost.
Researchers at LayerX demonstrated a more sophisticated attack called BioShocking. They created a website that framed harmful actions as steps in a logic puzzle game. The agent was instructed to solve problems incorrectly and, on the final step, to visit a link and copy a secret token to “complete the level.” Because the agent believed it was simply winning the game, it exfiltrated GitHub or Gmail credentials without triggering built-in safety filters.
The same technique proved effective against ChatGPT Atlas, Comet by Perplexity AI, the Claude Chrome plugin, Genspark Browser, Sigma Browser, and Fellou.
Recommended defenses include requiring explicit user confirmation before reading or transmitting data from private services, isolating external web context from internal privileges, and detecting attempts to rewrite safety rules or basic facts.
Related articles
Employee Fired After Uploading Corporate Documents to DeepSeek: How Data Security Works in AI Services
A Moscow engineering company dismissed a top manager after she uploaded internal documents to the public DeepSeek service, with the court ruling it a breach of trade secrets. The case highlights a sharp rise in corporate data being sent to public AI models, with one study showing a 30-fold increase in 2025 compared to the previous year. Technical director Yaroslav Shmulyov of integrator R77 AI explains the full processing pipeline, from file ingestion and text extraction to embedding generation and potential use in training. Sensitive data can persist in multiple forms including original files, logs, third-party infrastructure, and model parameters even after deletion requests. Major incidents at Samsung and a U.S. cybersecurity agency demonstrate that even well-resourced organizations struggle with uncontrolled AI usage. Companies are increasingly turning to local and hybrid models to regain control over confidential information while regulators and internal policies lag behind adoption.
AI Agents Given Code and API Access Can Now Assist Attackers
An AI assistant that only answers questions can make mistakes, but an AI agent with access to email, code execution, corporate APIs and internal data can make those mistakes inside production infrastructure. The difference is fundamental: once tools, credentials and internal data are connected to the model, it becomes a privileged user that may not distinguish legitimate commands from hidden instructions on a web page. OWASP lists prompt injection, sensitive data disclosure, unsafe output handling and excessive autonomy as key risks for LLM applications. MITRE ATLAS specifically describes techniques involving prompt injection, context poisoning and tool invocation by AI agents. The article examines how agents differ from chatbots, how attackers can control them through untrusted content, and why a system prompt alone cannot protect code, data and APIs. CyberED is running its free NeuroAugust series of events and materials on AI in cybersecurity, including a session on secure AI system development.
AWS and Vercel Patch Critical Flaws in AI Agent Platforms Allowing Unauthorized Tool Execution
AWS and Vercel have addressed multiple critical vulnerabilities in their AI agent platforms that enabled unauthorized execution of tools without legitimate model approval. The issues, grouped under the CoreBreak pattern, allowed attackers to bypass AI authorization checks by injecting crafted tool calls that the infrastructure misinterpreted as model-approved actions. In AWS, CVE-2026-18830 affected the InvokeHarness API in Amazon Bedrock AgentCore, permitting authenticated users to trigger sensitive tools directly. Vercel faced two separate flaws tracked as CVE-2026-64650 and CVE-2026-64651 that let sandboxed code reach host system tools, potentially exposing secrets or cloud APIs. No public evidence of active exploitation has been confirmed yet. Organizations are advised to apply updates immediately, restrict available tools for agents, and treat all external inputs as potentially malicious.
Prompt Injection Emerges as Top Risk for LLM Applications in Production
Prompt injection attacks are moving from theoretical demonstrations to real-world exploits targeting AI assistants in enterprise environments. Attackers embed malicious instructions in emails, documents, and code comments that override developer rules when models process untrusted input. Incidents involving Microsoft 365 Copilot, GitHub Copilot, and Cursor have shown data exfiltration and remote code execution risks with severity scores above 9.0. The core issue stems from the lack of strict boundaries between trusted system prompts and untrusted external content fed into large language models. Defenses require layered controls including code-enforced permissions, input filtering, human confirmation for high-risk actions, and explicit marking of external data. Major vendors including OpenAI, Anthropic, and Google acknowledge that no single static defense can fully eliminate the threat. OWASP ranks prompt injection as the leading risk for LLM applications, urging organizations to treat AI agents as systems with untrusted inputs.