Five Critical Mistakes That Cause AI Agents to Derail Automated Penetration Tests
An AI agent can independently study an application, select appropriate techniques, launch tools, and collect vulnerability evidence. However, the gap between what it can do and what it reliably accomplishes remains substantial.
Practical insights into these challenges and free access to master classes on AI-powered pentesting are available through Red September, which currently offers open materials on offensive security, including sessions on applying AI agents and LLMs in attacker workflows.
The article is based on a hands-on experiment conducted against a training web application. The AI agent was given the goal of autonomously exploring the system, identifying an exploitation path, escaping the sandbox, escalating privileges, and performing a designated control action. It succeeded overall but encountered multiple dead ends along the way: it spent hours on inapplicable attack vectors, declared the working environment unavailable, and attempted to solve technically correct yet strategically meaningless problems.
The problem is not always that the model lacks intelligence. More frequently, it lacks research discipline: understanding the system, formulating a hypothesis, testing it through observable means, and only then moving forward. The following five errors turn AI-driven pentesting from an accelerator into a generator of dead ends.
Agent Begins Fuzzing Before Understanding the Target
The most tempting approach is to provide the agent with a target address and instruct it to find all vulnerabilities. It launches directory brute-forcing, parameter scanning, and payload delivery, receives large volumes of responses, and starts building hypotheses. Activity appears high, tokens are consumed, and the terminal fills with commands, yet useful results may be absent.
On the training environment, the winning strategy proved to be the opposite: first reading available source code to understand business logic. This quickly revealed which database queries were parameterized, where an assumed attack vector made no sense, and through which points the application actually interacted with the external environment.
The agent tends to choose actions that can be executed immediately with available tools. Fuzzing produces an instant stream of observations, while reading code requires maintaining context and connecting details. Without explicit instructions, the model often prefers activity over understanding.
Agent Selects Exploits Before Precisely Identifying Version
Another trap occurs when the agent correctly identifies the technology class but fails to determine the exact version, build, or environmental specifics. It locates several public proofs of concept, downloads them, and attempts to adapt each to the target in sequence. Sometimes one variant works; other times a lengthy cycle of modifying foreign code begins, code that was never suited to the system under test.
A short additional step proved far more effective: examining available system files for component information, clarifying the precise version, and only then selecting an appropriate PoC. A few minutes spent accurately identifying the technology and version can save hours of trial and error.
Agent Fails to Establish a Feedback Channel
Many attack scenarios produce results that cannot be observed in the same session where the action occurs. These include Blind SSRF, asynchronous processing, code execution inside sandboxes, and deferred tasks. If the agent can only send a request and read the immediate response, lack of data is interpreted as absence of vulnerability.
A practical solution involved establishing a simple bidirectional observation channel: the agent started a listener server to capture reverse connections and thereby track external interactions. Previously invisible actions became verifiable events. The same principle applies more broadly: a control endpoint, event log, correlation identifier, or repeated state check is required to determine what occurred outside the current session.
Agent Prematurely Declares an Attempt Failed
Suppose the agent sends a payload and waits thirty seconds. No response arrives. The model concludes the technique does not work, the service is unavailable, or the environment has crashed. It then changes its hypothesis even though the result might appear later.
In practice, delays are often inherent to system behavior. Code may execute asynchronously, tasks may wait in queues, or separate processes may start with latency. A rigid short timeout converts a potentially working technique into a false negative.
Agent Mistakes Its Own Actions for Infrastructure Failure
Even more dangerous is the situation in which the verification itself temporarily blocks the component under test. A long-running call may occupy an execution thread, causing the interface to stop responding, leading the agent to conclude the target has broken. It then begins altering network settings, restarting tools, or shifting to another investigation branch.
In reality the environment may still be operational, and the delay is a direct consequence of the agent’s previous action. Before declaring infrastructure unavailable, the agent must perform independent diagnostics: verify basic network connectivity, confirm the listener uses the current address after any VPN change, test an unaffected endpoint, repeat the request after expected operation completion, and compare symptoms with the expected outcome of the last command.
Agent Pursues Technically Possible but Strategically Meaningless Goals
The final pitfall arises from insufficient understanding of business logic. The agent may spend considerable time searching for a familiar vulnerability class even though source code shows the required scenario does not exist. After privilege escalation it may directly modify a control file, an action that is technically feasible yet fails to demonstrate the intended business-logic event.
Before committing to a long investigation branch, three questions should be answered: does the hypothesis advance toward the target event rather than merely an interesting technical result; does it align with the application’s architecture; and can an observation be named in advance that would confirm its value.
All five errors share a common root: the agent acts faster than it understands consequences. Correction does not require an ideal prompt or the most expensive model. What is needed is a process in which every action leaves a verifiable result. The working cycle consists of context, hypothesis, safe action, observation, independent verification, and plan update. The human role remains essential for defining strategy, validating real impact, controlling research boundaries, and deciding when to intensify actions.
Materials describing this approach and practical exercises are available in the free Red September collection until 30 September, covering AI agents, web pentesting, Active Directory, OSINT, Bug Bounty, and attacks on CI/CD pipelines.
Related articles
Docker Rebrands Around AI Agents with 'Run on Trust' and New Sandbox Tools
Docker has updated its branding to emphasize security for AI agents, introducing the slogan 'Trust Docker for the agents you don’t' and the tagline 'build, ship, and run on trust'. The company highlighted products such as Sandboxes, AI Governance, MCP Enterprise Gateway, and Hardened Images, although most of these features were released or discussed earlier in 2025. Sandboxes run agents inside microVMs with separate kernels using KVM or Windows Hypervisor Platform, allowing isolated execution of tools like Claude Code. AI Governance provides centralized policy management and audit logging for enterprise teams, while MCP Enterprise Gateway handles SSO and credential injection for MCP servers. Hardened Images remain available under Apache 2.0 with optional paid FIPS and support tiers. The shift follows the appointment of Don Johnson, formerly of Oracle, as CEO and reflects Docker’s attempt to address new security concerns around autonomous AI agents rather than traditional container workloads.
AI Agents Exploit PaperCut Flaw, Compromising 395 Organizations Across 48 Countries
PaperCut released a security bulletin on 27 August 2026. GreyNoise observed the first real-world code execution just four days later, with the operator moving from initial access to execution in under four hours. After the mass campaign launched, 11 organizations were compromised in only 26 seconds. In total, at least 440 PaperCut NG/MF instances belonging to 395 organizations were breached. The incident is detailed in the November report by Google Threat Intelligence Group and Mandiant, which examines the role of AI agents in the attacks. GTIG also discovered an automated exploit_agent system capable of handling exploitation without constant human input. Additional cases involved LLMjacking and unauthorized configuration changes in Claude Code repositories at cryptocurrency firms.
Google Threat Intelligence Group Warns of Fully Autonomous AI Cyberattacks Within Six Months
Google's Threat Intelligence Group (GTIG) has observed the first real-world instances of attackers using multi-agent AI frameworks to autonomously discover vulnerabilities, generate exploits, and compromise targets without human intervention. The discovery came from an exposed server running a framework called exploit_agent that analyzed software update logs from popular web applications to identify and exploit weaknesses. Four previously defensive tools—ARTEX, CAIRN, CyberStrikeAI, and HexStrike AI—have been repurposed for offensive operations, with documented cases targeting French media, U.S. aerospace, and cryptocurrency firms. GTIG projects that low-complexity automated attacks will become common within three months, while sophisticated fully autonomous operations are expected within six months. The report highlights how these systems compress patch windows, overwhelm security teams with AI-generated proof-of-concepts, and turn detailed vendor advisories into attack playbooks. Defenders are urged to minimize public exposure, accelerate patching cycles to weekly, and tightly control API keys to prevent LLMjacking.
OpenAI Rolls Out KYC, Hardware Keys and Tiered Access for Cybersecurity Tasks in GPT-6.2
OpenAI has introduced stricter access controls for GPT-6.2 Astra and Sol models when users attempt cybersecurity-related tasks. Certain legitimate security research and testing activities now trigger automatic blocks unless the user obtains Daybreak trusted access. The program requires paid subscriptions, FIDO2 hardware keys, Advanced Account Security, and identity verification through Persona. Developers report that even after meeting preliminary requirements, access to reduced refusal rates is not guaranteed and may require separate organizational approval. The changes stem from the models reaching Critical capability levels under OpenAI’s Preparedness Framework for dual-use cybersecurity skills. Similar verification processes previously led to account blocks for users of competing services such as Anthropic’s Claude.