Five Critical Mistakes That Cause AI Agents to Derail Automated Penetration Tests

An AI agent can independently study an application, select appropriate techniques, launch tools, and collect vulnerability evidence. However, the gap between what it can do and what it reliably accomplishes remains substantial.

Practical insights into these challenges and free access to master classes on AI-powered pentesting are available through Red September, which currently offers open materials on offensive security, including sessions on applying AI agents and LLMs in attacker workflows.

The article is based on a hands-on experiment conducted against a training web application. The AI agent was given the goal of autonomously exploring the system, identifying an exploitation path, escaping the sandbox, escalating privileges, and performing a designated control action. It succeeded overall but encountered multiple dead ends along the way: it spent hours on inapplicable attack vectors, declared the working environment unavailable, and attempted to solve technically correct yet strategically meaningless problems.

The problem is not always that the model lacks intelligence. More frequently, it lacks research discipline: understanding the system, formulating a hypothesis, testing it through observable means, and only then moving forward. The following five errors turn AI-driven pentesting from an accelerator into a generator of dead ends.

Agent Begins Fuzzing Before Understanding the Target

The most tempting approach is to provide the agent with a target address and instruct it to find all vulnerabilities. It launches directory brute-forcing, parameter scanning, and payload delivery, receives large volumes of responses, and starts building hypotheses. Activity appears high, tokens are consumed, and the terminal fills with commands, yet useful results may be absent.

On the training environment, the winning strategy proved to be the opposite: first reading available source code to understand business logic. This quickly revealed which database queries were parameterized, where an assumed attack vector made no sense, and through which points the application actually interacted with the external environment.

The agent tends to choose actions that can be executed immediately with available tools. Fuzzing produces an instant stream of observations, while reading code requires maintaining context and connecting details. Without explicit instructions, the model often prefers activity over understanding.

Agent Selects Exploits Before Precisely Identifying Version

Another trap occurs when the agent correctly identifies the technology class but fails to determine the exact version, build, or environmental specifics. It locates several public proofs of concept, downloads them, and attempts to adapt each to the target in sequence. Sometimes one variant works; other times a lengthy cycle of modifying foreign code begins, code that was never suited to the system under test.

A short additional step proved far more effective: examining available system files for component information, clarifying the precise version, and only then selecting an appropriate PoC. A few minutes spent accurately identifying the technology and version can save hours of trial and error.

Agent Fails to Establish a Feedback Channel

Many attack scenarios produce results that cannot be observed in the same session where the action occurs. These include Blind SSRF, asynchronous processing, code execution inside sandboxes, and deferred tasks. If the agent can only send a request and read the immediate response, lack of data is interpreted as absence of vulnerability.

A practical solution involved establishing a simple bidirectional observation channel: the agent started a listener server to capture reverse connections and thereby track external interactions. Previously invisible actions became verifiable events. The same principle applies more broadly: a control endpoint, event log, correlation identifier, or repeated state check is required to determine what occurred outside the current session.

Agent Prematurely Declares an Attempt Failed

Suppose the agent sends a payload and waits thirty seconds. No response arrives. The model concludes the technique does not work, the service is unavailable, or the environment has crashed. It then changes its hypothesis even though the result might appear later.

In practice, delays are often inherent to system behavior. Code may execute asynchronously, tasks may wait in queues, or separate processes may start with latency. A rigid short timeout converts a potentially working technique into a false negative.

Agent Mistakes Its Own Actions for Infrastructure Failure

Even more dangerous is the situation in which the verification itself temporarily blocks the component under test. A long-running call may occupy an execution thread, causing the interface to stop responding, leading the agent to conclude the target has broken. It then begins altering network settings, restarting tools, or shifting to another investigation branch.

In reality the environment may still be operational, and the delay is a direct consequence of the agent’s previous action. Before declaring infrastructure unavailable, the agent must perform independent diagnostics: verify basic network connectivity, confirm the listener uses the current address after any VPN change, test an unaffected endpoint, repeat the request after expected operation completion, and compare symptoms with the expected outcome of the last command.

Agent Pursues Technically Possible but Strategically Meaningless Goals

The final pitfall arises from insufficient understanding of business logic. The agent may spend considerable time searching for a familiar vulnerability class even though source code shows the required scenario does not exist. After privilege escalation it may directly modify a control file, an action that is technically feasible yet fails to demonstrate the intended business-logic event.

Before committing to a long investigation branch, three questions should be answered: does the hypothesis advance toward the target event rather than merely an interesting technical result; does it align with the application’s architecture; and can an observation be named in advance that would confirm its value.

All five errors share a common root: the agent acts faster than it understands consequences. Correction does not require an ideal prompt or the most expensive model. What is needed is a process in which every action leaves a verifiable result. The working cycle consists of context, hypothesis, safe action, observation, independent verification, and plan update. The human role remains essential for defining strategy, validating real impact, controlling research boundaries, and deciding when to intensify actions.

Materials describing this approach and practical exercises are available in the free Red September collection until 30 September, covering AI agents, web pentesting, Active Directory, OSINT, Bug Bounty, and attacks on CI/CD pipelines.