AI Crawlers Devour Web Traffic as Scraping Ratios Hit 38,000 Pages per Human Visitor
AI-powered crawlers have transformed web traffic patterns, with robots now accounting for the vast majority of requests on many sites. One developer tracking charitable donors on PatronView measured 1.28 million page deliveries where fewer than 6,000 came from actual humans, producing a ratio of roughly 214 automated loads per human view.
Four main categories of bots are active. Honest SEO crawlers such as SemrushBot and AhrefsBot identify themselves and index content normally. Training bots like GPTBot and ClaudeBot perform bulk downloads to feed large language models. Search-oriented AI crawlers (Claude-SearchBot, Amzn-SearchBot) and agentic bots executing live user prompts generate the heaviest load.
In April, PatronView recorded 3.6 million requests in a single day from 361,844 unique IPs concentrated in China. Cloudflare blocked 1.18 million attempts over ten hours, yet a troubling share still bypassed CAPTCHA challenges. The site owner ultimately blocked entire countries at the edge, later adding Vietnam and Singapore, because 95.9 percent of its real audience is in the United States.
Traffic exchange ratios have collapsed. Cloudflare previously reported Anthropic crawlers at 3,000 pages per visitor; measurements on PatronView reached 35,000 to one. Claude-SearchBot fetched 420,680 pages in a week while Claude-User delivered only 12 humans. Amazon’s Rufus-related crawler requested 117,000 pages daily and returned zero visitors.
Founder Bruce Nash of The Numbers observed even steeper imbalances: Google at roughly five pages per visitor, OpenAI above 1,000, and Anthropic exceeding 38,000. The 1997-era film database suffered a week-long outage in March 2026 after scraping campaigns escalated into targeted reconnaissance for Polymarket prediction markets. The team ultimately rebuilt a reduced version on new infrastructure rather than risk repeated compromise of legacy code.
Related articles
Docker Rebrands Around AI Agents with 'Run on Trust' and New Sandbox Tools
Docker has updated its branding to emphasize security for AI agents, introducing the slogan 'Trust Docker for the agents you don’t' and the tagline 'build, ship, and run on trust'. The company highlighted products such as Sandboxes, AI Governance, MCP Enterprise Gateway, and Hardened Images, although most of these features were released or discussed earlier in 2025. Sandboxes run agents inside microVMs with separate kernels using KVM or Windows Hypervisor Platform, allowing isolated execution of tools like Claude Code. AI Governance provides centralized policy management and audit logging for enterprise teams, while MCP Enterprise Gateway handles SSO and credential injection for MCP servers. Hardened Images remain available under Apache 2.0 with optional paid FIPS and support tiers. The shift follows the appointment of Don Johnson, formerly of Oracle, as CEO and reflects Docker’s attempt to address new security concerns around autonomous AI agents rather than traditional container workloads.
AI Agents Exploit PaperCut Flaw, Compromising 395 Organizations Across 48 Countries
PaperCut released a security bulletin on 27 August 2026. GreyNoise observed the first real-world code execution just four days later, with the operator moving from initial access to execution in under four hours. After the mass campaign launched, 11 organizations were compromised in only 26 seconds. In total, at least 440 PaperCut NG/MF instances belonging to 395 organizations were breached. The incident is detailed in the November report by Google Threat Intelligence Group and Mandiant, which examines the role of AI agents in the attacks. GTIG also discovered an automated exploit_agent system capable of handling exploitation without constant human input. Additional cases involved LLMjacking and unauthorized configuration changes in Claude Code repositories at cryptocurrency firms.
Google Threat Intelligence Group Warns of Fully Autonomous AI Cyberattacks Within Six Months
Google's Threat Intelligence Group (GTIG) has observed the first real-world instances of attackers using multi-agent AI frameworks to autonomously discover vulnerabilities, generate exploits, and compromise targets without human intervention. The discovery came from an exposed server running a framework called exploit_agent that analyzed software update logs from popular web applications to identify and exploit weaknesses. Four previously defensive tools—ARTEX, CAIRN, CyberStrikeAI, and HexStrike AI—have been repurposed for offensive operations, with documented cases targeting French media, U.S. aerospace, and cryptocurrency firms. GTIG projects that low-complexity automated attacks will become common within three months, while sophisticated fully autonomous operations are expected within six months. The report highlights how these systems compress patch windows, overwhelm security teams with AI-generated proof-of-concepts, and turn detailed vendor advisories into attack playbooks. Defenders are urged to minimize public exposure, accelerate patching cycles to weekly, and tightly control API keys to prevent LLMjacking.
OpenAI Rolls Out KYC, Hardware Keys and Tiered Access for Cybersecurity Tasks in GPT-6.2
OpenAI has introduced stricter access controls for GPT-6.2 Astra and Sol models when users attempt cybersecurity-related tasks. Certain legitimate security research and testing activities now trigger automatic blocks unless the user obtains Daybreak trusted access. The program requires paid subscriptions, FIDO2 hardware keys, Advanced Account Security, and identity verification through Persona. Developers report that even after meeting preliminary requirements, access to reduced refusal rates is not guaranteed and may require separate organizational approval. The changes stem from the models reaching Critical capability levels under OpenAI’s Preparedness Framework for dual-use cybersecurity skills. Similar verification processes previously led to account blocks for users of competing services such as Anthropic’s Claude.