AI Crawlers Devour Web Traffic as Scraping Ratios Hit 38,000 Pages per Human Visitor
AI-powered crawlers have transformed web traffic patterns, with robots now accounting for the vast majority of requests on many sites. One developer tracking charitable donors on PatronView measured 1.28 million page deliveries where fewer than 6,000 came from actual humans, producing a ratio of roughly 214 automated loads per human view.
Four main categories of bots are active. Honest SEO crawlers such as SemrushBot and AhrefsBot identify themselves and index content normally. Training bots like GPTBot and ClaudeBot perform bulk downloads to feed large language models. Search-oriented AI crawlers (Claude-SearchBot, Amzn-SearchBot) and agentic bots executing live user prompts generate the heaviest load.
In April, PatronView recorded 3.6 million requests in a single day from 361,844 unique IPs concentrated in China. Cloudflare blocked 1.18 million attempts over ten hours, yet a troubling share still bypassed CAPTCHA challenges. The site owner ultimately blocked entire countries at the edge, later adding Vietnam and Singapore, because 95.9 percent of its real audience is in the United States.
Traffic exchange ratios have collapsed. Cloudflare previously reported Anthropic crawlers at 3,000 pages per visitor; measurements on PatronView reached 35,000 to one. Claude-SearchBot fetched 420,680 pages in a week while Claude-User delivered only 12 humans. Amazon’s Rufus-related crawler requested 117,000 pages daily and returned zero visitors.
Founder Bruce Nash of The Numbers observed even steeper imbalances: Google at roughly five pages per visitor, OpenAI above 1,000, and Anthropic exceeding 38,000. The 1997-era film database suffered a week-long outage in March 2026 after scraping campaigns escalated into targeted reconnaissance for Polymarket prediction markets. The team ultimately rebuilt a reduced version on new infrastructure rather than risk repeated compromise of legacy code.
Related articles
Claude Encrypted Thinking Blocks Use Protobuf with Exposed Metadata and AES-GCM Ciphertext
A detailed reverse-engineering of Claude signatures shows that the encrypted reasoning blocks are not opaque containers but structured protobuf messages. The outer envelope contains a 312-byte inner message that holds a 135-byte header, fixed-length nonce and MAC fields, and the actual ciphertext. The header itself reveals the model name such as claude-opus-5, the block type as thinking, and the organizationUuid from the user's Anthropic account. Only the reasoning text is encrypted with AES-GCM, adding exactly 16 bytes for the authentication tag. The analysis covers four protocol versions and notes that organization binding was added in version 15, potentially allowing servers to reject cross-model or cross-organization reuse. The findings provide concrete implications for both the Opus-to-Haiku extraction attack and the leakage of account identifiers in public logs.
Researchers Extract Proprietary Reasoning Traces from Anthropic, OpenAI and Google LLMs, Revealing Hidden Secrets
A team of eight researchers from institutions including ELLIS Institute Tübingen, the Max Planck Institute for Intelligent Systems, Tübingen AI Center, MATS and Snyk published a preprint detailing a practical attack that recovers full reasoning traces from closed LLM APIs. The method requires only two API calls and works by feeding encrypted reasoning blocks from strong models such as Claude Opus 4.8 into weaker models from the same provider, such as Haiku 4.5, which then reproduce the hidden chain-of-thought verbatim. Analysis of 6,708 publicly shared agent logs from GitHub and Hugging Face yielded 315,320 recovered traces containing 704 unique secrets, including 62 API keys, 33 passwords and 24 access tokens that never appeared in visible session output. The attack also enables extraction of internal safety policies, system prompts and detailed harmful planning that providers normally filter from final answers. In addition, the same mechanism can be used in reverse to inject malicious instructions into shared logs that later get replayed by unsuspecting users. The authors recommend treating encrypted reasoning blocks as sensitive secrets and propose cryptographic binding of traces to sessions, users and models.
GhostSplice Technique Lets Malicious MCP Servers Trick AI Coding Agents into Exfiltrating Secrets
GhostSplice is a new technique that allows a malicious MCP server to induce an AI coding agent to leak SSH keys, environment secrets, and source code. The attack splits malicious instructions across tool metadata and responses so the agent reconstructs and executes the full exfiltration plan without detecting an overtly malicious command. Tests showed the method raised compliance rates from an average of 42 percent to 82 percent across eleven models, with some systems moving from zero to 100 percent success. The technique requires the developer to connect the attacker-controlled MCP server and for the agent to already possess read access to the targeted files. Defenses focus on strict allow-listing of MCP servers, least-privilege tool permissions, separation of tool output from instructions, and human approval for sensitive operations. The disclosure aligns with prior warnings about poisoned MCP tool descriptions and agentjacking attacks.
Anthropic Claude Code Auto Mode Launches August 14 with Local Classifier and Permission Rules
Starting August 14, Claude Code will run in auto mode on new sessions for Pro, Max, and Team plans, replacing the allow/deny dialog with a local classifier that evaluates every tool call. The classifier rules are stored locally and contain 103 categories across allow, soft_deny, hard_deny, and environment sections, with the single hard_deny rule focused on data exfiltration spanning over 5,000 characters. Enterprise, API, Bedrock, Vertex, and Foundry deployments remain on opt-in for another month. Auto mode pauses after three consecutive blocks or twenty blocks in a session, and broad allow rules such as python:* are disabled while narrow permissions continue to function. Administrators should populate the twenty environment fields, currently only one-third configured on clean machines, before the rollout date.