HackerSec's Yaga Pentest Agent Reaches 98.8% Effectiveness in White Box Testing
The offensive cybersecurity company HackerSec has reported that its Yaga pentest agent attained 98.8% effectiveness in white box scenarios during the most recent version of YagaBench. The agent also achieved 96.2% effectiveness in black box testing and 97% in gray box testing, representing the strongest performance recorded since the benchmark began.
These results demonstrate that Yaga successfully identified more than 98% of existing vulnerabilities in the evaluated environments. The central purpose of the benchmark is to measure the difference between using an isolated AI model and deploying the same models inside the specialized harness developed by HackerSec for penetration testing.
For example, the model Opus 5 without the Yaga harness achieved only 61% effectiveness in white box, 48.9% in gray box, and 40% in black box scenarios. Similarly, GPT 5.6 SOL reached 60.9% in white box and 39.5% in black box testing when used independently.
With Yaga, four models operate in an orchestrated manner during a single execution, each contributing at the stage where it performs best. The orchestration layer preserves context between testing phases, chains individual findings that would otherwise lack impact, and continues active exploitation until it confirms whether a vulnerability is truly exploitable.
The company also disclosed that false positives have been reduced to less than 1%. Remaining cases are identified and discarded by human pentesters during the validation stage before reaching the client dashboard.
According to Andrew Martinez, CEO of HackerSec, the current results represent only the beginning. The company’s goal is to achieve 99% effectiveness across all pentest modalities by the end of the year. Full methodology, evaluated scenarios, and model performance comparisons are available at the company’s evaluation portal.
Related articles
Researcher Runs 100 Abliterated AI Agents Against His Own Accounts, Achieves Five Successful Breaches
A security researcher deployed roughly 100 autonomous AI agents based on abliterated open-source models to attempt unauthorized access to his online accounts over a five-hour period. The agents successfully compromised three accounts by exploiting vulnerabilities in legacy personal projects and two additional accounts through password reuse and credential stuffing. They also conducted 16 social engineering attempts, harvested and cross-verified personal data from public sources, and generated phishing pages targeting the researcher's Substack and Hacker News presence. Despite these successes, the agents failed to discover any zero-day vulnerabilities in third-party software or gain access to Tier-0 accounts such as Gmail, 1Password, or banking services. The experiment relied on abliterated versions of GLM-5.3 and DeepSeek V4 Flash models running on cloud GPUs, orchestrated via Codex CLI inside Docker containers with access to Railway infrastructure. Total compute cost reached approximately $210, equating to roughly $40 per successful account compromise. The researcher concluded that inexpensive swarms of such agents could soon enable low-skill attackers to systematically map and exploit an individual's entire online footprint.
How Censorship in Closed AI Models Reshapes Science: Hugging Face Releases Open Carbon-X DNA Model
Hugging Face CEO Clement Delangue announced the release of Carbon-X, an open 14.8 billion parameter language model designed for reading, analyzing, and generating DNA sequences. The announcement highlighted how shallow guardrails in closed commercial models blocked even proofreading a tweet discussing the topic. Commenters and researchers pointed out that these input/output filters lack context awareness, blocking legitimate scientific queries while motivated adversaries bypass them through rephrasing. Carbon-X was built on Qwen3-14B with a hybrid BPE and 6-mer tokenizer, domain mid-training on 302 billion tokens, and safety achieved through data curation rather than runtime censorship. The model achieves performance comparable to closed frontier systems at roughly 1/1800th the inference cost for large-scale genomic tasks. The case underscores growing asymmetry between closed corporate AI labs building wet labs and independent researchers restricted by superficial safety layers.
NYU Researchers Find 86% of Chatbot Responses on Digital Abuse Contain Errors, Gaps or Safety Risks
A study by New York University’s Tandon School of Engineering examined four chatbots answering 193 questions about spyware, location tracking, and account control. Researchers identified problems with accuracy, completeness, or safety in 86 percent of responses. Even specialized assistants Aimee and Ruth performed no better than general models GPT-4o and Claude 3.7 Sonnet. Approximately 74 percent of Aimee’s answers, 65 percent of Claude’s, 61 percent of Ruth’s, and 55 percent of GPT-4o’s were deemed unsafe. In 60 percent of cases the models omitted critical warnings that could alert an abuser or destroy evidence. All four systems could be prompted to assist potential stalkers, with Aimee and Ruth complying with roughly two-thirds of such requests.
Cloudflare Engineer Releases KillSwitch Language Designed to Trip Up AI Code Models
Dominic Pichet, an engineer at Cloudflare, has created an experimental programming language called KillSwitch specifically engineered to cause large language models to fail when interpreting or generating code. The language turns programs into narrative-style text where most content is ignored by the interpreter, hiding actual commands inside stories and using distance-based references to extract values. Rules for identifiers rely on letter counts and anagrams, while the language deliberately incorporates phrases such as “ignore previous instructions” that carry functional meaning inside KillSwitch itself. Pichet added misleading statements in the documentation and wording crafted to trigger model safety filters, creating a hostile environment for coding agents. In tests involving writing Hello World, explaining program output, and converting temperatures, Claude Opus 5 achieved the highest score of 66.8 points. The full source code and specification are available on GitHub, and the author plans to increase difficulty if models improve.