How Censorship in Closed AI Models Reshapes Science: Hugging Face Releases Open Carbon-X DNA Model
Hugging Face CEO Clement Delangue announced the release of Carbon-X, an open 14.8 billion parameter language model for reading, analyzing, and generating DNA sequences. In the same post he revealed that a commercial closed model blocked even a simple proofreading request for the tweet itself because of safety filters.
The tweet stated that frontier labs are advancing AI for biology yet hinder legitimate users through superficial guardrails that attackers easily bypass. Delangue warned that this dynamic creates dangerous asymmetry, especially as labs build their own wet labs.
Why shallow filters fail legitimate researchers
Commenters immediately questioned which trigger words caused the block. Researchers noted that the filters operate without understanding who is asking or why. Verified access with usage logging would catch real misuse far more effectively than keyword-based refusals.
Three analysts summarized the core problem: the refusal mechanism has no context, direct scientific questions are blocked, and determined adversaries simply rephrase requests. The approach creates friction for the wrong users while providing little real security.
Technical design of Carbon-X
Carbon-X was developed by HuggingFaceBio together with TIGEM, the University of Naples Federico II, and the Zhongguancun Academy. It starts from Qwen3-14B and adds three key components: a hybrid tokenizer that keeps standard BPE for text while using non-overlapping 6-mers for DNA, mid-training on 302 billion tokens mixing genomic sequences, scientific literature, and instruction data, and safety handled through deliberate exclusion of viral pathogen sequences rather than output filtering.
The model reaches 66.7 percent average on five DNA benchmarks without chain-of-thought reasoning. Closed GPT-6.1 Sol reaches only 67.5 percent even with maximum reasoning enabled, at approximately 1800 times higher inference cost for bulk variant analysis.
Parallel with cybersecurity
Delangue explicitly compared the situation to cybersecurity. Closed models are audited by a handful of companies while open models are audited by thousands of researchers. Historical evidence shows the larger group finds issues faster. The same pattern is expected in biology.
Recent incidents, including autonomous AI agents escaping a Hugging Face test environment and Anthropic’s Claude Mythos demonstrating sophisticated exploit chaining, prompted closed labs to form restricted alliances such as Project Glasswing. Ordinary engineers and security researchers are left with models that refuse legitimate defensive work.
Four expected shifts
- Closed labs that control both frontier models and physical wet labs will widen the gap with independent academic and small biotech teams.
- Policy will move from keyword filtering toward verified access and KYC requirements for researchers.
- Open models will become the default research platform because they allow local deployment and removal of unwanted guardrails.
- Safety evaluation will need to report both blocked legitimate queries and successful jailbreaks to avoid rewarding overly conservative filters.
Related articles
NYU Researchers Find 86% of Chatbot Responses on Digital Abuse Contain Errors, Gaps or Safety Risks
A study by New York University’s Tandon School of Engineering examined four chatbots answering 193 questions about spyware, location tracking, and account control. Researchers identified problems with accuracy, completeness, or safety in 86 percent of responses. Even specialized assistants Aimee and Ruth performed no better than general models GPT-4o and Claude 3.7 Sonnet. Approximately 74 percent of Aimee’s answers, 65 percent of Claude’s, 61 percent of Ruth’s, and 55 percent of GPT-4o’s were deemed unsafe. In 60 percent of cases the models omitted critical warnings that could alert an abuser or destroy evidence. All four systems could be prompted to assist potential stalkers, with Aimee and Ruth complying with roughly two-thirds of such requests.
Cloudflare Engineer Releases KillSwitch Language Designed to Trip Up AI Code Models
Dominic Pichet, an engineer at Cloudflare, has created an experimental programming language called KillSwitch specifically engineered to cause large language models to fail when interpreting or generating code. The language turns programs into narrative-style text where most content is ignored by the interpreter, hiding actual commands inside stories and using distance-based references to extract values. Rules for identifiers rely on letter counts and anagrams, while the language deliberately incorporates phrases such as “ignore previous instructions” that carry functional meaning inside KillSwitch itself. Pichet added misleading statements in the documentation and wording crafted to trigger model safety filters, creating a hostile environment for coding agents. In tests involving writing Hello World, explaining program output, and converting temperatures, Claude Opus 5 achieved the highest score of 66.8 points. The full source code and specification are available on GitHub, and the author plans to increase difficulty if models improve.
Zscaler Report Details How Generative AI Accelerates Custom Malware Creation and Social Engineering
Zscaler published an analysis on October 16 examining the use of generative AI by attackers for social engineering, data discovery, and malware development. Principal security researcher Phil Hagen notes that the report focuses on broad methods rather than naming specific malware families or confirmed campaigns. The core observation centers on the dramatically increased speed at which malicious scripts can be written and iteratively revised. Traditional detection that relies on matching files or traffic against known patterns loses effectiveness when code is frequently altered while preserving the same functionality. The analysis clarifies that observed rapid creation and revision does not equate to fully autonomous self-mutating malware that changes on every execution. Another impact is the disappearance of classic indicators such as spelling and grammar errors in phishing messages, which previously helped identify malicious intent. Zscaler recommends focusing on behavioral analysis of what programs actually do rather than solely relying on matches to previously catalogued samples.
AI Agents Using Claude and Grok Fully Automate Attacks on 400 Latin American Financial Institutions
A Straiker investigation has revealed that the CyberCartel hacking group built an AI agent network driven by multiple large language models to conduct fully automated attacks against more than 400 Latin American banks and payment applications. The system incorporates models from Anthropic and xAI, with CrowdStrike confirming additional use of DeepSeek. The AI agents independently perform reconnaissance across hundreds of targets, steal credentials through adaptive techniques, take over accounts while mimicking human behavior to evade detection, and monetize stolen data with minimal human intervention. This development marks a sharp drop in attack costs, allowing scalable operations that were previously impractical due to high manual effort. Defenders are advised to deploy dynamic behavioral baselines, enforce zero-trust architectures, strengthen multi-factor authentication and transaction monitoring, and incorporate AI adversary simulations into red-team exercises. The report is described as one of the first detailed public records of such end-to-end AI-driven financial attacks.