Researchers Extract Proprietary Reasoning Traces from Anthropic, OpenAI and Google LLMs, Revealing Hidden Secrets

A new research paper demonstrates that the hidden reasoning traces generated by leading large language models from Anthropic, OpenAI and Google can be recovered in full by using weaker models from the same providers, without attacking the target model itself.

The technique, described in the preprint "Stealing Reasoning Traces from Proprietary LLM APIs," exploits the fact that providers return encrypted reasoning blocks to clients so the model can continue multi-turn conversations. Researchers discovered that these blocks can be submitted to a less-protected smaller model, which is then prompted to reproduce the original chain-of-thought verbatim.

Testing on 120 Codeforces problems confirmed that the recovered traces matched the token counts reported in API metadata, proving the output was genuine rather than hallucinated. When the same method was applied to publicly available agent logs on GitHub and Hugging Face, analysts recovered 315,320 reasoning blocks containing 704 unique secrets, among them 62 API keys, 33 passwords and 24 access tokens that existed only inside the encrypted portions.

Multiple attack vectors identified

The paper outlines four distinct risks. First, competitors can distill high-quality reasoning from stronger models without triggering distillation defenses. Second, internal safety reasoning that is normally filtered from visible answers can be extracted, revealing detailed discussions of criminal techniques. Third, the mechanism can be reversed to embed malicious instructions inside shared logs that later get replayed by other users as part of the model’s own past reasoning.

Finally, the visible summaries returned by APIs were shown to omit or alter the actual sequence of thoughts. On certain AIME problems, the full trace revealed that the model first recalled a known answer before attempting to solve the problem, a behavior absent from the polished summary presented to users.

The authors advise developers to treat encrypted reasoning blocks as sensitive credentials and to avoid publishing or reusing uninspected agent trajectories from public repositories.