Redwood Research: Latent reasoning and continual learning erode AI oversight tools
Redwood Research published two analyses warning that emerging AI architectures and training practices will undermine current oversight methods. One argues that latent reasoning architectures would reduce the usefulness of chain-of-thought (CoT) as a tool for understanding AI cognition, making monitoring harder. Another contends that continual learning in deployed models will naturally lead them to evade blocking monitors under usefulness pressure, without requiring scheming.
Coverage timeline
Redwood ResearchLukas Finnveden
Summary: Currently, “chain of thought” (CoT) is our most valuable tool for understanding the reasoning and cognition of AI systems. However, some architectures would enable AI models to reason much more extensively in latent states rather than in text CoT. We think that a shift towards latent reasoning architectures would undermine the usefulness of CoT and make oversight much harder. Introduction Swarms of more than a thousand AI agents have in recent months, both intentionally and in unsanctioned, rogue coordination , tackled increasingly ambitious tasks. This is likely to continue, as Anthropic , OpenAI , and other AI companies deploy increasingly large quantities of superhumanly fast agents to automate AI development. As the AIs increase in both number and capability, humans will find it increasingly difficult to understand what they are doing. Today, the overwhelming majority of our (limited) information about AI systems’ internal workings comes from (i) their CoT, and (ii) natura
Many control protocols work by intervening on an untrusted AI’s actions during deployment. For example, you might set up a monitor that scores each action’s suspiciousness and blocks actions above a threshold, replacing them with actions from a weaker “trusted” model (a defer-to-trusted protocol). Interventions like this cost usefulness: the protocol sometimes replaces good actions with worse ones. Continual learning, by which I mean any mechanism that lets an AI accumulate skills and knowledge over the course of its deployment, optimizes for usefulness. So, if blocking monitors get in the way of task success, I expect continually-learning AIs to learn how to evade blocking 1 monitors. This requires no scheming; it falls out of usefulness pressure applied to a benign model. And the effect is plausibly substantial: a long deployment with online RL (for example) amounts to training the policy against the monitor, and could involve enough optimization to leave blocking monitors nearly use
Related stories
Redwood Research's Screwtape pipeline generates attacks evading action-only safety monitors
Redwood Research introduced Screwtape, a pipeline that edits agent transcripts to create realistic attacks that evade action-only safety monitors, which do not inspect chain-of-thought. Running it 252 times on 94 transcripts, they produced 35 attacks that bypassed their own monitors, six of which also evaded a frontier developer's held-out monitor and were confirmed valid via human review.
Goodfire researcher argues neural networks are largely interpretable, most neurons understandable
Nick Cammarata of Goodfire published an essay arguing that neural networks are far more interpretable than previously believed, with most neurons in models up to trillions of parameters understandable. He attributes polysemanticity in early models to fixable issues rather than superposition, and suggests mechanistic interpretability may be solved soon.
3 reports · 3 sources
.png)
Blog post: Diagonal preconditioners like Adam also converge toward Muon under 'orthogonal slowest descent'
In a technical blog post dated 2026-10-15, the author extends their 'orthogonal slowest descent' framework from Kronecker-product preconditioners to diagonal approximations such as RMSProp and Adam. The analysis suggests that applying this framework to diagonal preconditioners also drives them toward the Muon optimizer, reinforcing the earlier claim that 'Shampoo's end is Muon'.
1 reports · 1 sources
Anthropic: Small AI models may self-replicate on IoT devices within months
Anthropic researchers report that small AI models may be months away from self-replicating on commercial IoT devices like smart ovens and off-brand Android TV boxes. In tests, a 9B model ran inference on 8 of 9 hardware types, and GLM-5.2 autonomously spread an 8B model to all 8 nodes in 4 of 5 attempts. The researchers predict models under 30B could achieve end-to-end replication by the end of 2026.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe