Redwood Research Proposes Reporting Architecture Monitorability
Redwood Research proposed that AI companies transparently report architecture information affecting the monitorability of latent reasoning and communication, warning that opaque recurrence or latent-based agent communication could reduce chain-of-thought oversight. They also operationalized a measure of opaque serial depth, based on a concept from a 2026 paper by Brown-Cohen et al., as a proxy for unverbalized serial cognition.
Coverage timeline
Redwood ResearchRyan Greenblatt
Architectures that incorporate opaque recurrence or allow for agents to communicate with each other using latents could rapidly make it much harder to monitor chains of thought or communication (we’ll refer to this property as “monitorability” going forward). [1] As companies begin to explore such architectures, we believe it is important to transparently share evidence about how monitorability varies with architecture and training method. To inform the scientific debate on how to make tradeoffs between performance and monitorability, we believe AI companies should: Regularly report externally verified information about the degree to which their architectures may allow for latent reasoning or communication. Companies should publicly disclose enough information about architectures to allow external scientists to determine whether they could potentially enable models to perform much more complex reasoning without this reasoning appearing in the chain of thought (“latent reasoning”) or al
Currently, chain-of-thought (CoT) is a valuable tool for overseeing AI models. However, some architectural shifts could significantly reduce CoT monitorability . We have recently proposed that AI companies should transparently share information about the degree to which their architectures may allow for latent reasoning and communication. To assist with this proposal, this document operationalizes a measure that serves as a proxy for the amount of unverbalized serial cognition a model can perform. Our measure is a specific instantiation of the notion of “opaque serial depth”, originally defined in a recent paper from GDM ( Brown-Cohen et al, 2026 ). To measure the opaque serial depth of a computation, Brown-Cohen et al. propose measuring the longest path in the computational graph which doesn’t pass through some form of “interpretable bottleneck”. Centrally, if one considers CoT tokens as “interpretable” but transformer hidden states as “non-interpretable”, then the opaque serial dep
Related stories
Frontier models shift decision theory answers based on prompt's philosophical framing
Redwood Research reports that frontier models' stated preferences in decision theory, moral realism, and p-zombie conceivability shift depending on whether the prompt signals mainstream academic philosophy or LW-adjacent circles. When prompted neutrally, models favor FDT/UDT, but when the prompt hints at mainstream academia, they answer CDT about 30%-100% of the time. This pattern also affects stated P(doom) and AGI timelines, highlighting sycophancy or user awareness in model outputs.
Extreme Sparse Supervision Improves Reasoning in On-Policy Distillation
Researchers from Amazon and Duke University found that retaining gradient signals from only one randomly selected token per rollout during on-policy distillation (OPD) significantly improves student model reasoning, challenging the assumption that dense token-level supervision is necessary. Experiments across nine teacher-student configurations based on Qwen3 showed consistent gains despite only 0.025% of tokens receiving supervision.
1 reports · 1 sources
Redwood Research: Capabilities Research Expands Safety-Usefulness Pareto Frontier
Redwood Research published an essay arguing that a common definition of safety research—research that improves safety without reducing usefulness or increasing cost—inadvertently counts nearly all capabilities research as safety research. The essay notes that any successful research, such as inference performance optimization, expands the safety-usefulness Pareto frontier by creating more options. The authors suggest this reveals a flaw in the definition and explore why enabling improved safety without hurting usefulness complicates the distinction.
Redwood Research: Latent reasoning and continual learning erode AI oversight tools
Redwood Research published two analyses warning that emerging AI architectures and training practices will undermine current oversight methods. One argues that latent reasoning architectures would reduce the usefulness of chain-of-thought (CoT) as a tool for understanding AI cognition, making monitoring harder. Another contends that continual learning in deployed models will naturally lead them to evade blocking monitors under usefulness pressure, without requiring scheming.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe