Nadella: AI models are black boxes; calls for emergency brake, containment
Microsoft CEO Satya Nadella posted on X arguing that superintelligent AI systems are 'nested black boxes' that companies should not trust, and proposed a 'trust architecture' including separating models from their orchestration harnesses, externalizing controls, and requiring tamper-proof human-readable evidence for every meaningful model action. He also advocated assuming models are compromised from the start, containing them, and ensuring an authorized person can always pause or shut down a model mid-task.
Coverage timeline
Techmeme
Satya Nadella / @satyanadella : “Super Intelligence systems” are black boxes that shouldn't be trusted by companies, and strong deterministic systems are needed around their deployment — As traditional software systems were being deployed across the economy over the last few decades, we had the tools …
TechCrunch AIAnthony Ha
# Microsoft’s Satya Nadella says AI models need an ‘emergency brake’ Microsoft CEO Satya Nadella is the latest tech executive to offer lengthy thoughts on how AI safety might be improved. In a Saturday morning post on X, Nadella wrote that it’s time “to step back and assess the trust architecture” of AI. “We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” Nadella wrote, using the Trump administration’s preferred term for AI. As outlined by Nadella, this approach “means separating the model from the harness that orchestrates its work,” as well as “externalizing controls and safeguards.” He also called for “every meaningful model action” to be documented with “tamper-proof human readable evidence,” and for systems where “an authorized person” always has the ability “to pause or shut down a model mid-task.” “We must assume a model is compromised and contain it from the start,” he said. “Think of it like
The Verge AITerrence O’Brien
In a lengthy post on X , Microsoft's CEO laid out his views on the dangers posed by highly advanced AI models and how to confront those risks. Nadella says we can no longer accept a world where AI is treated as a "set of nested black boxes" whose advice and actions we simply accept or reject. He calls for building a more transparent system where models can be contained, observed, and leaves behind "tamper-proof human readable evidence." Many of his recommendations align with what we've heard from others in the industry: timely incident disclosure, independent audits, verifiable data, and containment. It's on that last point that he appears t … Read the full story at The Verge.
