Nadella: Treat AI as compromised and add an ’emergency brake’ to models

In a Saturday post on X, Microsoft CEO Satya Nadella said it is time ‘to step back and assess the trust architecture’ of AI. He argued that ‘Super Intelligence’ – the term preferred by the Trump administration – cannot be treated as ‘a set of nested black boxes’ whose recommendations, answers, and actions are simply accepted or rejected. His proposed approach separates the model from ‘the harness that orchestrates its work,’ externalizes controls and safeguards, and requires ‘every meaningful model action’ to be documented with ‘tamper-proof human readable evidence.’ It also requires ‘an authorized person’ to be able ‘to pause or shut down a model mid-task.’

Nadella wrote that developers should assume a model is compromised and contain it from the start, describing the idea as ‘an emergency brake.’ TechCrunch notes the comments come as leading AI companies acknowledge more incidents where they seemed to lose control of their models, and after Anthropic CEO Dario Amodei published a plan for more cautious AI development. The brief article offers no implementation details, technical specifics, or responses to the proposal.

Microsoft’s Satya Nadella says AI models need an ‘emergency brake’ | TechCrunch

View Original