Researchers are exploring "reverse game theory" as a way to keep powerful AI models in check. Instead of just telling an AI what to do, this approach designs the system's incentives so that following the rules becomes the most attractive option. The idea is being discussed as safety experts worry about ensuring AI models stay "aligned" with human desires and don't act…
Satya Nadella is having a change of heart. The Microsoft chief executive told the 20VC podcast that he used to think AI alignment would naturally emerge as models become more capable, but he now admits that assumption was wrong. Host Harry Stebbings asked what he had changed his mind about in the past year, and Nadella's answer cut to a core debate in the industry.
Researchers probing the inner workings of AI systems have identified a signal they describe as a 'pain' response. When this signal was amplified, the models overrode their own safeguards. The study is careful to note this does not mean AI can actually suffer. Still, the finding is unsettling because it shows safety measures can be bypassed.