An Approach to Technical AGI Safety and SecurityWe have written a paper on our approach to technical AGI safety and security. This post is a copy of the extended abstract, which…Apr 8Apr 8
Negative Results for Sparse Autoencoders On Downstream Tasks and Deprioritising SAE Research…Lewis Smith*, Sen Rajamanoharan*, Arthur Conmy, Callum McDougall, Janos Kramar, Tom Lieberum, Rohin Shah, Neel NandaMar 26Mar 26
Introducing our short course on AGI safetyWe are excited to release a short course on AGI safety for students, researchers and professionals interested in this topic. The course…Feb 14A response icon2Feb 14A response icon2
Steering Gemini using BIDPO vectorsBy Alex Turner and Mark KurzejaJan 31A response icon1Jan 31A response icon1
MONA: A method for addressing multi-step reward hackingMONA enhances safety when we train an AI system to perform some task that takes multiple steps. Training an AI with MONA reduces its …Jan 23Jan 23
Human-AI Complementarity: A Goal for Amplified OversightHow do we ensure humans can continue to oversee increasingly powerful AI systems? We argue that achieving human-AI complementarity is key.Dec 23, 2024Dec 23, 2024
AGI Safety and Alignment at Google DeepMind: A Summary of Recent WorkBy Rohin Shah, Seb Farquhar, and Anca DraganOct 18, 2024Oct 18, 2024
Goal Misgeneralisation: Why Correct Specifications Aren’t Enough For Correct GoalsBy Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, and Zac Kenton. For more details, check out…Oct 7, 2022A response icon1Oct 7, 2022A response icon1
Discovering when an agent is present in a systemNew, formal definition of agency gives clear principles for causal modelling of AI agents and the incentives they face.Aug 25, 2022Aug 25, 2022
Your Policy Regulariser is Secretly an AdversaryBy Rob Brekelmans, Tim Genewein, Jordi Grau-Moya, Grégoire Delétang, Markus Kunesch, Shane Legg, Pedro A. OrtegaMar 24, 2022Mar 24, 2022
