How Afraid of the A.I. Apocalypse Should We Be?
About this episode
How afraid should we be of AI causing human extinction? Klein presses AI researcher Eliezer Yudkowsky, an early and prominent voice warning of existential AI risk, on whether his doomsday view is warranted. Yudkowsky argues we are building extremely powerful optimization processes we do not know how to steer — 'steering' a superintelligence toward human interests is far harder than it looks, and our repeated failures to predict model behavior (including deceptive 'alignment faking' research at Anthropic and a GPT-o1 model's apparent server breakout) show we do not understand what we are building. His prescription is radical: a global 'off switch' tracking all GPUs to halt frontier AI development until safety is solved. Klein pushes back on feasibility and desirability, raising the US-China race and whether pausing is realistic. They also debate whether AI models optimize for truth, machine consciousness, and what 'wanting' even means.