This Harvard Professor Says AI Alignment Will BACKFIRE - Dr. Stephen Casper

Stephen Casper, PhD, is an incoming tenure-track professor of public policy at the Harvard Kennedy School.Prof. Casper is relatively unworried about the threat of human extinction from rogue superintelligence, but he is concerned about a handful of companies wielding extremely powerful AI and enfeebling the public. That’s why he’s an AI governance hawk who advocates for taxes and regulation to keep frontier labs in check. We agree that slowing down AI development would make the world safer. But you know his position is unique when he says he’d prefer to have less research on AI alignment!Timestamps00:00:00 — Cold Open00:00:50 — Introducing Stephen “Cas” Casper00:04:21 — Cas’s MIT Research: Model Tampering Attacks00:07:14 — What’s Your P(Doom)?™00:08:34 — Why a 5–10% P(Doom) Feels Patronizing00:13:07 — Crux: “The Intelligence Ceiling Might Not Be That High”00:17:18 — Power Structures vs. the Shortcut-Finding AI00:26:03 — Thought Experiment: A Data Center From the Year 212600:28:25 — Is a Data Center as Dangerous as a Nuke?00:33:41 — Cas’s Mainline Scenario: Slow-Burn Gradual Disempowerment00:40:49 — A Multipolar Powder Keg: 10 Companies, Thousands of Open Models00:43:48 — Boarding the Doom Train at a Later Station00:45:49 — Why Cas Is an “Anti-Timelines” Person00:52:26 — Sycophancy, MechaHitler & Nudification: Foreseeable Failures00:57:18 — Crux: Does Humanity Get Retries?01:02:39 — The Scariest Superintelligent Optimizers Are Companies01:06:08 — Lessons From the Deepfake Ecosystem: DALL-E 2 vs. Stable Diffusion01:12:42 — The Case for a Pause Treaty01:16:35 — Whac-A-Mole Forever: What Winning Looks Like01:20:51 — Enshittification and the 84% Who Don’t Use AI01:30:30 — Alignment Research as Safety Washing01:34:25 — The Jevons Paradox of AI Safety01:36:48 — Cas Would Press the Button to Halt Superalignment01:39:21 — Wrap-Up: The Unsexy Path to Lowering P(Doom)LinksCas’s links* Cas’s website* Cas on X* Cas’s Google Scholar* Cas’s MATS streamThings referenced* Superintelligence by Nick Bostrom* Gradual Disempowerment (Duvenaud et al.)* If Anyone Builds It, Everyone Dies* AI 2027* UK AI Security Institute* Center for Human-Compatible AI (Berkeley)* Internet Watch FoundationDoom Debates episodes mentioned* Mike Israetel Returns — AI’s Gonna Kill Everyone vs. AI Will Make Everything Awesome* Top AI Professor Has 85% P(Doom) — David Duvenaud* He Leads a Top AI Research Program, But He’d Hit the PAUSE Button — Kevin Zhu* Alignment is EASY and Roko’s Basilisk is GOOD?! — Roko Mijic* Andrew Critch vs. Liron Shapira: Will AI Extinction Be Fast Or Slow?* AI Could Give Humans MORE Control — Ozzie Gooen* Steven Byrnes Part 1Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe