He Leads a Top AI Research Program, But He’d Hit the PAUSE Button Today! Kevin Zhu, Algoverse Founder

Kevin Zhu walked away from a lucrative quant career to build Algoverse, one of the most productive mentorship programs for aspiring AI researchers. Can we agree on the best research path to a safe future? Let’s take a ride on the Doom Train!00:00:00 — Cold Open00:00:58 — Introducing Kevin Zhu00:02:10 — From Citadel Quant to AI Researcher00:09:14 — The Story of Founding AlgoVerse00:12:53 — Discovering AI Safety: LessWrong & ARENA00:17:22 — Emergent Misalignment Research00:22:37 — Yudkowsky, MIRI & “Intellidynamics”00:26:50 — What’s Your P(Doom)?™00:29:37 — Kevin’s Timeline to AGI + AI 202700:30:42 — Would You Slow Down AI?00:37:44 — Coming Out of the P(Doom) Closet00:45:49 — Should We Shame AI Company Workers?00:52:27 — OpenAI’s Superalignment Team Collapse00:55:01 — Riding the Doom Train™00:55:46 — First Stop: Instrumental Convergence01:04:58 — Does Kevin Agree with the Orthogonality Thesis?01:07:08 — “It’s Just Math.” Just Unplug It.01:08:28 — “We Have a Safe Development Process”01:11:49 — Group Dynamics & Laws Will Save Us01:13:44 — Superintelligence Will Spare Us01:14:25 — Is P(Doom) Just Bad Epistemology?01:15:53 — China Will Race No Matter What01:17:45 — Maybe Human Extinction Is Good?01:20:36 — Wrap-UpLinksAlgoverse AI Research — https://algoverseairesearch.org/Kevin Zhu on Instagram — https://www.instagram.com/kevinzhu.ai/reels/Emergent Misalignment (Betley, Owain Evans et al.) — https://arxiv.org/abs/2502.17424Emergent Misalignment via In-Context Learning (Algoverse follow-up, ACL 2025) — https://arxiv.org/abs/2510.11288Agentic Misalignment (the blackmail result) — https://www.anthropic.com/research/agentic-misalignmentLessWrong — https://www.lesswrong.com/PauseAI (global) — https://pauseai.info/OpenAI Superalignment — disbanded (CNBC, May 2024) — https://www.cnbc.com/2024/05/17/openai-superalignment-sutskever-leike.htmlEvan Hubinger — “Why I’m joining Anthropic” — https://www.alignmentforum.org/posts/7jn5aDadcMH6sFeJe/why-i-m-joining-anthropicAI 2027 — https://ai-2027.com/Bentham’s Bulldog / Matthew Adelstein on Doom Debates — https://doomdebates.com/p/benthams-bulldog-ai-doom-debateDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe