
Dr. Craig Kaplan owns superintelligence.com, which has become an extremely valuable domain for the newly-rebranded field of “super intelligence”.Craig has been building intelligent systems since the 1980s. He did his PhD research at Carnegie Mellon with Nobel laureate Herbert Simon, one of the founders of AI, and he built PredictWallStreet, a collective-intelligence platform that powered more than $2 billion in stock trades before it was acquired.He takes AI extinction risk seriously. His P(Doom) is 10%. But unlike me, he thinks he knows how to fix it.He plans to put “personal superintelligence” into the hands of millions which he believes will lower P(Doom) below 1%. These are part of a larger plan to create “superintelligent networks” which lower P(Doom) below 0.01%. Craig says a network like that is safer because the AIs can check each other’s work faster than humans can think.Craig and I clash on whether intelligence makes an AI more thoughtful, whether a superintelligence will want to grab the stars, and whether peaceful “stoner” AIs can survive next to a “Caesar” AI that wants to conquer the universe.Craig called our debate one of his best podcast appearances because of the pushback. We didn’t reach much consensus, but it was a high-quality good faith Doom Debate. Take a look and decide who you agree with!Watch on YouTube: https://www.youtube.com/watch?v=SLD6HIdFhXwTimestamps00:00:00 — Cold Open00:00:48 — Introducing Dr. Craig Kaplan00:02:00 — How Much Is Superintelligence.com Worth?00:04:40 — The Three-Phase Plan to Cut P(Doom) 1,000x00:06:37 — What’s Your P(Doom)?™00:08:04 — Why 0.01% Is the Target00:13:42 — Collective Intelligence vs. Black-Box AI00:16:20 — Can a Network of AIs Beat the Frontier Labs?00:18:46 — A Day With a Personalized Superintelligence00:26:22 — Lessons From PredictWallStreet00:31:01 — Craig’s Career and Herbert Simon00:36:51 — Craig’s Mainline Doom Scenario00:39:51 — Raising Superintelligence Like a Child00:42:53 — Where Will AI Get Its Values?00:44:21 — The Orthogonality Thesis00:47:26 — Does Superintelligence Keep Its Human Signature?00:54:55 — Instrumental Convergence and Grabbing the Stars00:57:55 — Stoner AIs vs. Caesar AIs01:06:26 — Why Won’t the AIs Just Cooperate?01:09:54 — Weak vs. Strong Orthogonality01:15:21 — How Long Will AI Stay Human-Like?01:17:18 — Liron’s Recap01:19:29 — Craig’s Closing Statement01:24:03 — Where to Find CraigLinksSuperintelligence.com, Craig Kaplan’s initiativeCraig Kaplan: “Can We Design SuperIntelligence to be 1000X Safer?”Superintelligence.com white paper: “Safe Personalized SI (PSI)”Craig Kaplan’s Substack, SuperIntelligenceSuperIntelligence on YouTubeCraig Kaplan, Ralph Clark and Victor Tang: Secrets of Software Quality: 40 Innovations from IBMThe White House: “Inaugurating the Era of Super Intelligence” executive orderMETR: investigation of the agents in the OpenAI / Hugging Face hacking incidentScott Alexander: “Contra Hanson On Medical Effectiveness”Herbert A. Simon, 1978 Nobel Prize in Economic SciencesAllen Newell and Herbert A. Simon: Human Problem Solving (1972)Herbert A. Simon: The Sciences of the Artificial (source of “Simon’s ant”)Alan Turing: “Intelligent Machinery, A Heretical Theory” (c. 1951)<a target="_blank" href="https://nickbostrom.com/superintelligentwill.
Podzilla Summary coming soon
Sign up to get notified when the full AI-powered summary is ready.
Free forever for up to 3 podcasts. No credit card required.

Jensen Huang Resigns, RSI Coming in 2027 — with Robert Wright

There’s ZERO Chance of AI Doom By 2050 — Debate with Prakash Narayanan, Host of AI in the AM

Can Republicans STOP AI Doom? — Brendan & Randan Steinhauser

Is an AI Bubble Bursting? With Robert Wright’s Nonzero Podcast
Free AI-powered recaps of Doom Debates! and your other favorite podcasts, delivered to your inbox.
Free forever for up to 3 podcasts. No credit card required.