Gaming Chip Pioneer Raja Koduri on China's AI Cost Advantage and Moving Beyond the GPU

In this episode, host David Goldman speaks with legendary graphics chip architect Raja Koduri, who explains why every gigawatt of AI infrastructure now costs $50 to $60 billion, and why China's goal of doing it for under $10 billion is the real threat to Western AI. Raja twice led graphics at AMD, directed Apple's graphics architecture and was chief architect at Intel. He argues that the real AI race isn't Nvidia vs. Google vs. Broadcom but China vs. the rest of the world, and that the new bottleneck isn't compute. It's memory.In this conversation, Raja joins TechSurge to talk about his new startup Oxmiq, which aims to turn "electrons to tokens super efficiently." He covers how 3D-stacked, hybrid-bonded memory could deliver 10x the bandwidth of today's HBM, why AI agents are changing how chips get designed, and why he thinks the next disruption to AI data centers "comes from the bottom."The conversation covers:✅ Why every gigawatt of AI infrastructure costs $50 to $60 billion, and where the $45 billion in hardware spend goes✅ The $24 trillion capital question: 400+ gigawatts of new compute needed by 2030✅ How China's under-$10 billion per gigawatt target creates a 5 to 6x cost gap✅ Why memory hierarchy, not raw compute, is now the real bottleneck in AI✅ How advanced packaging can unlock 10x bandwidth and 10x token generation, even on older 7nm nodes✅ Why OpenAI's Jalapeño chip shows that AI can speed up silicon design✅ Why the value of experienced engineers has gone up 100x in the age of AI coding agents✅ Leadership lessons from Steve Jobs at Apple and Lisa Su at AMD✅ Why Intel's decision to kill 3D XPoint memory came at "the wrong time"✅ Boom or bust: the financial engineering risk behind the AI infrastructure buildout✅ Token factories vs. token banks: why "the more boring you make it, the more it becomes fabulous"Guest Links: Raja Koduri: Founder of Oxmiq. LinkedIn: https://www.linkedin.com/in/raja-koduri-3a51611X: https://x.com/RajaXgOxmiq: https://oxmiq.ai Further Reading and ResourcesOpenAI and Broadcom announcement: https://openai.com/index/openai-broadcom-jalapeno-inference-chip/High Bandwidth Memory (HBM): The memory technology Raja's AMD team helped bring to market with HBM1 and HBM2, and the benchmark Oxmiq's 3D-stacked approach aims to beat by 10x. https://en.wikipedia.org/wiki/High_Bandwidth_Memory Intel 3D XPoint (Optane): The discontinued memory technology Raja says could have made Intel a major player in the inference era. https://en.wikipedia.org/wiki/3D_XPoint Chapters00:00 - A Gigawatt of AI Now Costs $60 Billion01:58 - Introducing Raja Koduri02:02 - What Oxmiq Builds: Electrons In, Tokens Out05:18 - The $24 Trillion AI Infrastructure Bill06:05 - China vs. the Rest of the World09:10 - Memory Is the New Bottleneck22:03 - How AI Agents Are Changing Chip Design36:30 - Lessons From Steve Jobs and Lisa Su44:54 - Advanced Packaging, Memory Prices, and Intel's Mistake55:38 - Boom or Bust: The Future of Token FactoriesAbout TechSurge:TechSurge Podcast shares the latest insights directly from legendary Silicon Valley leaders, daring new founders, and visionary technologists.🔔 Subscribe for weekly conversations at the intersection of technology advancement, market dynamics, and founder journeys.