Why CPUs still matter in the age of AI agents

As AI evolves from conversational chatbots to autonomous agents, CPUs are becoming an increasingly important part of the infrastructure equation. In this episode, The New Stack speaks with Bhumik Patel of Arm and Mo Farhat of Google about how CPUs act as an “air traffic controller” for agentic workloads, handling orchestration, data preparation, semantic search, vector databases, code execution and API calls alongside GPUs and TPUs. Smaller AI models, including summarizers and evaluators, can also run effectively on CPUs for specialized tasks.  As agents increasingly generate and execute code, secure sandboxing becomes critical. Google’s gVisor and GKE Agent Sandbox provide isolation and scalable environments, with the latter supporting up to 300 sandboxes per second per cluster. The discussion also explores efficiency and cost, with Google highlighting Axion’s price-performance and energy-efficiency advantages across different workload types. Ultimately, the shift toward agentic AI is creating a more diverse compute environment where CPUs, GPUs and TPUs each play complementary roles in delivering scalable, efficient AI applications. Learn more from The New Stack around the latest in CPUs in the world of AI agents:  AI Agents Will Eat Enterprise Software, Just Not in One Bite  How to ground AI agents in accurate, context-rich data  Join our community of newsletter subscribers to stay on top of the news and at the top of your game.