Episode 20: First Boot, First Inference

In this episode of The IT Guy Show, Eric sits down with Damen Knight, CIQ Sr. Principal Automation Engineer, to talk about something every AI team knows but nobody budgets for: the hidden cost of configuring Linux for GPU workloads. Together they dig into why general-purpose Linux distributions were never really built for AI, what actually happens when you spend 30 to 60 minutes per node doing manual CUDA setup, and why a "pre-installed" stack and a "validated" stack are two very different things. From driver conflicts and framework dependency failures to the CIQ Linux Kernel (CLK) and NVIDIA authorization, this episode covers the infrastructure decisions that determine whether your AI program ships fast or gets stuck in configuration hell. Whether you're an ML engineer tired of fighting CUDA every time a kernel update drops, a sysadmin managing a growing GPU fleet, or a tech leader wondering why production AI is harder than it should be, this one is for you. Welcome to The IT Guy Show, your go-to destination for all things tech with Eric, your friendly neighborhood IT guy! Join Eric as he shares his wealth of knowledge, insights, and experiences gained from years of working in the IT industry. From troubleshooting common tech issues to exploring the latest trends and innovations, each episode is packed with practical tips, skilled advice, and engaging discussions. Connect with me! https://linktr.ee/itguyeric CIQ Press Release: https://ciq.com/blog/rlc-pro-ai-is-here/ RLC Pro AI: https://ciq.com/products/rocky-linux/pro/ai/ NVIDIA CUDA Toolkit: https://developer.nvidia.com/cuda-toolkit Chapters: 00:00 Stream start 00:50 Introduction 02:43 Sponsor: CIQ 04:08 AI at CIQ 06:02 State of AI 19:20 General-purpose limitations 31:37 RLC Pro AI 43:53 What's next in AI? 52:47 Wrap up Graphics and bumpers by Free Hive Agency Photo by Igor Omilaev on Unsplash