Reinforcement Learning and Quantum Decoding | Vlad Sivak Google Quantum AI

How can reinforcement learning help quantum computers learn from their own errors?In this episode, Vlad Sivak, researcher at Google Quantum AI, joins us to discuss how reinforcement learning can improve quantum computing, from error correction and decoding to the calibration of quantum hardware. Sivak explains how reinforcement learning allows an AI agent to interact with a quantum system, learn from its errors, and discover how to improve its performance without being explicitly told which actions to take.We explore how quantum error correction works, why decoding error syndromes is such a challenging problem, and how machine learning can help quantum computers adapt to changing error patterns. Sivak describes his team's work using reinforcement learning to dynamically adjust the weights of a quantum error-correction decoder, allowing it to make better-informed decisions as the properties of the hardware change over time. We also discuss why conventional optimization methods can struggle with quantum systems that are high-dimensional, stochastic, and constantly drifting.Sivak explains how reinforcement learning can improve quantum hardware calibration by compensating for unknown effects that traditional, model-based approaches may miss. Rather than stopping a quantum computer to run dedicated calibration experiments, the long-term vision is to use the stream of error signals generated during computation to continuously update the system's control parameters. This could eventually allow quantum computers to adapt to hardware drift while continuing to operate.Beyond quantum computing, we discuss the broader potential of reinforcement learning as a paradigm for autonomous discovery. From AlphaGo's unexpected strategies to recent advances in mathematical reasoning and programming, Sivak explores how systems can learn by experimenting, evaluating outcomes, and discovering solutions for themselves. We examine the challenges of scaling reinforcement learning, the relationship between classical AI and quantum computing, and the possibility of building quantum machines that can learn to maintain and improve their own performance.Follow us for more technical interviews with the world’s greatest scientists:Facebook: https://www.facebook.com/632nmPodcastOfficialTwitter: https://x.com/632nmPodcastInstagram: https://www.instagram.com/632nmpodcast?utm_source=ig_web_button_share_sheet&igsh=ZDNlZDc0MzIxNw==LinkedIn: https://www.linkedin.com/company/632nm/about/Substack: https://632nmpodcast.substack.com/Follow our hosts!Mikhail Shalaginov: https://www.linkedin.com/in/mikhail-shalaginov/Yudong Cao: https://www.linkedin.com/in/yudong-cao-25b6a929/Subscribe:Apple Podcasts: https://podcasts.apple.com/us/podcast/632nm/id1751170269Spotify: https://open.spotify.com/show/4aVH9vT5qp5UUUvQ6Uf6ORWebsite: https://www.632nm.comTimestamps:00:00 - Intro00:58 - Protecting Quantum Information09:28 - Reinforcement Learning for QEC13:47 - Qubit Calibration19:10 - Where Do Noise and Error Come From?21:13 - Scalability of RL Methods25:28 - Google's AlphaQubit31:54 - RL vs Human Calibration33:26 - Settling on RL vs Other ML Approaches39:57 - Discovery in the Age of AI44:59 - Collaboration Between Google Deepmind and Quantum AI46:17 - Combining Quantum and AI48:35 - Will Human Input Become Obsolete?55:00 - Controlling Bosonic Codes1:05:17 - Break Even Point1:121:51 - Can Syndrome Extraction Be Fault Tolerant?1:15:41 - Building Gates1:20:22 - Surface Codes1:27:38 - Scaling Superconducting Qubits1:30:22 - How Quantum Decoders Work1:35:47 - Model Sizes For Larger Quantum Computers1:37:48 - What Problems Could Be Solved With Unlimited Resources?