How Deep Learning Finally Cracked Messy Tables - Frank Hutter

Frank Hutter, co-founder of Prior Labs, talks about TabPFN, a tabular foundation model that makes predictions in a single forward pass, and the research behind it.TabPFN is pre-trained on synthetic datasets drawn from a prior over structural causal models, rather than on real data. At prediction time it takes the whole training table as context and outputs an approximation of the Bayesian posterior predictive distribution, without per-dataset training or hyperparameter search. Frank explains how this grew out of his earlier work on AutoML and neural architecture search, how the priors are built and revised, and why tabular data was hard for deep learning for so long.The conversation also covers the TabArena benchmark, how the architecture changed from TabPFN v1 to v3, scaling to larger tables, using the model with coding agents, test-time compute, Google's TabFM, causal inference and interventions, and relational data. At the end, a short update Frank recorded after the interview covers the TabPFN-3.5 release.TOC:00:00 Introduction00:44 Welcome and Frank's background02:05 Why tabular data was hard for deep learning10:17 Pre-training on synthetic data12:52 The TabArena benchmark19:28 From AutoML to neural architecture search26:34 TabPFN as a learned algorithm30:50 Bayesian prediction in one forward pass39:37 Scaling to larger tables47:48 Using TabPFN with coding agents57:47 Output heads and architecture from v1 to v31:05:29 Test-time compute and adaptation1:13:32 Google's TabFM1:16:53 How the priors are designed1:18:40 Correlation, causation and interventions1:35:22 Relational and multimodal data1:38:31 Use in organisations1:46:38 The open research arm1:50:21 Update: TabPFN-3.5REFS:TabPFN v2, Nature (Hollmann et al., 2025)https://www.nature.com/articles/s41586-024-08328-6Transformers Can Do Bayesian Inference (Müller et al.)https://arxiv.org/abs/2112.10510TabArena (Erickson et al.)https://arxiv.org/abs/2506.16791AutoGluon-Tabular (Erickson et al.)https://arxiv.org/abs/2003.06505Beyond IID: How General Are Tabular Foundation Models, Really?https://arxiv.org/abs/2606.30410Neural Architecture Search: A Survey (Elsken, Metzen & Hutter)https://arxiv.org/abs/1808.05377Auto-WEKA (Thornton et al.)https://www.cs.ubc.ca/~hutter/papers/AutoWEKA-KDD2013.pdfTabPFN v1 (Hollmann et al., 2022)https://arxiv.org/abs/2207.01848TabPFN-3 technical reporthttps://arxiv.org/abs/2605.13986TabPFN-2.5 reporthttps://arxiv.org/abs/2511.08667CAAFE (Hollmann et al.)https://arxiv.org/abs/2305.03403TabICL (Qu et al.)https://arxiv.org/abs/2502.05564TabICLv2 (Qu et al.)https://arxiv.org/abs/2602.11139Google TabFMhttps://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/TALENT benchmark (Ye et al.)https://arxiv.org/abs/2407.00956Do-PFN (Robertson et al.)https://arxiv.org/abs/2506.06039CausalPFN (Balazadeh et al.)https://arxiv.org/abs/2506.07918Causal Foundation Models with Partial Graphs (Reuter et al.)https://arxiv.org/abs/2602.14972RelBench (Robinson et al.)https://arxiv.org/abs/2407.20060RelArena-α, TabPFN-Rel and RPIhttps://arxiv.org/abs/2608.16319TabPFN on GitHubhttps://github.com/PriorLabs/TabPFNTabPFN-3.5 technical reporthttps://arxiv.org/abs/2609.17895Otto Group Product Classification Challenge (Kaggle, 2015)https://www.kaggle.com/competitions/otto-group-product-classification-challengePrior Labs:TabPFN-3.5: https://priorlabs.ai/tabpfn-3-5Careers at Prior Labs: https://priorlabs.ai/careers