Adding ML layer to Search: Hybrid Search Optimizer with Daniel Wrigley and Eric Pugh

Vector Podcast website: https://vectorpodcast.comHaystack US 2025: https://haystackconf.com/2025/Federated search, Keyword & Neural Search, ML Optimisation, Pros and Cons of Hybrid searchIt is fascinating and funny how things develop, but also turn around. In 2022-23 everyone was buzzing about hybrid search. In 2024 the conversation shifted to RAG, RAG, RAG. And now we are in 2025 and back to hybrid search - on a different level: finally there are strides and contributions towards making hybrid search parameters learnt with ML. How cool is that?Design: Saurabh Rai, https://www.linkedin.com/in/srbhr/The design of this episode is inspired by a scene in Blade Runner 2049. There's a clear path leading towards where people want to go to, yet they're searching for something.00:00 Intro00:54 Eric's intro and Daniel's background02:50 Importance of Hybrid search: Daniel's take07:26 Eric's take10:57 Dmitry's take11:41 Eric's predictions13:47 Doug's blog on RRF is not enough16:18 How to not fall short of the blind picking in RRF: score normalization, combinations and weights25:03 The role of query understanding: feature groups35:11 Lesson 1 from Daniel: Simple models might be all you need36:30 Lesson 2: query features might be all you need38:30 Reasoning capabilities in search40:02 Question from Eric: how is this different from Learning To Rank?42:46 Carrying the past in Learning To Rank / any rank44:21 Demo!51:52 How to consume this in OpenSearch55:15 What's next58:44 Haystack US 2025YouTube: https://www.youtube.com/watch?v=quY769om1EY