How To Build a Hybrid AI System with Any-LLM (ft Nathan Brake)

Join the Tool Use Discord: https://discord.gg/PnEGyXpjaXAre you looking to optimize your AI applications with the best open source Large Language Models? This week, we explore open source AI, rapid model switching, and local self-hosting with Nathan Break, a Machine Learning Engineer at Mozilla AI and the creator of Any-LLM. Any-LLM is an open-source Python SDK that allows developers to seamlessly talk to any LLM provider through a single, unified interface.Nathan demonstrates how developers can instantly swap between cloud-hosted AI providers and local models like Llamafile by changing just a single line of code. We dive into the growing capabilities of smaller, cost-effective models, how to reduce your API expenses, and the security benefits of running local AI on your own hardware. Nathan also introduces the Any-LLM Platform, a secure vault for managing API keys that tracks your token usage and enforces budget limits without ever accessing your prompt data. Finally, we check out Porch Songs, Nathan's open-source application that leverages LLMs to rewrite the lyrics of popular songs while maintaining their original rhyme schemes and chord charts.Check out the projects mentioned in this episode:Any-LLM GitHub: https://github.com/mozilla-ai/any-llmAny-LLM Platform: https://any-llm.aiMozilla AI Blog Post: https://blog.mozilla.ai/introducing-any-llm-a-unified-api-to-access-any-llm-providerPorch Songs GitHub: https://github.com/njbrake/porchsongsConnect with ushttps://x.com/ToolUsePodcasthttps://x.com/MikeBirdTech00:00:00 - Intro 00:01:25 - What is Any-LLM? Unified API for Open Source AI 00:05:32 - Benefits of Rapid Model Switching & Open Weight Models 00:11:06 - Any-LLM Python SDK Demo: Mistral Image Recognition 00:19:51 - Best Local AI Models: Llamafile, LM Studio & Local LLMs 00:24:38 - Any-LLM Platform Vault & Porch Songs App Demo 00:34:45 - AI Coding Agents, Claude Code & LLM EvaluationsSubscribe for more insights on AI tools, productivity, and open source development.Tool Use is a weekly conversation with the top AI experts.