Episode 271: Foot Terminal

Podcasting 2.0 Episode 271 - "Foot Terminal" Adam & Dave have their SCBA's on and go deep into index refactoring and ad loads! <b>ShowNotes</b> ------------------------------------------------------------------------------------------------------------------------------------- 00 - 🔥 STEP TWO OF THE SCARY STUFF — THE ENDPOINTS FLIP TO THE IDENTITY LAYER <b>Last week the table went in. This week the API started using it — one endpoint at a time, four days running.</b> <b>Sat 12 Sep:</b> "#api I'm running a script to sync up podcasts.id and newsfeeds.id so that when a new feed is born they are in sync. During the cutover to podcasts.id as the canonical form, there were <b>about 11k feeds that drifted away from being in sync</b> with their podcast because of a gap in the auto_increment values. This isn't strictly necessary, it just will make human review a little less cumbersome." <b>Mon night 14 Sep:</b> "#api Making some final refinements to the new api endpoints before deploying. These will be the first ones using the new podcast identity layer. 😰" <b>Tue 15 Sep — the first flip:</b> "#api The <b>podcasts/byfeedurl</b> endpoint is deployed with the first changes to the new podcast identity layer. <b>We will do these one at a time, watching for problems.</b> You will see nothing different in the response shape under normal circumstances. The response 'feed' object's <b>`id` property now means the podcast ID</b>. The <b>`feedId` property now means the underlying feeds table row</b> that podcast is treating as canonical." @dave — byfeedurl flips (15 Sep) "It's not ideal to have the podcast id be represented as feeds.id in the new responses, but <b>anything else would have broken existing apps</b>. And, the meaning is still faithful to what it was before. That was always meant to be the ID of the podcasts. We just didn't have a way to logically express that." <b>Same day:</b> "#api The new <b>podcasts/byfeedid</b> endpoint is deployed. That's all for today. Will watch the logs until tomorrow." · "#api First run of the <b>itunes ID 404 reconciler</b> under the new podcast identity regime. 🤞" <b>Thu 17 Sep:</b> "#api The <b>podcasts/byguid</b> and <b>podcasts/batch/byguid</b> endpoints are now flipped to the new db schema." · <b>Fri 18 Sep, this morning:</b> "#api The new <b>podcasts/byitunesid</b> endpoint is live." @dave — byitunesid live (18 Sep) <b>📝 The app developers noticed.</b> Mitch (Podverse): "listening to last week's P2.0... the <b>nextgen Podverse schema (and ogen) relies heavily on the Podcast Index IDs as basically the authority on feed uniqueness</b>. It sounds like we may need/want to do a substantial rewrite to account for how Podcast Index API nextgen will handle ids?… after it is settled, <b>I'd appreciate any documentation</b>." @mitch — the Podverse question (15 Sep) Dave: "If I do my job right, <b>you will see nothing different and will not need to do anything</b>. I made sure the starting point for every canonical podcast index 'podcast id' is the current live feed id. If you see some sort of drift or problem it would be a bug I need to fix… <b>the existing podcast ID's have always identified feeds, but now they will identify podcasts. The ID's themselves will not and have not changed.</b> I'll definitely be producing docs to flesh this out further though." <b>😄 Then the drawing:</b> "It's actually kind of confusing to explain. It's easier if I draw it out on paper. It sort of needs to be visualized." → "#api These might be very hard to follow. There are <b>two 'entrances' into the redirection chain</b> and the Index follows whichever one you come in on." → "<b>Paper and pen was faster than some AI agent.</b>" @dave — the two entrances, drawn by hand (15 Sep) The rule as Dave wrote it: "If you come in on the <b>'byid'</b> path, you start at the podcast layer and follow the redirection to the correct feed and serve it. If you come in on the <b>'byurl'</b> path, you start at the feeds layer and follow up the chain to the podcasts layer and then follow the