2 ms·
Yep, every day. I do notice that it’s getting worse, though. Many websites’ feeds are just broken these days. They’ll randomly refresh after months of inactivit
by derefr 7y ago
Yep, every day. I do notice that it’s getting worse, though. Many websites’ feeds are just broken these days. They’ll randomly refresh after months of inactivity with dozens of new articles from the intervening period; or they’ll re-list a slew of previously listed entries; or it’ll be entirely random whether a new entry shows up when a new item is posted on the website, such that I can only take an entry as a sort of “unread badge reminder” for the website rather than taking each RSS entry in my reader as an individual article I should read.
I think some of this is the fault of bad RSS feeds being generated by bad CMS software that never gets its RSS path tested; but other parts of this are the fault of the RSS feed getting fronted by a CDN like Cloudflare, or the URL pulling the feed in through some sort of PuSH hub with non-compliant retrieval semantics, or etc.
It’s gotten me to feeling that I should be writing software to synthesize my own RSS feeds from these websites using scraping, with the RSS feed itself just being an edge-triggered “no longer fresh” trigger to get the site put back into the scrape queue. Most of the sites I subscribe to have very simple linear indices available somewhere anyway, so scraping that into a feed really shouldn’t be that hard. (I remember when Dapper was trying to do this, but Dapper was trying to solve the problem in full generality for even cases where the site generates “changes” without creating new permalinked entry URLs, which is hardly ever a problem any more.)