3 ms·

Here is what I found looking at my server logs a few months ago: Feedly: 7min Feedbin: 15min Bloglovin: 30min Dreamwidth Studios: 30min

by jefftk 5y ago

Here is what I found looking at my server logs a few months ago: Feedly: 7min Feedbin: 15min Bloglovin: 30min Dreamwidth Studios: 30min Feed Wrangler: 30min NewsBlur: 30min BazQux: 40min inoreader.com: 1hr theoldreader.com: 2hr pine.blog: 24hr https://www.jefftk.com/p/looking-at-rss-user-agents https://www.jefftk.com/p/looking-at-rss-user-agents

blitzar 5y ago
Also worth mentioning the obvious, feedly etc is updating all the users on their platrom with a single call to your site (every 7 minutes in your case). No doubt they scale the update frequency by how popular it is on their platform / frequently updated / however often the ai decides it is needed.
- onli 5y ago
  Should also depend on whether a push extension is used, then the reader can fetch the feed less often. The feed of that blog seems to be a plain RSS feed without that.
  - vidarh 5y ago
    Everyone used to "ping" sites whose only purpose was to provide feeds of when blogs changed, so all the aggregators would use that as a starting point anyway. But really it isn't that necessary - there are few feeds where getting things to your subscribers that urgently matters enough that you necessarily want them to hammer your site the moment you publish.
  - jefftk 5y ago
    I'm not aware of any push functionality in RSS -- something needs to poll the feed, no? Or is there an RSS extension that uses long polling or something?
    - detaro 5y ago
      PubSubHubbub/WebSub (the latter is effectively the slightly modernized W3C standardization of the former) is the standard solution for that. (Feed declares where to register for updates (hubs), readers register their callback URLs with hubs, publishers tell hubs when something changed, hubs run through the registered callbacks and ping them with the update. Thus needs a server-component for the reader side.)
- vidarh 5y ago
  Yeah, I ran a niche aggregator for a few years and we polled ca. 3m feeds a day. Every time a feed had an update since the last time we'd polled it we'd subtract a percentage of the polling interval with a floor of about 5m. Every time there wasn't an update we'd reduce the rate until a ceiling of about 24h. Very simple and no doubt you could do better but better wasn't necessary.

by jefftk 5y ago

blitzar 5y ago

Also worth mentioning the obvious, feedly etc is updating all the users on their platrom with a single call to your site (every 7 minutes in your case). No doubt they scale the update frequency by how popular it is on their platform / frequently updated / however often the ai decides it is needed.

onli 5y ago
Should also depend on whether a push extension is used, then the reader can fetch the feed less often. The feed of that blog seems to be a plain RSS feed without that.
- vidarh 5y ago
  Everyone used to "ping" sites whose only purpose was to provide feeds of when blogs changed, so all the aggregators would use that as a starting point anyway. But really it isn't that necessary - there are few feeds where getting things to your subscribers that urgently matters enough that you necessarily want them to hammer your site the moment you publish.
- jefftk 5y ago
  I'm not aware of any push functionality in RSS -- something needs to poll the feed, no? Or is there an RSS extension that uses long polling or something?
  - detaro 5y ago
    PubSubHubbub/WebSub (the latter is effectively the slightly modernized W3C standardization of the former) is the standard solution for that. (Feed declares where to register for updates (hubs), readers register their callback URLs with hubs, publishers tell hubs when something changed, hubs run through the registered callbacks and ping them with the update. Thus needs a server-component for the reader side.)
vidarh 5y ago
Yeah, I ran a niche aggregator for a few years and we polled ca. 3m feeds a day. Every time a feed had an update since the last time we'd polled it we'd subtract a percentage of the polling interval with a floor of about 5m. Every time there wasn't an update we'd reduce the rate until a ceiling of about 24h. Very simple and no doubt you could do better but better wasn't necessary.