3 ms·
I checked what my feed reader does. It seems that it typically gets 304s (I send both If-None-Match and If-Modified-Since) but still about half of the requests
by kevincox 2y ago
I checked what my feed reader does. It seems that it typically gets 304s (I send both If-None-Match and If-Modified-Since) but still about half of the requests get 429s.
So the request pattern basically looks like:
1. Get 304.
2. Wait 5min.
3. Get 429.
4. Error backoff evenly distributed between 30-60min
5. Go back to step 1.
So I guess 5min is "too fast" even for conditional requests. However the reason my reader picks 5min is:
1. There is no caching headers to suggest the author's preference.
2. Conditional requests are supported.
3. The feed is fast.
4. The feed is popular.
5. The feed is active (posts every week or so)
Sure, for her feed it could probably be lower but how is a robot supposed to know that? I would highly recommend setting a cache header. That gives an automatic signal of when the last fetch should be considered stale. I'll admit that many feed readers just use a fixed schedule, but many will still use a library that accidentally caches requests (maybe the browser extension readers that she was complaining about would?). For my feed reader we won't poll more often than your cache header (Cache-Control or Expires will both work) unless it is more than 24h in which case we will poll no more frequently than daily. If you just post blog posts about it it is never going to change, it is better to start pushing for broader support of an actual protocol that could be implemented.
Another great option is supporting WebSub. It is easy to get set up with a public hub and then readers that support it will poll very rarely. (Mine will poll weekly.)