3 ms·

> to waste data Useless traffic. RSS has a structural problem: you download a "window" of data (the feeds between some timestamp in day A and some other in da

by mdp2021 2y ago

> to waste data Useless traffic. RSS has a structural problem: you download a "window" of data (the feeds between some timestamp in day A and some other in day B), which may or may not contain new data. You will easily lose some and receive a lot of duplicates. It would have been better to be able to first check if new data is available. (And even better to only download the new ones, and all of them - "everything after YY-MM-DD hh:mm:ss")

paulryanrogers 2y ago
Such as via HEAD and Etag?
- masklinn 2y ago
  HEAD is counter-productive Just GET with if-none-match and if-modified-since (based on the etag and last-modified you got in the previous response), and the server will return a 304 not modified with no content if nothing has changed, and the content otherwise. With a HEAD you'd get the same result except now you'd need to ignore the cache headers from the HEAD response in order to fetch the content in a second request.
  - OutOfHere 2y ago
    You speak of this confidently, but there are sites that return a not modified response when in truth they have modified the feed. It doesn't happen often, but I have seen it happen with more than one site. It is why I take the not modified response with a grain of salt.
graemep 2y ago
> It would have been better to be able to first check if new data is available. Or, as the article says, to actually check when you are able. All she is asking for is that readers make conditional requests, at reasonable intervals, and respect 429s.
deleted 2y ago
[deleted]
masklinn 2y ago
> It would have been better to be able to first check if new data is available. That... is what conditional requests do...
- mdp2021 2y ago
  Which means: do not place extra mechanisms in the RSS when you can do the same with HTTP. But then there is the problem of RSS clients that may not work properly (may not use that trick), without the user knowing it, and that of RSS servers that do not work correctly with "If-Modified-Since" (as noted in a nearby post). Edit: but if the burden were placed on RSS instead, we could have had the trick of "I already have ...#10050, #10051 and #10052: just send me from #10053 on" - the feeds XML that is updated recently will contain more items than just the new ones. Similarly for the gaps: "the XML would contain from #10050 on, but there had been a surge of publications and now I am missing #10045 to #10049..."
  - notpushkin 2y ago
    > but if the burden were placed on RSS instead That would be nice to have, but it's less realistic. Some clients already speak proper HTTP, and others can too, with little modification. For modified RSS like this, you have to make a standard first, then push both servers and clients to use it.
    - masklinn 2y ago
      It also requires a "smart" server, whereas you can serve your atom or rss feed statically with a standard HTTP server handling caching and conditional requests essentially free.
      - notpushkin 2y ago
        Yeah, although dumb servers could just return last N posts as usual. It's one more case to handle on the client, though. (The alternative is breaking compatibility with older feeds, which is way worse IMO.)

paulryanrogers 2y ago

Such as via HEAD and Etag?

masklinn 2y ago
HEAD is counter-productive Just GET with if-none-match and if-modified-since (based on the etag and last-modified you got in the previous response), and the server will return a 304 not modified with no content if nothing has changed, and the content otherwise. With a HEAD you'd get the same result except now you'd need to ignore the cache headers from the HEAD response in order to fetch the content in a second request.
- OutOfHere 2y ago
  You speak of this confidently, but there are sites that return a not modified response when in truth they have modified the feed. It doesn't happen often, but I have seen it happen with more than one site. It is why I take the not modified response with a grain of salt.

graemep 2y ago

> It would have been better to be able to first check if new data is available. Or, as the article says, to actually check when you are able. All she is asking for is that readers make conditional requests, at reasonable intervals, and respect 429s.

deleted 2y ago

[deleted]

masklinn 2y ago

> It would have been better to be able to first check if new data is available. That... is what conditional requests do...

mdp2021 2y ago
Which means: do not place extra mechanisms in the RSS when you can do the same with HTTP. But then there is the problem of RSS clients that may not work properly (may not use that trick), without the user knowing it, and that of RSS servers that do not work correctly with "If-Modified-Since" (as noted in a nearby post). Edit: but if the burden were placed on RSS instead, we could have had the trick of "I already have ...#10050, #10051 and #10052: just send me from #10053 on" - the feeds XML that is updated recently will contain more items than just the new ones. Similarly for the gaps: "the XML would contain from #10050 on, but there had been a surge of publications and now I am missing #10045 to #10049..."
- notpushkin 2y ago
  > but if the burden were placed on RSS instead That would be nice to have, but it's less realistic. Some clients already speak proper HTTP, and others can too, with little modification. For modified RSS like this, you have to make a standard first, then push both servers and clients to use it.
  - masklinn 2y ago
    It also requires a "smart" server, whereas you can serve your atom or rss feed statically with a standard HTTP server handling caching and conditional requests essentially free.
    - notpushkin 2y ago
      Yeah, although dumb servers could just return last N posts as usual. It's one more case to handle on the client, though. (The alternative is breaking compatibility with older feeds, which is way worse IMO.)