3 ms·

A publisher does not want countless browsers scraping arbitrary web pages just to see if they’ve changed when they can instead offer a single lightweight end po

A publisher does not want countless browsers scraping arbitrary web pages just to see if they’ve changed when they can instead offer a single lightweight end point specifically for content that is intended to be updated. If browsers started doing this scraping I can only imagine the arms race. I can only see this happening as a service. A company crawls the web—probably a search company. Their LLM classifies changes. Their users can subscribe to individual sites or pages. Even then there are downsides to publishers including loss of some tracking information and users spending less time on their sites being subjected to advertisements.

bluebarbet 2y ago
This was part of the technical justification for RSS: concentrate all the redundant page hits in once place. But another reason was that parsing an article list from messy tables-based HTML was harder than it is today with HTML5. There's even a feed attribute `h-feed` available today, which effectively turns list pages into feeds. Nobody uses it. The problem with the "single lightweight endpoint" is that it has to be maintained. RSS is literally a whole shadow site, with finicky XML validation to worry about on top. I've worked on multiple projects where the RSS endpoint was broken much of the time. It's both fragile and not very visible. A solution involving client parsing of semantic markup at least has the benefit of requiring zero maintenance.
- rglullis 2y ago
  > There's even a feed attribute `h-feed` available today, which effectively turns list pages into feeds. Are you referring to the microformats tags? They were never standard, and they were never meant to "turn pages into things". Microformats, while nice and useful, were never meant to be more as compilation of certain adopted practices by the indieweb crowd.
  - bluebarbet 2y ago
    OK I won't dispute that. The problem is that they would function nicely as a pragmatic solution to the underlying problem. Maintaining a few HTML attributes on an existing webpage is significantly easier than maintaining an RSS XML endpoint.
    - rglullis 2y ago
      RSS 1.0 was created in 2000. We are talking about a time where the browser that dominated the market was Internet Explorer 5, where virtually no website would render on strict standards mode, and MS refused to introduce any significant changes if it meant breaking its compatibility mode. People realized that that asking the whole web to adopt XHTML fully would never happen, so a new XML-based format hat was separate from the webpages was needed. (Also, a bit of a side rant: now that I am working on ActivityPub stuff I'm finding less and less sympathy for those that jump into code and push for "pragmatic" solutions. The ActivityPub spec is not perfect, but it's incredible how almost everyone implementing ActivityPub ignores JSON-LD and RDF and just want to wing with the JSON messages. It gets to the point where perfectly valid JSON-LD will get refused to anything that is not called Mastodon, because everyone else is just parsing the JSON they receive and completely ignoring @context directives)
      - bluebarbet 2y ago
        Yep I know the history, I was building sites at the time. And my own unfashionable take is that "pragmatic" HTML5 was a cop-out, it was still early days for the web and XHTML would have been worth the initial breakages. Interesting rant about ActivityPub, you have my sympathy.
      - account42 2y ago
        > Also, a bit of a side rant: now that I am working on ActivityPub stuff I'm finding less and less sympathy for those that jump into code and push for "pragmatic" solutions. The ActivityPub spec is not perfect, but it's incredible how almost everyone implementing ActivityPub ignores JSON-LD and RDF and just want to wing with the JSON messages. It gets to the point where perfectly valid JSON-LD will get refused to anything that is not called Mastodon, because everyone else is just parsing the JSON they receive and completely ignoring @context directives Maybe making everyone who wants to interact with the fediverse deal with all that complexity isn't such a great idea after all. What we really should have gotten was RSS++ with optional push notifications for new content instead of this mess. But it wouldn't be the web without reinventing the wheel for every new "standard" I guess.
- account42 2y ago
  > A solution involving client parsing of semantic markup at least has the benefit of requiring zero maintenance. This is a pipe dream. In reality your HTML markup needs just as much maintenance if not more - at least the separate feed is unbothered by design changes to the HTML site.

bluebarbet 2y ago

This was part of the technical justification for RSS: concentrate all the redundant page hits in once place. But another reason was that parsing an article list from messy tables-based HTML was harder than it is today with HTML5. There's even a feed attribute `h-feed` available today, which effectively turns list pages into feeds. Nobody uses it. The problem with the "single lightweight endpoint" is that it has to be maintained. RSS is literally a whole shadow site, with finicky XML validation to worry about on top. I've worked on multiple projects where the RSS endpoint was broken much of the time. It's both fragile and not very visible. A solution involving client parsing of semantic markup at least has the benefit of requiring zero maintenance.

rglullis 2y ago
> There's even a feed attribute `h-feed` available today, which effectively turns list pages into feeds. Are you referring to the microformats tags? They were never standard, and they were never meant to "turn pages into things". Microformats, while nice and useful, were never meant to be more as compilation of certain adopted practices by the indieweb crowd.
- bluebarbet 2y ago
  OK I won't dispute that. The problem is that they would function nicely as a pragmatic solution to the underlying problem. Maintaining a few HTML attributes on an existing webpage is significantly easier than maintaining an RSS XML endpoint.
  - rglullis 2y ago
    RSS 1.0 was created in 2000. We are talking about a time where the browser that dominated the market was Internet Explorer 5, where virtually no website would render on strict standards mode, and MS refused to introduce any significant changes if it meant breaking its compatibility mode. People realized that that asking the whole web to adopt XHTML fully would never happen, so a new XML-based format hat was separate from the webpages was needed. (Also, a bit of a side rant: now that I am working on ActivityPub stuff I'm finding less and less sympathy for those that jump into code and push for "pragmatic" solutions. The ActivityPub spec is not perfect, but it's incredible how almost everyone implementing ActivityPub ignores JSON-LD and RDF and just want to wing with the JSON messages. It gets to the point where perfectly valid JSON-LD will get refused to anything that is not called Mastodon, because everyone else is just parsing the JSON they receive and completely ignoring @context directives)
    - bluebarbet 2y ago
      Yep I know the history, I was building sites at the time. And my own unfashionable take is that "pragmatic" HTML5 was a cop-out, it was still early days for the web and XHTML would have been worth the initial breakages. Interesting rant about ActivityPub, you have my sympathy.
    - account42 2y ago
      > Also, a bit of a side rant: now that I am working on ActivityPub stuff I'm finding less and less sympathy for those that jump into code and push for "pragmatic" solutions. The ActivityPub spec is not perfect, but it's incredible how almost everyone implementing ActivityPub ignores JSON-LD and RDF and just want to wing with the JSON messages. It gets to the point where perfectly valid JSON-LD will get refused to anything that is not called Mastodon, because everyone else is just parsing the JSON they receive and completely ignoring @context directives Maybe making everyone who wants to interact with the fediverse deal with all that complexity isn't such a great idea after all. What we really should have gotten was RSS++ with optional push notifications for new content instead of this mess. But it wouldn't be the web without reinventing the wheel for every new "standard" I guess.
account42 2y ago
> A solution involving client parsing of semantic markup at least has the benefit of requiring zero maintenance. This is a pipe dream. In reality your HTML markup needs just as much maintenance if not more - at least the separate feed is unbothered by design changes to the HTML site.