4 ms·

I was one of the inventors of RSS. I was on the RSS 1.0 RSS-DEV Working Group. I'm also the creator of Datastreamer (http://www.datastreamer.io/ http://www.dat

I was one of the inventors of RSS. I was on the RSS 1.0 RSS-DEV Working Group. I'm also the creator of Datastreamer (http://www.datastreamer.io/ http://www.datastreamer.io/) which you could think of as a massive petabyte-scale RSS aggregator. Seeing that I helped invent RSS, and have probably parsed more RSS than anyone (hundreds of petabytes), I'm going to call it and say that RSS is dead. Datastreamer used to index RSS and we've long since deprecated it as a secondary data source. It's really not our primary content source. We prefer to index raw HTML. HTML can do exactly what you want and you don't need RSS thanks to microformats and microdata. Here's the main problem: 1. Publishers don't care about your use case. They WANT you go to back to the site and click their ads. 2. Publishers want to hoard their content and sell it if possible. This is the exact antithesis of RSS content distribution (that the Internet should be open). What's the way forward? 1. HTML microdata parsing RSS but implemented in microdata. Publishers actually WANT their data to be distributed this way as they're hoping their content is distributed over Twitter and Facebook. The only downside here is that full content is often not used. 2. Metadata sharing. SELF metadata sharing where communities of users markup and share their own content. I'm going to be working on some ideas around this in Polar (my project around distributed content sharing). https://getpolarized.io/ https://getpolarized.io/ I think you're going to start seeing some of the ideas around the next gen of content syndication in a few months. Right now I'm trying to build a large network of users by building an amazing tool to handle your content. Once the user base is there then I can enable content collaboration where users are sharing content to each other. This will also include some RSS-like features at some point but indexed around microdata.

panic 8y ago
Every blog I've tried to follow for years has had an RSS feed. Are microformats really as reliable? Like, picking a blog at random from my feed, I don't see any microdata on the page.
- burtonator 8y ago
  The biggest challenge with this strategy is discovering which links to index. The microdata is much more data. You can see this metadata when you post to Twitter or Facebook and it shows a preview. We have the internal stats to show that this is MUCH more reliable than RSS but I don't really have it in a format that I can readily share. Discovering the links can be done by just spidering every post on the front page. The biggest challenge is the timestamp. We handle that by just keeping a fingerprint of all the URLs and we only index URLs that have either never been seen before or have timestamps. I honestly with timestamps were required by Facebook and Twitter or at least given special treatment. The special treatment strategy by Google and others has REALLY done a great job at upgrading the quality of the Internet by not breaking it. Basically saying that your site will get some tangible financial benefit if you run X spec. So if you enable IPv6 or HTTPS you get an SEO boost. Companies actually pay attention to this and implement these changes.
  - girzel 8y ago
    Out of everything you've said, my only takeaway is that og:image is useful for Facebook and Twitter. I'll happily admit that's true, and I use it on my own sites. Otherwise... What does "MUCH more reliable" even mean? The rest of it seems to be you trying to catch up to RSS...
- jayalpha 8y ago
  "Every blog I've tried to follow for years has had an RSS feed. " I follow many blogs (Rule of thumb: the fewer posts per time, the higher the content of the blog likely is) and you would be surprised how many feeds fail over time. Sometimes new domain, sometimes new URL (e.g. blog.domain.com to domain.com/blog very common). And often an RSS feed contains errors that never go away.
philbo 8y ago
I'm not the inventor of anything but I'm going to call it and say that microformats and microdata are dead.
- johnchristopher 8y ago
  Doesn't OG fulfill microformats purposes ?
Ajedi32 8y ago
> Publishers don't care about your use case. They WANT you go to back to the site and click their ads. Isn't that actually an _incentive_ for them to offer RSS feeds? After all, I'm probably not going to go to their site if my RSS reader never notifies me that they posted a new article.
- jolmg 8y ago
  Unless you have tons of blogs you want to follow or a really slow internet connection, it seems simpler to just take a minute and check each blog site routinely. When I first tried out RSS in the PSP days, I was hoping that could serve as a consistent interface for news. It was pretty disappointing to find out that no one wanted their full article on RSS. It was always just a snippet and a link to the full article on their website. This is what I believe burtonator is referring to in the part you quoted.
  - kgwxd 8y ago
    Checking even a few sites routinely for new content is a huge waste of life. You have to figure out if there is new content, then you have to remember where you left off and, assuming a site even makes that easy (most don't), while figuring it out, you have to avoid falling into a bottomless pit of click bait. I have 12 feeds in my reader; HN, a handful of YouTube channels and a handful of personal blogs. Some have posts every minute, some have posts every day, some have posts months apart. Of all these feeds, only 1 contains the actual content (Planet GNU), everything else is a link and maybe a small summary, and I'm perfectly fine with that, I just want the uniform process of notifications that RSS provides because the alternatives are all horrible.
  - Ajedi32 8y ago
    > Unless you have tons of blogs you want to follow You mean you don't? I've got maybe about a hundred at this point. Even if it were only a half-dozen, that'd still be _way_ more than I'd want to check manually every day.
  - thirdsun 8y ago
    > Unless you have tons of blogs you want to follow or a really slow internet connection, it seems simpler to just take a minute and check each blog site routinely Obviously I can only speak for myself but there's no way I'm going to do that. My feed aggregator (Feedbin) serves as my daily inbox of web content. From there and since many feeds are truncated I'll open specific articles that seem interesting to me. Truncation is fine, I understand that commercial sites need a way to get visitors. However not providing an RSS feed at all is a good way to miss out on my page view.

panic 8y ago

Every blog I've tried to follow for years has had an RSS feed. Are microformats really as reliable? Like, picking a blog at random from my feed, I don't see any microdata on the page.

burtonator 8y ago
The biggest challenge with this strategy is discovering which links to index. The microdata is much more data. You can see this metadata when you post to Twitter or Facebook and it shows a preview. We have the internal stats to show that this is MUCH more reliable than RSS but I don't really have it in a format that I can readily share. Discovering the links can be done by just spidering every post on the front page. The biggest challenge is the timestamp. We handle that by just keeping a fingerprint of all the URLs and we only index URLs that have either never been seen before or have timestamps. I honestly with timestamps were required by Facebook and Twitter or at least given special treatment. The special treatment strategy by Google and others has REALLY done a great job at upgrading the quality of the Internet by not breaking it. Basically saying that your site will get some tangible financial benefit if you run X spec. So if you enable IPv6 or HTTPS you get an SEO boost. Companies actually pay attention to this and implement these changes.
- girzel 8y ago
  Out of everything you've said, my only takeaway is that og:image is useful for Facebook and Twitter. I'll happily admit that's true, and I use it on my own sites. Otherwise... What does "MUCH more reliable" even mean? The rest of it seems to be you trying to catch up to RSS...
jayalpha 8y ago
"Every blog I've tried to follow for years has had an RSS feed. " I follow many blogs (Rule of thumb: the fewer posts per time, the higher the content of the blog likely is) and you would be surprised how many feeds fail over time. Sometimes new domain, sometimes new URL (e.g. blog.domain.com to domain.com/blog very common). And often an RSS feed contains errors that never go away.

philbo 8y ago

I'm not the inventor of anything but I'm going to call it and say that microformats and microdata are dead.

johnchristopher 8y ago
Doesn't OG fulfill microformats purposes ?

Ajedi32 8y ago

> Publishers don't care about your use case. They WANT you go to back to the site and click their ads. Isn't that actually an _incentive_ for them to offer RSS feeds? After all, I'm probably not going to go to their site if my RSS reader never notifies me that they posted a new article.

jolmg 8y ago
Unless you have tons of blogs you want to follow or a really slow internet connection, it seems simpler to just take a minute and check each blog site routinely. When I first tried out RSS in the PSP days, I was hoping that could serve as a consistent interface for news. It was pretty disappointing to find out that no one wanted their full article on RSS. It was always just a snippet and a link to the full article on their website. This is what I believe burtonator is referring to in the part you quoted.
- kgwxd 8y ago
  Checking even a few sites routinely for new content is a huge waste of life. You have to figure out if there is new content, then you have to remember where you left off and, assuming a site even makes that easy (most don't), while figuring it out, you have to avoid falling into a bottomless pit of click bait. I have 12 feeds in my reader; HN, a handful of YouTube channels and a handful of personal blogs. Some have posts every minute, some have posts every day, some have posts months apart. Of all these feeds, only 1 contains the actual content (Planet GNU), everything else is a link and maybe a small summary, and I'm perfectly fine with that, I just want the uniform process of notifications that RSS provides because the alternatives are all horrible.
- Ajedi32 8y ago
  > Unless you have tons of blogs you want to follow You mean you don't? I've got maybe about a hundred at this point. Even if it were only a half-dozen, that'd still be _way_ more than I'd want to check manually every day.
- thirdsun 8y ago
  > Unless you have tons of blogs you want to follow or a really slow internet connection, it seems simpler to just take a minute and check each blog site routinely Obviously I can only speak for myself but there's no way I'm going to do that. My feed aggregator (Feedbin) serves as my daily inbox of web content. From there and since many feeds are truncated I'll open specific articles that seem interesting to me. Truncation is fine, I understand that commercial sites need a way to get visitors. However not providing an RSS feed at all is a good way to miss out on my page view.