5 ms·

There definitely exist scrapers that work to get the full body. The reason for this is reading in a feed reader will never get your ad revenue or analytics, for

by bpye 8y ago

There definitely exist scrapers that work to get the full body. The reason for this is reading in a feed reader will never get your ad revenue or analytics, forcing your reader to the full page will.

AdmiralAsshat 8y ago
Oh I definitely understand why sites do it this way. I simply wanted clarification on whether it was always the intention to do it that way, or if the content sites took the standard and crippled it in order to bring more ad revenue.
- blihp 8y ago
  It could probably be argued that the standard is ambiguous as what content goes where is up to the creator of it. However, I think as a user it's fairly obvious that using it for clickbait and teaser headlines/articles rather than at least a complete thought is abusive. It's probably a non-trivial part of the reason its popularity with end users has waned in the much the same way spam polluted peoples email inboxes.
  - TeMPOraL 8y ago
    Yup. Standards from that age were written with friendly, cooperative actors in mind. The Internet is no longer a friendly place. Unfortunately, you can't just force people to not be greedy. (Clickbait itself is an example of that. You absolutely can get to the point in the title. But most media sources don't, because it's not their goal to inform you - their goal is to just exploit you for all they can.
    - matt4077 8y ago
      It's also unfriendly, and not very cooperative, to dismiss the stated wishes and motivations of content creators to save clicking on a link. Similarly unfriendly is assuming bad faith, i. e. "most media sources don't [seek to] inform you - their goal is to just exploit you". Yeah, yeah, I know. You'd gladly spoil the writers you read by deigning to visit, if only they didn't use javascript/have big images that load slowly/used your favorite font/...
      - Dylan16807 8y ago
        Nobody's wishes about link clicks should matter one bit except the user's.
scarface74 8y ago
99.999% of the time you’re right. But Daring Fireball offers the full article as part of the RSS feed and it has one sponsored RSS article at the beginning of the week and a thank you post in the RSS feed at the end of the week. He charges around $6500 per week the last time I checked. It’s amazing what you can do if you stay small and have “1000 true fans”.
burtonator 8y ago
Yes. The analytics is another key issue. AMP for example ads analytics.
- r3bl 8y ago
  In the server logs I see that RSS services tend to send the number of subscribers when they check for new content. It's kind of a useful data point, and I could imagine the controversy if RSS services made an attempt to share more.
aaaaaaaaaab 8y ago
I remember when I could sync my feed reader at home, then read the articles while commuting on public transport with no internet connection. Then sites started offering only the first paragraph in their feeds, so RSS became useless for this use case.
- akho 8y ago
  You could set up a “reader” to download full pages, using RSS as basically a notification service.
hiccuphippo 8y ago
Yahoo!Pipes was awesome at this. You could give it a feed, pull the page for each item and generate a new feed with the full content. There was also ways to specify what parts of the page to get by using xpath or regex. I miss it dearly.

by bpye 8y ago

There definitely exist scrapers that work to get the full body. The reason for this is reading in a feed reader will never get your ad revenue or analytics, forcing your reader to the full page will.

AdmiralAsshat 8y ago

Oh I definitely understand why sites do it this way. I simply wanted clarification on whether it was always the intention to do it that way, or if the content sites took the standard and crippled it in order to bring more ad revenue.

blihp 8y ago
It could probably be argued that the standard is ambiguous as what content goes where is up to the creator of it. However, I think as a user it's fairly obvious that using it for clickbait and teaser headlines/articles rather than at least a complete thought is abusive. It's probably a non-trivial part of the reason its popularity with end users has waned in the much the same way spam polluted peoples email inboxes.
- TeMPOraL 8y ago
  Yup. Standards from that age were written with friendly, cooperative actors in mind. The Internet is no longer a friendly place. Unfortunately, you can't just force people to not be greedy. (Clickbait itself is an example of that. You absolutely can get to the point in the title. But most media sources don't, because it's not their goal to inform you - their goal is to just exploit you for all they can.
  - matt4077 8y ago
    It's also unfriendly, and not very cooperative, to dismiss the stated wishes and motivations of content creators to save clicking on a link. Similarly unfriendly is assuming bad faith, i. e. "most media sources don't [seek to] inform you - their goal is to just exploit you". Yeah, yeah, I know. You'd gladly spoil the writers you read by deigning to visit, if only they didn't use javascript/have big images that load slowly/used your favorite font/...
    - Dylan16807 8y ago
      Nobody's wishes about link clicks should matter one bit except the user's.

scarface74 8y ago

99.999% of the time you’re right. But Daring Fireball offers the full article as part of the RSS feed and it has one sponsored RSS article at the beginning of the week and a thank you post in the RSS feed at the end of the week. He charges around $6500 per week the last time I checked. It’s amazing what you can do if you stay small and have “1000 true fans”.

burtonator 8y ago

Yes. The analytics is another key issue. AMP for example ads analytics.

r3bl 8y ago
In the server logs I see that RSS services tend to send the number of subscribers when they check for new content. It's kind of a useful data point, and I could imagine the controversy if RSS services made an attempt to share more.

aaaaaaaaaab 8y ago

I remember when I could sync my feed reader at home, then read the articles while commuting on public transport with no internet connection. Then sites started offering only the first paragraph in their feeds, so RSS became useless for this use case.

akho 8y ago
You could set up a “reader” to download full pages, using RSS as basically a notification service.

hiccuphippo 8y ago

Yahoo!Pipes was awesome at this. You could give it a feed, pull the page for each item and generate a new feed with the full content. There was also ways to specify what parts of the page to get by using xpath or regex. I miss it dearly.