4 ms·

It's a funny quip but it's getting more attention than the important piece of news it highlights. Google is finally doing something about scraping sites doing b

It's a funny quip but it's getting more attention than the important piece of news it highlights. Google is finally doing something about scraping sites doing better in search results than original creators. Good. Many people don't write for money, to put ads on their website, or as part of some "content marketing" campaign. All they want is a little recognition. A boost in positioning on the SERP means we will be getting useful stuff at no cost. And there are genuine replies there. Ryan Jones[1] even got the scrapers to confess their sins[2]. [1] https://twitter.com/RyanJones/status/439123533349015553 https://twitter.com/RyanJones/status/439123533349015553 [2] https://www.google.com/search?q=%20%22istwfn%22+%22stole+this+word+from+noslang%22 https://www.google.com/search?q=%20%22istwfn%22+%22stole+thi...

psykovsky 13y ago
Yes, because only Google can earn advertising money by scrapping sites.
leephillips 13y ago
"Google is finally doing something about scraping" I hope this is genuine and not a disingenuous diversion on Google's part. The fact that the Huffington Post still ranks very high for trendy searches makes me wonder. As usual, follow the money: the scraping sites exist to make money, often through Google's advertising; Google gets a cut. The original content is often on sites with no advertising or real traffic, from which Google profits nothing. EDIT: To expand on this: Google-search for any hot topic in the news, say the name of some misbehaving pop star. See the HuffPo result near the top of the page. Look down to see several results from real newspapers. This is where the original content can be found. Most of these newspapers are about to die because they're not making any money. HuffPo investors are filthy rich because they're gaming the search engines to profit from copy-and-paste. ANOTHER EDIT: I apologize for my characterization of the Huffington Post. I was describing, accurately, the nature of that site as it was the last time I visited it some time before its purchase by AOL three years ago. The HuffPo I see today is utterly transformed. They use wire services, do plenty of their own reporting, and many of the links on the front page go directly to other news sites. They are no longer a copy-and-paste site.
- ezequiel-garzon 13y ago
  Is their copy and paste verbatim?
- perbu 13y ago
  That would be a pretty blatant copyright violation. Can you provide an example to substantiate the claim?
- acjohnson55 13y ago
  Huffington Post isn't a scraper site. Aside from the original content they produce, they republish blog posts with permission from the authors. If you have an example of Huffington Post literally cut-and-pasting content from someone without attribution, please share. I also assume that by "HuffPo investors" you mean AOL? Huffington Post is a fully owned subsidiary. (Disclosure: I consult for Huffington Post)
  - leephillips 13y ago
    You are right. Please see my second edit in my comment.
- untog 13y ago
  There's a big difference between writing an article based on another article you read, and web scrapers. Not that I particularly feel like defending the Huffington Post, but they're not a web scraper.
- grey-area 13y ago
  Google-search for any hot topic in the news, say the name of some misbehaving pop star. See the HuffPo result near the top of the page. Look down to see several results from real newspapers. Many newspapers get a lot of their content from syndication services like Reuters. You may be seeing similar content because lazy editorial assistants just copied out a reuters story verbatim, slapped a pic on it and put it up at multiple organisations, not because HuffPo is scraping other sites. Do you have an example of this sort of thing you can point to? It'd be interesting to trace the origin of the content.
- 13y ago
pinakothek 13y ago
Here's hoping they get it right, because the way it stands now, building a good aggregator is very tricky. The "this is why we can't have nice things" attitude is really putting a damper on valuable developments, and it's very close to stifling competition. Why is it anyone's problem that their algo is not smart enough to see potential value in new kinds of services? Even human vs. software curation should be irrelevant here.
loceng 13y ago
If this is the case, depending on how deep they follow this philosophy, it's important because the people or places that actually gather the content or lead to the content creation have invested and are investing more resources than those who simply scrape the data - and they the opportunity to recover costs, otherwise you damage that positive system's ability to continue on their successful path. That matters if you care how quickly we reach a better and better world for everyone. The source of where content is generated (in form of data or other) is also a strong signal of value, and that seems to be a leading metric of what Google wants to offer - the highest value search results.
debacle 13y ago
It is highlighting the more important aspect - Google is the largest scraping site in the world.
pointillistic 13y ago
Scrapping is only possible because the value of original content creation is 0% and the value of the toll collecting aggregation is 100% via ads. This is a structural problem in the universe Goggle created, not some anomaly that can be handled by ranking.
robomartin 13y ago
I've always wondered about news organizations. The vast majority of them have websites full of regurgitated content. They don't usually discover and report their own news, they feed off Associated Press, Reuters and often times individuals posting videos on youtube or articles in their blogs. It would be nice to see them not dominate news on the internet by simply engaging in re-writing what someone else originated. Is it content curation? Don't know. Everyone is reporting pretty much exactly the same things. They can't all be the original source. Who gets the juice?

psykovsky 13y ago

Yes, because only Google can earn advertising money by scrapping sites.

leephillips 13y ago

"Google is finally doing something about scraping" I hope this is genuine and not a disingenuous diversion on Google's part. The fact that the Huffington Post still ranks very high for trendy searches makes me wonder. As usual, follow the money: the scraping sites exist to make money, often through Google's advertising; Google gets a cut. The original content is often on sites with no advertising or real traffic, from which Google profits nothing. EDIT: To expand on this: Google-search for any hot topic in the news, say the name of some misbehaving pop star. See the HuffPo result near the top of the page. Look down to see several results from real newspapers. This is where the original content can be found. Most of these newspapers are about to die because they're not making any money. HuffPo investors are filthy rich because they're gaming the search engines to profit from copy-and-paste. ANOTHER EDIT: I apologize for my characterization of the Huffington Post. I was describing, accurately, the nature of that site as it was the last time I visited it some time before its purchase by AOL three years ago. The HuffPo I see today is utterly transformed. They use wire services, do plenty of their own reporting, and many of the links on the front page go directly to other news sites. They are no longer a copy-and-paste site.

ezequiel-garzon 13y ago
Is their copy and paste verbatim?
perbu 13y ago
That would be a pretty blatant copyright violation. Can you provide an example to substantiate the claim?
acjohnson55 13y ago
Huffington Post isn't a scraper site. Aside from the original content they produce, they republish blog posts with permission from the authors. If you have an example of Huffington Post literally cut-and-pasting content from someone without attribution, please share. I also assume that by "HuffPo investors" you mean AOL? Huffington Post is a fully owned subsidiary. (Disclosure: I consult for Huffington Post)
- leephillips 13y ago
  You are right. Please see my second edit in my comment.
untog 13y ago
There's a big difference between writing an article based on another article you read, and web scrapers. Not that I particularly feel like defending the Huffington Post, but they're not a web scraper.
grey-area 13y ago
Google-search for any hot topic in the news, say the name of some misbehaving pop star. See the HuffPo result near the top of the page. Look down to see several results from real newspapers. Many newspapers get a lot of their content from syndication services like Reuters. You may be seeing similar content because lazy editorial assistants just copied out a reuters story verbatim, slapped a pic on it and put it up at multiple organisations, not because HuffPo is scraping other sites. Do you have an example of this sort of thing you can point to? It'd be interesting to trace the origin of the content.
13y ago

pinakothek 13y ago

Here's hoping they get it right, because the way it stands now, building a good aggregator is very tricky. The "this is why we can't have nice things" attitude is really putting a damper on valuable developments, and it's very close to stifling competition. Why is it anyone's problem that their algo is not smart enough to see potential value in new kinds of services? Even human vs. software curation should be irrelevant here.

loceng 13y ago

If this is the case, depending on how deep they follow this philosophy, it's important because the people or places that actually gather the content or lead to the content creation have invested and are investing more resources than those who simply scrape the data - and they the opportunity to recover costs, otherwise you damage that positive system's ability to continue on their successful path. That matters if you care how quickly we reach a better and better world for everyone. The source of where content is generated (in form of data or other) is also a strong signal of value, and that seems to be a leading metric of what Google wants to offer - the highest value search results.

debacle 13y ago

It is highlighting the more important aspect - Google is the largest scraping site in the world.

pointillistic 13y ago

Scrapping is only possible because the value of original content creation is 0% and the value of the toll collecting aggregation is 100% via ads. This is a structural problem in the universe Goggle created, not some anomaly that can be handled by ranking.

robomartin 13y ago

I've always wondered about news organizations. The vast majority of them have websites full of regurgitated content. They don't usually discover and report their own news, they feed off Associated Press, Reuters and often times individuals posting videos on youtube or articles in their blogs. It would be nice to see them not dominate news on the internet by simply engaging in re-writing what someone else originated. Is it content curation? Don't know. Everyone is reporting pretty much exactly the same things. They can't all be the original source. Who gets the juice?