4 ms·

On one hand, if there is a new media outlet via new distribution model that's winning the market, fair. So if an online-only news source is taking away traffic

On one hand, if there is a new media outlet via new distribution model that's winning the market, fair. So if an online-only news source is taking away traffic and money away from physical newspapers, there's part of me that'll feel sad, but whatcha gonna do. On the other hand though, it's a completely different model in my mind to scrape/link/copy/extract somebody else's hard work and news and editorials; and then become the majority/sole recipient of monetization (direct or via ads or other means). It's not good for society, it's not good capitalism either as it'll simply dry out the sources and then nobody can play and have fun anymore.

nobodyandproud 5y ago
Captured my exact sentiments. How many US tech websites were involved in lambasting decisions by France and Australia to force Google to play nice?
Swizec 5y ago
Fewer than 50% of Google searches result in a click these days. Largely thanks to the helpful (to the user) excerpts they provide ... and (I think) the trashy SEO-focused barely readable writing that you often find upon clicking through. https://sparktoro.com/blog/less-than-half-of-google-searches-now-result-in-a-click/ https://sparktoro.com/blog/less-than-half-of-google-searches...
shadowgovt 5y ago
News has always enjoyed / suffered a tricky carve-out in our understanding of the interaction between the first amendment and copyright because (by its nature) its value to the public is in dissemination. So while it's generally not okay to wholesale-copy a news article, it's always been okay to quote it (and, in the concrete and particular case of Google and Facebook, it's also always been okay to point people to it). So the disruption of the revenue model (surprise; turns out people were only getting the ads as a consequence of buying the paper, they weren't why they bought the paper) is particularly tough because the core activity that is disrupting the model (making news universally accessible and useful) is by itself a public good. (It's worth noting that the newspapers in this story have to go through the anti-trust door on this topic specifically because [I assume] there isn't any meat on the bones of claiming ownership of the news Facebook / Google are aggregating.)
13415 5y ago
Potentially stupid questions: Aren't there efficient techniques to prevent web scraping? And if Google really scrapes web content from them without their permission, wouldn't that be commercial copyright infringement which falls under criminal law? That's the thing I don't understand about the current situation.
- claudiulodro 5y ago
  > Aren't there efficient techniques to prevent web scraping? Not especially, since internet browsers make GET requests the same as scrapers. There are techniques around blocking user agents, rate limiting, etc. but that's an arms race against the scrapers. The tricky part about blocking Google is that if you do, your site won't show up in Google search results. :)
  - shaftway 5y ago
    Google in particular always uses an identifiable agent. So you could serve one set of content to Google and a separate set to actual visitors, preventing Google from scraping the content. They clearly want the benefit of being scraped (the traffic) without the cost (the loss of traffic from people who don't click past the headline to the article).
- NikolaNovak 5y ago
  1. Are there techniques to prevent web scraping? Yes. Are they "Efficient", and even more importantly, effective? Debatable 2. However, more to the point, what's the end game there? If there are 10,000 outlets out there; and arbitrary number, say 100 of them block scrapes; now the other 9,900 will show up in Google/Facebook search and shares, and those who implemented scraping protection will quietly fade into non-existence. It's a damned-if-you-do:damned-if-you-don't choice of modern media consumption model. I assume that most outlets have decided that the exposure, trickle of users and monetization they DO get is superior to the influx of users drying out. It's consequence of the oft-quoted platitude that these days, for most people Google and Facebook ARE "The Internet" :-/
cblconfederate 5y ago
The sad thing is that newspapers allow that themselves. They can block out google and FB but they don't want to. It's a similar tragic fate to that of a drug addict. However if content-makers want to overcome this they need to cut off big tech access to their content, at least temporarily. The frontpage of HN is mostly news articles and some blogs. They get a lot of exposure, a lot of readership. Yet somehow google + FB end up making the money.
- colinmhayes 5y ago
  Facebook and Google control demand. If a newspaper doesn't let them post their content the aggregators will just grab an article that is nominally the same from a different publisher that does provide them access.
  - _aavaa_ 5y ago
    If their content is so interchangeable, why should they all stay in business?
    - colinmhayes 5y ago
      Someone has to actually write the content before everyone else copies it.
      - AniseAbyss 5y ago
        Ultimately Google will just give some desperate Afgan kid a 100 dollar bill and a smartphone. That kid will be killed a few weeks later and replaced by a new one. The new journalists.
      - PeterisP 5y ago
        Copyright is a thing. When many newspapers and news sites have the exact same content, they're generally licensing it from a large news agency which actually wrote the story, it's not someone just illegally ripping off their paywalled content. On the other hand, if multiple sites can each write equivalent content, then there's nothing wrong with competition eliminating whomever is weakest - the more expensive (not free), the less viral (harder to share), etc.

nobodyandproud 5y ago

Captured my exact sentiments. How many US tech websites were involved in lambasting decisions by France and Australia to force Google to play nice?

Swizec 5y ago

Fewer than 50% of Google searches result in a click these days. Largely thanks to the helpful (to the user) excerpts they provide ... and (I think) the trashy SEO-focused barely readable writing that you often find upon clicking through. https://sparktoro.com/blog/less-than-half-of-google-searches-now-result-in-a-click/ https://sparktoro.com/blog/less-than-half-of-google-searches...

shadowgovt 5y ago

News has always enjoyed / suffered a tricky carve-out in our understanding of the interaction between the first amendment and copyright because (by its nature) its value to the public is in dissemination. So while it's generally not okay to wholesale-copy a news article, it's always been okay to quote it (and, in the concrete and particular case of Google and Facebook, it's also always been okay to point people to it). So the disruption of the revenue model (surprise; turns out people were only getting the ads as a consequence of buying the paper, they weren't why they bought the paper) is particularly tough because the core activity that is disrupting the model (making news universally accessible and useful) is by itself a public good. (It's worth noting that the newspapers in this story have to go through the anti-trust door on this topic specifically because [I assume] there isn't any meat on the bones of claiming ownership of the news Facebook / Google are aggregating.)

13415 5y ago

Potentially stupid questions: Aren't there efficient techniques to prevent web scraping? And if Google really scrapes web content from them without their permission, wouldn't that be commercial copyright infringement which falls under criminal law? That's the thing I don't understand about the current situation.

claudiulodro 5y ago
> Aren't there efficient techniques to prevent web scraping? Not especially, since internet browsers make GET requests the same as scrapers. There are techniques around blocking user agents, rate limiting, etc. but that's an arms race against the scrapers. The tricky part about blocking Google is that if you do, your site won't show up in Google search results. :)
- shaftway 5y ago
  Google in particular always uses an identifiable agent. So you could serve one set of content to Google and a separate set to actual visitors, preventing Google from scraping the content. They clearly want the benefit of being scraped (the traffic) without the cost (the loss of traffic from people who don't click past the headline to the article).
NikolaNovak 5y ago
1. Are there techniques to prevent web scraping? Yes. Are they "Efficient", and even more importantly, effective? Debatable 2. However, more to the point, what's the end game there? If there are 10,000 outlets out there; and arbitrary number, say 100 of them block scrapes; now the other 9,900 will show up in Google/Facebook search and shares, and those who implemented scraping protection will quietly fade into non-existence. It's a damned-if-you-do:damned-if-you-don't choice of modern media consumption model. I assume that most outlets have decided that the exposure, trickle of users and monetization they DO get is superior to the influx of users drying out. It's consequence of the oft-quoted platitude that these days, for most people Google and Facebook ARE "The Internet" :-/

cblconfederate 5y ago

The sad thing is that newspapers allow that themselves. They can block out google and FB but they don't want to. It's a similar tragic fate to that of a drug addict. However if content-makers want to overcome this they need to cut off big tech access to their content, at least temporarily. The frontpage of HN is mostly news articles and some blogs. They get a lot of exposure, a lot of readership. Yet somehow google + FB end up making the money.

colinmhayes 5y ago
Facebook and Google control demand. If a newspaper doesn't let them post their content the aggregators will just grab an article that is nominally the same from a different publisher that does provide them access.
- _aavaa_ 5y ago
  If their content is so interchangeable, why should they all stay in business?
  - colinmhayes 5y ago
    Someone has to actually write the content before everyone else copies it.
    - AniseAbyss 5y ago
      Ultimately Google will just give some desperate Afgan kid a 100 dollar bill and a smartphone. That kid will be killed a few weeks later and replaced by a new one. The new journalists.
    - PeterisP 5y ago
      Copyright is a thing. When many newspapers and news sites have the exact same content, they're generally licensing it from a large news agency which actually wrote the story, it's not someone just illegally ripping off their paywalled content. On the other hand, if multiple sites can each write equivalent content, then there's nothing wrong with competition eliminating whomever is weakest - the more expensive (not free), the less viral (harder to share), etc.