5 ms·

Over time, I believe more public non-profit sites will introduce this. Then for-profit sites. Until Google eventually pays for most of the valuable content it g

Over time, I believe more public non-profit sites will introduce this. Then for-profit sites. Until Google eventually pays for most of the valuable content it gets today for free. I own multiple sites where I and my users work to produce valuable data (e.g “so so company reviews”, “Is tenet on Disney” and other data of that kind). And what does Google do? Scrap it all and display it on their page. As a result, the page links gets millions of impressions but tens of clicks. Thus, the sites cannot be monetized. Any reasonable person knows this can’t go on for long before the free and open web comes crashing down or Google (and others like it) pays its due.

Alupis 6y ago
> I own multiple sites where I and my users work to produce valuable data How much do you pay your users for the content they generate?
- loloquwowndueo 6y ago
  Well nothing because as they said, they can’t monetize the site due to google snatching all the content :)
riku_iki 6y ago
I suspect this project was pushed by Google, to make importing wiki data to their knowledge graph more convenient for them.
- bigtones 6y ago
  Google already have their own knowledge graph that is much bigger than the Wikipedia graph, and they already scrape every Wikipedia page daily so they don't need a Wikipedia API.
  - riku_iki 6y ago
    For hot topics, search engine wants to scrap every minute, not daily, now wikimedia will provide them with such feed. Also, they need to have team of engineers, who support infobox extractor, now this work will be done by wikimedia.
  - brokensegue 6y ago
    wikipedia is a large input into their knowledge graph
theamk 6y ago
If Google scraping your sites is a bad thing, you want to set "nosnippet" tags on your page [0]. If Google scraping your sites is a good thing, then why are you complaining? I hope Google never starts paying for the links. Once there is a precedent, this becomes an effective blocker for the new search engines, visualizers, and other exciting web search startups. A new search engine startup is not going to be able to establish a commercial relationship with every site on the web like Google could. [0] https://developers.google.com/search/docs/advanced/appearance/featured-snippets https://developers.google.com/search/docs/advanced/appearanc...
- jlawer 6y ago
  The one issue I see with this is it is always Opt Out. I feel that google really should be lining up partners to opt-in. While I am sure there is reasons why Google believe they have the right (and a good case can be made), it always feels slightly entitled to just assume that people are OK with this being done to their content. That being said, of all the sources, Wikipedia actively license their content in such a way that google are well within their rights to slurp it all down and serve it however they want. Google is already effectively paying for links to news sites as part of the negotiations in Australia. And I agree that this will be a dampener on any competition, I think the era of "ask for forgiveness, rather then permission" needs to stop.
  - notatoad 6y ago
    if you post information publicly on the internet, google is entitled to scrape it. you've opted in by publishing it. if you want to specifically exclude one entity from accessing information that you've posted for anybody to see, i'm not sure how there's a way that could be "opt-in"
    - watchdogtimer 6y ago
      You could do this using a robots.txt file (assuming the scraper obeys it, of course).
    - tobylane 6y ago
jedc 6y ago
While Google can use Wikimedia for free, they do make financial contributors to the Wikimedia ecosystem. https://wikimediafoundation.org/news/2019/01/22/google-and-wikimedia-foundation-partner-to-increase-knowledge-equity-online/ https://wikimediafoundation.org/news/2019/01/22/google-and-w...
karmasimida 6y ago
You can actually download the whole wikipedia if you like.

Alupis 6y ago

> I own multiple sites where I and my users work to produce valuable data How much do you pay your users for the content they generate?

loloquwowndueo 6y ago
Well nothing because as they said, they can’t monetize the site due to google snatching all the content :)

riku_iki 6y ago

I suspect this project was pushed by Google, to make importing wiki data to their knowledge graph more convenient for them.

bigtones 6y ago
Google already have their own knowledge graph that is much bigger than the Wikipedia graph, and they already scrape every Wikipedia page daily so they don't need a Wikipedia API.
- riku_iki 6y ago
  For hot topics, search engine wants to scrap every minute, not daily, now wikimedia will provide them with such feed. Also, they need to have team of engineers, who support infobox extractor, now this work will be done by wikimedia.
- brokensegue 6y ago
  wikipedia is a large input into their knowledge graph

theamk 6y ago

If Google scraping your sites is a bad thing, you want to set "nosnippet" tags on your page [0]. If Google scraping your sites is a good thing, then why are you complaining? I hope Google never starts paying for the links. Once there is a precedent, this becomes an effective blocker for the new search engines, visualizers, and other exciting web search startups. A new search engine startup is not going to be able to establish a commercial relationship with every site on the web like Google could. [0] https://developers.google.com/search/docs/advanced/appearance/featured-snippets https://developers.google.com/search/docs/advanced/appearanc...

jlawer 6y ago
The one issue I see with this is it is always Opt Out. I feel that google really should be lining up partners to opt-in. While I am sure there is reasons why Google believe they have the right (and a good case can be made), it always feels slightly entitled to just assume that people are OK with this being done to their content. That being said, of all the sources, Wikipedia actively license their content in such a way that google are well within their rights to slurp it all down and serve it however they want. Google is already effectively paying for links to news sites as part of the negotiations in Australia. And I agree that this will be a dampener on any competition, I think the era of "ask for forgiveness, rather then permission" needs to stop.
- notatoad 6y ago
  if you post information publicly on the internet, google is entitled to scrape it. you've opted in by publishing it. if you want to specifically exclude one entity from accessing information that you've posted for anybody to see, i'm not sure how there's a way that could be "opt-in"
  - watchdogtimer 6y ago
    You could do this using a robots.txt file (assuming the scraper obeys it, of course).
  - tobylane 6y ago

jedc 6y ago

While Google can use Wikimedia for free, they do make financial contributors to the Wikimedia ecosystem. https://wikimediafoundation.org/news/2019/01/22/google-and-wikimedia-foundation-partner-to-increase-knowledge-equity-online/ https://wikimediafoundation.org/news/2019/01/22/google-and-w...

karmasimida 6y ago

You can actually download the whole wikipedia if you like.