9 ms·
Introducing the Wikimedia Enterprise API
- jmercouris 6y agoVery cool. I hope Wikipedia sees success with this strategy!
- hienyimba 6y agoOver time, I believe more public non-profit sites will introduce this. Then for-profit sites. Until Google eventually pays for most of the valuable content it gets today for free. I own multiple sites where I and my users work to produce valuable data (e.g “so so company reviews”, “Is tenet on Disney” and other data of that kind). And what does Google do? Scrap it all and display it on their page. As a result, the page links gets millions of impressions but tens of clicks. Thus, the sites cannot be monetized. Any reasonable person knows this can’t go on for long before the free and open web comes crashing down or Google (and others like it) pays its due.
- Alupis 6y ago> I own multiple sites where I and my users work to produce valuable data How much do you pay your users for the content they generate?
- loloquwowndueo 6y agoWell nothing because as they said, they can’t monetize the site due to google snatching all the content :)
- riku_iki 6y agoI suspect this project was pushed by Google, to make importing wiki data to their knowledge graph more convenient for them.
- bigtones 6y agoGoogle already have their own knowledge graph that is much bigger than the Wikipedia graph, and they already scrape every Wikipedia page daily so they don't need a Wikipedia API.
- riku_iki 6y agoFor hot topics, search engine wants to scrap every minute, not daily, now wikimedia will provide them with such feed. Also, they need to have team of engineers, who support infobox extractor, now this work will be done by wikimedia.
- brokensegue 6y agowikipedia is a large input into their knowledge graph
- theamk 6y agoIf Google scraping your sites is a bad thing, you want to set "nosnippet" tags on your page [0]. If Google scraping your sites is a good thing, then why are you complaining? I hope Google never starts paying for the links. Once there is a precedent, this becomes an effective blocker for the new search engines, visualizers, and other exciting web search startups. A new search engine startup is not going to be able to establish a commercial relationship with every site on the web like Google could. [0] https://developers.google.com/search/docs/advanced/appearance/featured-snippets https://developers.google.com/search/docs/advanced/appearanc...
- jlawer 6y agoThe one issue I see with this is it is always Opt Out. I feel that google really should be lining up partners to opt-in. While I am sure there is reasons why Google believe they have the right (and a good case can be made), it always feels slightly entitled to just assume that people are OK with this being done to their content. That being said, of all the sources, Wikipedia actively license their content in such a way that google are well within their rights to slurp it all down and serve it however they want. Google is already effectively paying for links to news sites as part of the negotiations in Australia. And I agree that this will be a dampener on any competition, I think the era of "ask for forgiveness, rather then permission" needs to stop.
- notatoad 6y agoif you post information publicly on the internet, google is entitled to scrape it. you've opted in by publishing it. if you want to specifically exclude one entity from accessing information that you've posted for anybody to see, i'm not sure how there's a way that could be "opt-in"
- watchdogtimer 6y agoYou could do this using a robots.txt file (assuming the scraper obeys it, of course).
- tobylane 6y ago
- jedc 6y agoWhile Google can use Wikimedia for free, they do make financial contributors to the Wikimedia ecosystem. https://wikimediafoundation.org/news/2019/01/22/google-and-wikimedia-foundation-partner-to-increase-knowledge-equity-online/ https://wikimediafoundation.org/news/2019/01/22/google-and-w...
- karmasimida 6y agoYou can actually download the whole wikipedia if you like.
- mrkramer 6y agoWhy would Google pay for this when they already crawled and are crawling whole Wikipedia and have complete index of it? Better way for Wikipedia to earn extra revenue are affiliate links. A lot of people when they read and learn about some topic go to Amazon and buy a book about that topic. Wikipedia could embed book affiliate links and earn commission from book sales.
- judge2020 6y agoWell, they are already a big donor: https://wikimediafoundation.org/about/annualreport/2019-annual-report/donors/ https://wikimediafoundation.org/about/annualreport/2019-annu... > Google Matching Gifts Program
- Clewza313 6y agoIf you work at Google, they will 1:1 match donations to virtually any non-profit, plus there's various charity drives where employees get to donate company money. So the match program can become a huge donor just off random Googlers donating.
- throwaway53453 6y agoThe distinction doesn't really matter though, does it? Makes no different if it's Googlers as opposed to Google itself.
- superluserdo 6y agoThat sounds like a horribly perverse incentive for the world's main free and open source of information.
- mrkramer 6y agoI mean Wikipedia offers basic information about some topic it's not like you are about to get deep insight unless you buy a book. And I bet Wikipedia generated millions of book sales like I said people who read an article from Wikipedia and went to Amazon or Google to search for a book.
- OJFord 6y agoGreat, just a shame it isn't more 'tradititionally' transparent & democratised IMO. Claims no custom contracts, but is enterprise sales team contact us anyway, for example. .proto on GitHub is nice, but no pricing, no public docs? This is probably great for Wikimedia coffers, but at the headline I hoped for new/improved Wikidata; instead it's.. different bordering on 'don't care'.
- kenrick95 6y agoI guess the article are targeted towards general public, not technical people I found this page which has more technical details on what it actually is: https://www.mediawiki.org/wiki/Wikimedia_Enterprise https://www.mediawiki.org/wiki/Wikimedia_Enterprise Also found out that it is open source: https://github.com/wikimedia/OKAPI https://github.com/wikimedia/OKAPI
- OJFord 6y agoThose are linked from the article, (that's the .proto on GitHub I mentioned) but what're we going to do with that? (And why does everything have to be formatted like a wiki page..) I mean, it's fine, I just got momentarily excited for something that the announcement isn't. I wanted to find a pricing page, free tier, API docs, etc. Like Wikidata but.. I don't want to say 'modernised', but made more accessible, and with APIs for higher level content like this rather than just rawer data.
- tylerrobinson 6y agoAre any details known yet about the format or structure of the API? I didn’t see anything in the article.
- kenrick95 6y agohttps://www.mediawiki.org/wiki/Wikimedia_Enterprise#Active_Development_(Beta) https://www.mediawiki.org/wiki/Wikimedia_Enterprise#Active_D...
- schappim 6y agoDoes anyone know how much the Wikimedia Enterprise API costs?
- kradroy 6y agoIt's "enterprise", so you have to talk to a human and haggle. More info is located on the FAQ link in the article.
- shinkim0914 6y agoIf this sets them on a self-sustaining path without having to rely on running highly conspicuous donation campaigns on Wikipedia, I think that's a wonderful thing.
- 29athrowaway 6y agoNot so fast. Imagine Wikimedia Enterprise becomes the #1 source of revenue for Wikimedia. Shortly after, people will see that Wikimedia is doing OK and become reluctant to open their wallets and donate. Then, the top Wikimedia Enterprise customers will acquire leverage over Wikimedia and try to get Wikipedia curated to their convenience. Wikipedia articles will start being indistinguishable from advertisement. Governments will intervene and want their share of influence too. Top volunteers will start asking to be paid, many others will leave, some others will become critics of the project. People will start being skeptical of Wikipedia because of their biased editorial line and then the project will be declared a failure, once everyone is angry and a beautiful project is torn apart by greed.
- ckoerner 6y agoThe Foundation has already considered this. https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ#How_much_money_will_this_raise https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ#How...
- ROARosen 6y agoWith all due respect to Wikipedia for what it is, I believe their success is partly because of the 'altruistic' nature of their model. Sure, they should seek donations from huge companies like Google - which make tons of money off of their data - for the services they provide, but I feel like locking down the 'better' api to the public is not the way to go about it. It's just too often that a commercial offering just disincentives bettering the 'free' product. Wikipedia as the product of a public good foundation should be just that; by the public, for the public, and accessible to the public (including all access methods and API's).
- dannyw 6y agoWikipedian here. We edit because we're contributing to free information. Free information means anyone can use it for any purpose, including commercially. I think this move is great. I'd rather have this money go to the foundation, than ParseAPIco.
- ROARosen 6y ago> Free information means anyone can use it ... Of course anyone can use it commercially and for whichever reason they like. But by creating a walled 'premium' offering you are going against the premise of 'anyone can use it' since not anyone can use the commercial api. Surely if Google or anyone wants the api so badly they're willing to pay for it, they should fund it. Why does that mean it needs to be 'locked'?
- abbe98 6y agoGoogle and co will essentially pay for the SLA, free access will be available through Wikimedia Cloud Services and I'm sure that the team is investigating how to make the API available to the wider public.
- pimlottc 6y agoDevils advocate: Google clearly already has a working pipeline to import and format Wikipedia data for its needs. Why would they stop using it and start paying Wikipedia? Will Wikipedia be able to build an enterprise API thats faster/cheaper/more reliable/more scalable than the internal one build by one of the world’s top engineering companies? No doubt the enterprise API will add attractive value for smaller companies without the resources to process the raw dumps but I’m skeptical that this will convert Google et al into well-paying customers. Unless they start restricting the free dumps...
- ThinkBeat 6y agoI hope this is not the first step into a worrisome future. It appears now that they are offering "read-only" access to existing data structured and packaged in a more convenient way. How long before paying enterprises would like to be able to "update" content on a more efficient basis? Perhaps Sony would like to add articles about movies that will be released soon, or as they are released? That is a pretty benign example. Creating alerts that enterprises can subscribe to so that they will be informed if anyone adds any negative content would also be valuable. These systems already exist in some manner, it would just make it more efficient and more common.
- rambojazz 6y agoThey didn't mention pricing, did they?
- cblconfederate 6y agoThis just feels the wrong, trying to push the new concept of open source that SV created in the past decade to the general public. Most wikipedians are still in early 00s idealism , and will push back against this "dual model" crap, and they will be right.
- elect_engineer 6y agoWikimedia Enterprise timeline: https://meta.wikimedia.org/wiki/Wikimedia_Forum#Wikimedia_Enterprise_timeline https://meta.wikimedia.org/wiki/Wikimedia_Forum#Wikimedia_En... Also see Wikipedia: Camel's nose: https://en.wikipedia.org/wiki/Camel%27s_nose https://en.wikipedia.org/wiki/Camel%27s_nose