7 ms·

Wikidata, the free knowledge base that anyone can edit

153 points by a3_nm 14y ago

fasouto 14y ago
So... it's like a Freebase clone?
- lifthrasiir 14y ago
  Similar but not exactly. To my understanding, Wikibase was introduced to avoid the duplication of language-neutral information (like infoboxes) and to improve the coherence of same subject on multiple projects (like cross-language interwikis). Both are similar goals (and can be realized with the very same tools like RDF) but the latter is quite specific to Wikimedia projects.
- neilk 14y ago
  If I understand it correctly, it has more modest goals. Freebase was trying to make the uber-map of all data entities. Wikidata is just trying to make data reuse easier on Wikipedia. For instance, imagine a table of all the populations of the countries of the world. Today, someone might make a really good one for the French Wikipedia. But then someone has to make it from scratch, all over again, for the Greek Wikipedia. And when someone updates the French one, the Greek one doesn't update, and vice versa. With Wikidata you can define the data once, and then transclude it to different pages, with translated labels if necessary. The first release attacks the problem of "inter-wiki links". On the left hand side of some Wikipedia pages, there are the links to equivalent pages in different languages. Check out the one for http://en.wikipedia.org/wiki/Jimmy_Wales http://en.wikipedia.org/wiki/Jimmy_Wales, for instance. Right now these are updated with a system that looks at every possible connection (scaling at O(n2)), and with Wikidata it will be more manageable.
  - zeckalpha 14y ago
    Interestingly, interwiki links sometimes have some semantic drifting effect. from Omnipedia http://brenthecht.com/papers/bhecht_CHI2012_omnipedia.pdf http://brenthecht.com/papers/bhecht_CHI2012_omnipedia.pdf [pdf]: """ One major source of ambiguities in the ILL graph is conceptual drift across language editions. Conceptual drift stems from the well-known finding in cognitive science that the boundaries of concepts vary across language-defined communities [13]. For instance, the English articles “High school” and “Secondary school” are grouped into a single connected concept. While placing these two articles in the same multilingual article may be reasonable given their overlapping definitions around the world, excessive conceptual drift can result in a semantic equivalent of what happens in the children’s game known as “telephone”. For instance, chains of conceptual drift expand the aforementioned connected concept to include the English articles “Primary school”, “Etiquette”, “Manners”, and even “Protocol (diplomacy)”. Omnipedia users would be confused to see “Kyoto Protocol” as a linked topic when they looked up “High school”. A similar situation occurs in the large connected concept that spans the semantic range from “River” to “Canal” to “Trench warfare”, and in another which contains “Woman” and “Marriage” (although, interestingly, not “Man”). """
  - fasouto 14y ago
    Thanks, it makes more sense now.
  - xirzon 14y ago
    Wikidata is trying to make re-use possible beyond Wikipedia, as well. In fact there's already a couple of apps built with it. Here's a trivial genealogy visualization using the API: https://toolserver.org/~magnus/ts2/geneawiki/?q=Q1339 https://toolserver.org/~magnus/ts2/geneawiki/?q=Q1339 It's also worth noting that Freebase itself heavily relied on parsing Wikipedia database dumps to build its ontology -- to a large extent Wikidata is giving structure to data that's been in Wikipedia all along.
- troymc 14y ago
  For one thing, Wikidata data has a different intellectual property regime. Wikidata data is dedicated to the public domain, using http://creativecommons.org/publicdomain/zero/1.0/ http://creativecommons.org/publicdomain/zero/1.0/ Most Freebase data licensed under a CC-BY license. Details are here: http://www.freebase.com/policies/attribution http://www.freebase.com/policies/attribution A CC-BY license can be a burden, if you really want to fulfill all the terms of the license, namely: "You must attribute the work in the manner specified by the author or licensor." If there are 20,00 authors, are you really going to find out how each one wants you to give them attribution? It's impractical, so what you end up doing is giving what you think is reasonable attribution. But you never really know for sure. Even worse, some of the material in Freebase is under other licenses, such as CC-BY-SA or GFDL.
ctdonath 14y ago
Assurances of accuracy?
- hsshah 14y ago
  Thats the same question you can ask for Wikipedia as well!
- anigbrowl 14y ago
  Citations.
- lifthrasiir 14y ago
  Similar to Wikipedia. That is, every triple ("statement") present in the Wikidata may have an associated source. This is a mechanized version of [citation needed]...
- tommorris 14y ago
  Each statement will have the ability to have sources. This is not currently supported by the UI (hence everyone is being a bit tentative and only putting in really obvious, uncontentious things) but when it does, it'll basically contain expressions of the form "X has property Y with a value of Z (type T), according to sources A, B and C".
mkx 14y ago
Interesting to see if this will open up the web to more linked data based trends.
roryokane 14y ago
Note that the four-part barcode logo is “WIKI” in Morse code.
danso 14y ago
What...is...this? Is there any organization of these datapoints? All I see on the front page is links to single datapoints, such as this: http://www.wikidata.org/wiki/Q50000 http://www.wikidata.org/wiki/Q50000 Which corresponds to "victory", defined as "term that applies to success" , and also known as "win" and "success". Sorry, but what's the need here? This just seems to dilute whatever's going on at Wikipedia.
- zeckalpha 14y ago
  Languages don't have bijective mappings of concepts, so this is a hard problem. Do you have an ontology you'd like to propose?
  - danso 14y ago
    No, I mean I was really confused. The term "WikiData" seems to connote data of the tabular type, like a central repository for public data. Though I'm also confused at how the mapping for this particular term (for "victory") can't be done in Wikipedia or the Wiki dictionary.
    - zeckalpha 14y ago
      It is, but the problem is whether you are talking about an individual language version of Wikipedia or the Wikipedia project as a whole. In the article they talk about the problem of maintaining Interwiki links on each individual language version, rather than centrally. This is also just one aspect of Wikidata. The centrality of shared table content is important, too. Why have data in a specific language version Wikipedia and point to it from other versions when you can have a central repository that is pointed to using templates from each language version?
    - gngeal 14y ago
      "The term "WikiData" seems to connote data of the tabular type" There's a strong correspondence between "tabular data" (you probably mean relational) and triples (<predicate,X,Y>). Bot are based on the first-order predicate logic, so there's actually a natural mapping.
MetalMASK 14y ago
Let's see, is it similar to WordNet but with open access (i.e., anyone can edit)?
- lignuist 14y ago
  Wordnet is more a linguistical ressource, that focusses on word senses. You would not find things like POPULATION OF CITY-X there.
jpatokal 14y ago
First of all, having worked with wikis for about ten years, finally seeing this live in Wikipedia is huge -- but it'll take a long time for it to have full impact. The first baby step is interwiki links. Previously, every article in every version of Wikipedia had a long list of links to every other version of that article, maintained by a small army of flaky bots that clogged up edit histories and often stepped on each others' toes. Now, there's a single Wikidata node that has the single mapping of what article corresponds to what, reflected across all Wikipedia versions. Here's "Tokyo" now: http://www.wikidata.org/wiki/Q1490 http://www.wikidata.org/wiki/Q1490 And here's what it replaces, repeated dozens of times in every language: http://en.wikipedia.org/w/index.php?title=Tokyo&diff=539264613&oldid=539032776 http://en.wikipedia.org/w/index.php?title=Tokyo&diff=539... The next step will be infoboxes. Instead of every Wikipedia having a separate copy of the population of Tokyo or the GDP of Canada, updated ad hoc by different people whenever they get around to it, there will be a single place storing that data automatically reflected into all Wikipedias. And it keeps going. Taxonomies of plants and animals change all the time, Wikidata can become their single repository. Wikivoyage currently has to store the phone numbers of each hotel separately in each language version, Wikidata will allow centralizing them. Here's the master plan: http://en.wikipedia.org/wiki/Wikipedia:Wikidata http://en.wikipedia.org/wiki/Wikipedia:Wikidata
- noahl 14y ago
  That page for Tokyo looks suspiciously like the internals of a natural language processing system. I can't wait until someone hooks up a probabilistic speech parser to Wikidata. A talking computer with the knowledge of Wikipedia could be incredible.
wahnfrieden 14y ago
Cool to see my jQuery tag widget[1] in use here :) [1] http://aehlke.github.com/tag-it/ http://aehlke.github.com/tag-it/
andreyf 14y ago
Populating this by hand seems like an enormous task that will never end. How hard could it be to automatically populate the data from publicly available data sources (e.g. SEC filings)?
- nightrose 14y ago
  That's actually already happening and will happen more in the future.
SquareWheel 14y ago
I'm not big into the Wiki world, but it's also struck me as odd how different pages refer to the same facts and yet are totally disparate. If one page updates, does somebody manually have to go update the other page? Does a bot do it? This looks like a great response to that. I just hope they've made it easy to interface with.
- jpatokal 14y ago
  Yup, manual updates and the occasional bot is how it works (or, more often, doesn't) in the pre-Wikidata world.
stck 14y ago
Where does the fact numbering come from? Q1 - universe Q2 - Earth Q3 - life ... Q24 - Jack Bauer ... Q76 - Barack Obama
- tommorris 14y ago
  It's sequential.
  - nightrose 14y ago
    Right. With possibly some easter eggs ;-)
pella 14y ago
Atlantic.com (Apr2012) "The Problem With Wikidata" http://www.theatlantic.com/technology/archive/2012/04/the-problem-with-wikidata/255564/ http://www.theatlantic.com/technology/archive/2012/04/the-pr...
Aissen 14y ago
There's already http://dbpedia.org http://dbpedia.org , but it's nice to see the Wikimedia foundation finally take matters into their own hands.
Stratoscope 14y ago
I'm pretty excited about this, if part of the plan is to encourage Wikipedia contributors to add their data as data instead of as finished output. I've spent some time lately building election results maps for Google, and while Wikipedia has been a useful resource, it's also been very frustrating. There is a wealth of geographic data on Wikipedia, but hardly any of it is in a usable form. Look at all the interesting maps you see there. Somebody generated each of those maps from actual usable data such as shapefiles [1] or KML files [2]. The generated maps are nice to look at, but I can't do anything else with them. I need the shapefiles or equivalent to build other kinds of maps with the data. For the Brazil election, we had shapefiles for the municipalities which had been mangled by a process that converted all the municipality names to uppercase with no accent marks. I wanted to display the correctly capitalized and accented names and found this list: http://en.wikipedia.org/wiki/List_of_municipalities_of_Brazil http://en.wikipedia.org/wiki/List_of_municipalities_of_Brazi... This was great! It had all the data I needed. But it's in a jumbled format that looks nice in the wiki but doesn't lend itself to machine use. It took me a few hours to write some Python code to parse the page and get it into a CSV format that I could import into my database. This is a frustrating kind of work, because I was pretty sure that somebody had started with nice tabular data and generated the initial version of this Wikipedia page from that. If we're eventually able to get this kind of data as real usable data, that will save a lot of people a lot of work. [1] http://en.wikipedia.org/wiki/Shapefile http://en.wikipedia.org/wiki/Shapefile [2] http://en.wikipedia.org/wiki/Keyhole_Markup_Language http://en.wikipedia.org/wiki/Keyhole_Markup_Language
eykanal 14y ago
I just read through the entire front page and I have no idea: 1) what need wikidata will fill 2) what they want me to do 3) how I would go about doing (2) (other than "starting the wikidata community", mentioned at the bottom of the page, which sounds like a lot of work). Maybe I'm not their target audience, but this sure as hell isn't a good elevator pitch.
whoisstan 14y ago
This could be a heaven-sent for data exchange and knowledge structures. If data subjects are labeled with wikidata nouns it could much more interchangeable. RDF should have gone that route a long time ago.

153 points by a3_nm 14y ago

fasouto 14y ago

So... it's like a Freebase clone?

lifthrasiir 14y ago
Similar but not exactly. To my understanding, Wikibase was introduced to avoid the duplication of language-neutral information (like infoboxes) and to improve the coherence of same subject on multiple projects (like cross-language interwikis). Both are similar goals (and can be realized with the very same tools like RDF) but the latter is quite specific to Wikimedia projects.
neilk 14y ago
If I understand it correctly, it has more modest goals. Freebase was trying to make the uber-map of all data entities. Wikidata is just trying to make data reuse easier on Wikipedia. For instance, imagine a table of all the populations of the countries of the world. Today, someone might make a really good one for the French Wikipedia. But then someone has to make it from scratch, all over again, for the Greek Wikipedia. And when someone updates the French one, the Greek one doesn't update, and vice versa. With Wikidata you can define the data once, and then transclude it to different pages, with translated labels if necessary. The first release attacks the problem of "inter-wiki links". On the left hand side of some Wikipedia pages, there are the links to equivalent pages in different languages. Check out the one for http://en.wikipedia.org/wiki/Jimmy_Wales http://en.wikipedia.org/wiki/Jimmy_Wales, for instance. Right now these are updated with a system that looks at every possible connection (scaling at O(n2)), and with Wikidata it will be more manageable.
- zeckalpha 14y ago
  Interestingly, interwiki links sometimes have some semantic drifting effect. from Omnipedia http://brenthecht.com/papers/bhecht_CHI2012_omnipedia.pdf http://brenthecht.com/papers/bhecht_CHI2012_omnipedia.pdf [pdf]: """ One major source of ambiguities in the ILL graph is conceptual drift across language editions. Conceptual drift stems from the well-known finding in cognitive science that the boundaries of concepts vary across language-defined communities [13]. For instance, the English articles “High school” and “Secondary school” are grouped into a single connected concept. While placing these two articles in the same multilingual article may be reasonable given their overlapping definitions around the world, excessive conceptual drift can result in a semantic equivalent of what happens in the children’s game known as “telephone”. For instance, chains of conceptual drift expand the aforementioned connected concept to include the English articles “Primary school”, “Etiquette”, “Manners”, and even “Protocol (diplomacy)”. Omnipedia users would be confused to see “Kyoto Protocol” as a linked topic when they looked up “High school”. A similar situation occurs in the large connected concept that spans the semantic range from “River” to “Canal” to “Trench warfare”, and in another which contains “Woman” and “Marriage” (although, interestingly, not “Man”). """
- fasouto 14y ago
  Thanks, it makes more sense now.
- xirzon 14y ago
  Wikidata is trying to make re-use possible beyond Wikipedia, as well. In fact there's already a couple of apps built with it. Here's a trivial genealogy visualization using the API: https://toolserver.org/~magnus/ts2/geneawiki/?q=Q1339 https://toolserver.org/~magnus/ts2/geneawiki/?q=Q1339 It's also worth noting that Freebase itself heavily relied on parsing Wikipedia database dumps to build its ontology -- to a large extent Wikidata is giving structure to data that's been in Wikipedia all along.
troymc 14y ago
For one thing, Wikidata data has a different intellectual property regime. Wikidata data is dedicated to the public domain, using http://creativecommons.org/publicdomain/zero/1.0/ http://creativecommons.org/publicdomain/zero/1.0/ Most Freebase data licensed under a CC-BY license. Details are here: http://www.freebase.com/policies/attribution http://www.freebase.com/policies/attribution A CC-BY license can be a burden, if you really want to fulfill all the terms of the license, namely: "You must attribute the work in the manner specified by the author or licensor." If there are 20,00 authors, are you really going to find out how each one wants you to give them attribution? It's impractical, so what you end up doing is giving what you think is reasonable attribution. But you never really know for sure. Even worse, some of the material in Freebase is under other licenses, such as CC-BY-SA or GFDL.

ctdonath 14y ago

Assurances of accuracy?

hsshah 14y ago
Thats the same question you can ask for Wikipedia as well!
anigbrowl 14y ago
Citations.
lifthrasiir 14y ago
Similar to Wikipedia. That is, every triple ("statement") present in the Wikidata may have an associated source. This is a mechanized version of [citation needed]...
tommorris 14y ago
Each statement will have the ability to have sources. This is not currently supported by the UI (hence everyone is being a bit tentative and only putting in really obvious, uncontentious things) but when it does, it'll basically contain expressions of the form "X has property Y with a value of Z (type T), according to sources A, B and C".

mkx 14y ago

Interesting to see if this will open up the web to more linked data based trends.

roryokane 14y ago

Note that the four-part barcode logo is “WIKI” in Morse code.

danso 14y ago

What...is...this? Is there any organization of these datapoints? All I see on the front page is links to single datapoints, such as this: http://www.wikidata.org/wiki/Q50000 http://www.wikidata.org/wiki/Q50000 Which corresponds to "victory", defined as "term that applies to success" , and also known as "win" and "success". Sorry, but what's the need here? This just seems to dilute whatever's going on at Wikipedia.

zeckalpha 14y ago
Languages don't have bijective mappings of concepts, so this is a hard problem. Do you have an ontology you'd like to propose?
- danso 14y ago
  No, I mean I was really confused. The term "WikiData" seems to connote data of the tabular type, like a central repository for public data. Though I'm also confused at how the mapping for this particular term (for "victory") can't be done in Wikipedia or the Wiki dictionary.
  - zeckalpha 14y ago
    It is, but the problem is whether you are talking about an individual language version of Wikipedia or the Wikipedia project as a whole. In the article they talk about the problem of maintaining Interwiki links on each individual language version, rather than centrally. This is also just one aspect of Wikidata. The centrality of shared table content is important, too. Why have data in a specific language version Wikipedia and point to it from other versions when you can have a central repository that is pointed to using templates from each language version?
  - gngeal 14y ago
    "The term "WikiData" seems to connote data of the tabular type" There's a strong correspondence between "tabular data" (you probably mean relational) and triples (<predicate,X,Y>). Bot are based on the first-order predicate logic, so there's actually a natural mapping.

MetalMASK 14y ago

Let's see, is it similar to WordNet but with open access (i.e., anyone can edit)?

lignuist 14y ago
Wordnet is more a linguistical ressource, that focusses on word senses. You would not find things like POPULATION OF CITY-X there.

jpatokal 14y ago

First of all, having worked with wikis for about ten years, finally seeing this live in Wikipedia is huge -- but it'll take a long time for it to have full impact. The first baby step is interwiki links. Previously, every article in every version of Wikipedia had a long list of links to every other version of that article, maintained by a small army of flaky bots that clogged up edit histories and often stepped on each others' toes. Now, there's a single Wikidata node that has the single mapping of what article corresponds to what, reflected across all Wikipedia versions. Here's "Tokyo" now: http://www.wikidata.org/wiki/Q1490 http://www.wikidata.org/wiki/Q1490 And here's what it replaces, repeated dozens of times in every language: http://en.wikipedia.org/w/index.php?title=Tokyo&diff=539264613&oldid=539032776 http://en.wikipedia.org/w/index.php?title=Tokyo&diff=539... The next step will be infoboxes. Instead of every Wikipedia having a separate copy of the population of Tokyo or the GDP of Canada, updated ad hoc by different people whenever they get around to it, there will be a single place storing that data automatically reflected into all Wikipedias. And it keeps going. Taxonomies of plants and animals change all the time, Wikidata can become their single repository. Wikivoyage currently has to store the phone numbers of each hotel separately in each language version, Wikidata will allow centralizing them. Here's the master plan: http://en.wikipedia.org/wiki/Wikipedia:Wikidata http://en.wikipedia.org/wiki/Wikipedia:Wikidata

noahl 14y ago
That page for Tokyo looks suspiciously like the internals of a natural language processing system. I can't wait until someone hooks up a probabilistic speech parser to Wikidata. A talking computer with the knowledge of Wikipedia could be incredible.

wahnfrieden 14y ago

Cool to see my jQuery tag widget[1] in use here :) [1] http://aehlke.github.com/tag-it/ http://aehlke.github.com/tag-it/

andreyf 14y ago

Populating this by hand seems like an enormous task that will never end. How hard could it be to automatically populate the data from publicly available data sources (e.g. SEC filings)?

nightrose 14y ago
That's actually already happening and will happen more in the future.

SquareWheel 14y ago

I'm not big into the Wiki world, but it's also struck me as odd how different pages refer to the same facts and yet are totally disparate. If one page updates, does somebody manually have to go update the other page? Does a bot do it? This looks like a great response to that. I just hope they've made it easy to interface with.

jpatokal 14y ago
Yup, manual updates and the occasional bot is how it works (or, more often, doesn't) in the pre-Wikidata world.

stck 14y ago

Where does the fact numbering come from? Q1 - universe Q2 - Earth Q3 - life ... Q24 - Jack Bauer ... Q76 - Barack Obama

tommorris 14y ago
It's sequential.
- nightrose 14y ago
  Right. With possibly some easter eggs ;-)

pella 14y ago

Atlantic.com (Apr2012) "The Problem With Wikidata" http://www.theatlantic.com/technology/archive/2012/04/the-problem-with-wikidata/255564/ http://www.theatlantic.com/technology/archive/2012/04/the-pr...

Aissen 14y ago

There's already http://dbpedia.org http://dbpedia.org , but it's nice to see the Wikimedia foundation finally take matters into their own hands.

Stratoscope 14y ago

I'm pretty excited about this, if part of the plan is to encourage Wikipedia contributors to add their data as data instead of as finished output. I've spent some time lately building election results maps for Google, and while Wikipedia has been a useful resource, it's also been very frustrating. There is a wealth of geographic data on Wikipedia, but hardly any of it is in a usable form. Look at all the interesting maps you see there. Somebody generated each of those maps from actual usable data such as shapefiles [1] or KML files [2]. The generated maps are nice to look at, but I can't do anything else with them. I need the shapefiles or equivalent to build other kinds of maps with the data. For the Brazil election, we had shapefiles for the municipalities which had been mangled by a process that converted all the municipality names to uppercase with no accent marks. I wanted to display the correctly capitalized and accented names and found this list: http://en.wikipedia.org/wiki/List_of_municipalities_of_Brazil http://en.wikipedia.org/wiki/List_of_municipalities_of_Brazi... This was great! It had all the data I needed. But it's in a jumbled format that looks nice in the wiki but doesn't lend itself to machine use. It took me a few hours to write some Python code to parse the page and get it into a CSV format that I could import into my database. This is a frustrating kind of work, because I was pretty sure that somebody had started with nice tabular data and generated the initial version of this Wikipedia page from that. If we're eventually able to get this kind of data as real usable data, that will save a lot of people a lot of work. [1] http://en.wikipedia.org/wiki/Shapefile http://en.wikipedia.org/wiki/Shapefile [2] http://en.wikipedia.org/wiki/Keyhole_Markup_Language http://en.wikipedia.org/wiki/Keyhole_Markup_Language

eykanal 14y ago

I just read through the entire front page and I have no idea: 1) what need wikidata will fill 2) what they want me to do 3) how I would go about doing (2) (other than "starting the wikidata community", mentioned at the bottom of the page, which sounds like a lot of work). Maybe I'm not their target audience, but this sure as hell isn't a good elevator pitch.

whoisstan 14y ago

This could be a heaven-sent for data exchange and knowledge structures. If data subjects are labeled with wikidata nouns it could much more interchangeable. RDF should have gone that route a long time ago.