3 ms·

Using RDF to store multiple languages of the same article is gross overengineering. Interlinking with other projects isn't nessecary when there are no other pro

by j-pb 5y ago

Using RDF to store multiple languages of the same article is gross overengineering. Interlinking with other projects isn't nessecary when there are no other projects.

bawolff 5y ago
It isn't stored as RDF. RDF is provided as an export format. The language contents also aren't stored in wikidata, its just matching which title in language X matches which title in language Y. Wikidata isn't just used for interlanguage linking. Its also used to keep infoboxes synced across different languages, and a way to easily query that data (Depends a bit on the language, english wikipedia isn't fully on board with the system, but its been a big boon to smaller languages). As a result, people can update these "facts" independently of language, keeping them altogether more up to date, and it allows the data to be queried independently (via https://query.wikidata.org https://query.wikidata.org - try some of the example queries (button top left) if you haven't, the level of power SPARQL gives for querying this type of data set really is very cool). Part of Wikimedia's mission is to spread knowledge. Allowing querying of factual data with a query language like sparql helps advance that goal by letting people use extract the knowledge for new purposes.
- j-pb 5y ago
  > its just matching which title in language X matches which title in language Y Like I said, gross overengineering. "You can save cents and seconds of time updating knowledge boxes, by spending millions and hours developing federated SPARQL query capabilities!"
  - bawolff 5y ago
    More like, we put the facts and knowledge in a central (mysql) db, so that instead of updating them in 300 places (Wikipedia has a lot of languages), you just update them in one place. And since the data happened to be there anyways, we added an RDF export feature, and threw that into a BlazeGraph DB so that people could do SPARQL queries if they want. The (not really federated) SPARQL queries have basically nothing to do with the central-place-to-store-facts that are used in multiple places, feature, but it was pretty cheap to do once everything was in a central place.
    - lifeisstillgood 5y ago
      Is there a entry point for "how to make FOSS contributions to wiki-projects". I at the very least would be interested in finding out how all the pieces fit together, and maybe browsing the big list to see if anything was in my wheelhouse. Edit: And thank you for some insight into the back end of wikipedia - I did not know multiple languages was such a big push and it is such a sensible direction . Keep up the good work and keep pestering me for cash each year.
      - bawolff 5y ago
        https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hac... is probably what you are looking for. There's a very old architecture doc from 2012: https://www.mediawiki.org/wiki/Manual:MediaWiki_architecture https://www.mediawiki.org/wiki/Manual:MediaWiki_architecture its kind of outdated and missing a lot of the newer pieces. p.s. I used to work for the Wikimedia foundation about 1.5 years ago, but don't anymore. Prior to working there I was a volunteer for the FOSS project, and in theory I still am but I honestly don't do much anymore. My opinions are my own and don't represent WMF or anyone else.
    - j-pb 5y ago
      As if thowing the messy complexities of MySQL queries into the serene simplicity of static hypertext is somehow better. How often is numerical (which is the only knowledge you can really automate this way) knowledge updated? How often is said knowledge dependent on cultural idiosyncracies? How much effort is the code maintenance of the specialised editing system? How much effort is maintainance of the centralised infrastructure. How much effort is it really to change the data in multiple places, given that enough eyes every bug is shallow? What are the oppirtunity costs of not being able to properly distribute and share articles because they are suddenly tied to a very specific codebase? How much infrastructure cost do I have because I can't get volunteers to host my data? Overengineering solving the wrong problems at it's finest, no wonder the software has stayed this bad for decades now.
      - bawolff 5y ago
  - auggierose 5y ago
    Sounds like you have no clue what engineering is.
    - jbuhbjlnjbn 5y ago
      I wonder why you have nothing substantial to argue and feel the need to resort to ad hominem
  - nickik 5y ago
    As somebody who has used these facilities I highly disagree. They are really awesome. And if you have some magically technical solution that is much better and much easier, then please share it. And don't just complain that everybody else is doing everything wrong.

by j-pb 5y ago

Using RDF to store multiple languages of the same article is gross overengineering. Interlinking with other projects isn't nessecary when there are no other projects.

bawolff 5y ago

It isn't stored as RDF. RDF is provided as an export format. The language contents also aren't stored in wikidata, its just matching which title in language X matches which title in language Y. Wikidata isn't just used for interlanguage linking. Its also used to keep infoboxes synced across different languages, and a way to easily query that data (Depends a bit on the language, english wikipedia isn't fully on board with the system, but its been a big boon to smaller languages). As a result, people can update these "facts" independently of language, keeping them altogether more up to date, and it allows the data to be queried independently (via https://query.wikidata.org https://query.wikidata.org - try some of the example queries (button top left) if you haven't, the level of power SPARQL gives for querying this type of data set really is very cool). Part of Wikimedia's mission is to spread knowledge. Allowing querying of factual data with a query language like sparql helps advance that goal by letting people use extract the knowledge for new purposes.

j-pb 5y ago
> its just matching which title in language X matches which title in language Y Like I said, gross overengineering. "You can save cents and seconds of time updating knowledge boxes, by spending millions and hours developing federated SPARQL query capabilities!"
- bawolff 5y ago
  More like, we put the facts and knowledge in a central (mysql) db, so that instead of updating them in 300 places (Wikipedia has a lot of languages), you just update them in one place. And since the data happened to be there anyways, we added an RDF export feature, and threw that into a BlazeGraph DB so that people could do SPARQL queries if they want. The (not really federated) SPARQL queries have basically nothing to do with the central-place-to-store-facts that are used in multiple places, feature, but it was pretty cheap to do once everything was in a central place.
  - lifeisstillgood 5y ago
    Is there a entry point for "how to make FOSS contributions to wiki-projects". I at the very least would be interested in finding out how all the pieces fit together, and maybe browsing the big list to see if anything was in my wheelhouse. Edit: And thank you for some insight into the back end of wikipedia - I did not know multiple languages was such a big push and it is such a sensible direction . Keep up the good work and keep pestering me for cash each year.
    - bawolff 5y ago
      https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hac... is probably what you are looking for. There's a very old architecture doc from 2012: https://www.mediawiki.org/wiki/Manual:MediaWiki_architecture https://www.mediawiki.org/wiki/Manual:MediaWiki_architecture its kind of outdated and missing a lot of the newer pieces. p.s. I used to work for the Wikimedia foundation about 1.5 years ago, but don't anymore. Prior to working there I was a volunteer for the FOSS project, and in theory I still am but I honestly don't do much anymore. My opinions are my own and don't represent WMF or anyone else.
  - j-pb 5y ago
    As if thowing the messy complexities of MySQL queries into the serene simplicity of static hypertext is somehow better. How often is numerical (which is the only knowledge you can really automate this way) knowledge updated? How often is said knowledge dependent on cultural idiosyncracies? How much effort is the code maintenance of the specialised editing system? How much effort is maintainance of the centralised infrastructure. How much effort is it really to change the data in multiple places, given that enough eyes every bug is shallow? What are the oppirtunity costs of not being able to properly distribute and share articles because they are suddenly tied to a very specific codebase? How much infrastructure cost do I have because I can't get volunteers to host my data? Overengineering solving the wrong problems at it's finest, no wonder the software has stayed this bad for decades now.
    - bawolff 5y ago
- auggierose 5y ago
  Sounds like you have no clue what engineering is.
  - jbuhbjlnjbn 5y ago
    I wonder why you have nothing substantial to argue and feel the need to resort to ad hominem
- nickik 5y ago
  As somebody who has used these facilities I highly disagree. They are really awesome. And if you have some magically technical solution that is much better and much easier, then please share it. And don't just complain that everybody else is doing everything wrong.