5 ms·

I think I need an explainer for this. Is this like the number sequences library but for pure functions?

duskwuff 3y ago
The description is a little vague and hand-wavey. Here's a concrete example: A lot of Wikipedia sites have scripts embedded in the wikitext which automatically generate or transform information on a page, e.g. automatically performing unit conversions to generate text like "I would walk 500 miles (804.67 km)", performing date math to automatically generate and update a person's age based on their birthdate, or querying structured data from Wikidata [1] to display in an infobox. One example of these scripts is the {{convert}} [2] template on the English Wikipedia. Initially, these scripts were written in MediaWiki template logic [3], and were maintained individually on each wiki. This quickly proved unmaintainable, and some of those scripts were rewritten in Lua using the Scribunto extension [4], but these were still per-wiki, and there were frequently issues where different wikis would copy scripts from each other and introduce their own incompatible features. The WikiFunctions project is an attempt to centralize development of these scripts, much like how Wikimedia Commons [5] centralizes hosting of freely licensed images and other media. [1]: https://wikidata.org/ https://wikidata.org/ [2]: https://en.wikipedia.org/wiki/Template:Convert https://en.wikipedia.org/wiki/Template:Convert [3]: https://www.mediawiki.org/wiki/Help:Extension:ParserFunctions https://www.mediawiki.org/wiki/Help:Extension:ParserFunction... [4]: https://www.mediawiki.org/wiki/Extension:Scribunto https://www.mediawiki.org/wiki/Extension:Scribunto [5]: https://commons.wikimedia.org/ https://commons.wikimedia.org/
- hiAndrewQuinn 3y ago
  That makes way more sense to me, and my handful of edits and additions to Wiktionary's Finnish lexicon has definitely had me wondering how exactly those {{shortcode looking things}} actually worked. I'm seriously considering getting involved in this project now thanks to your explanation, that seems like it could be a very good thing indeed.
  - duskwuff 3y ago
    For a fairly horrifying example of how parser function templates looked, here's an old version of the Convert template from the English Wikipedia: https://en.wikipedia.org/w/index.php?title=Template:Convert&action=edit&oldid=577834328 https://en.wikipedia.org/w/index.php?title=Template:Convert&... This isn't even all of the code for the template, either; there were a bunch of subtemplates for individual operations. And now you probably have a decent idea of why Wikimedia wants to move away from that. :)
    - nmstoker 3y ago
      I've not kept up with the template complexity situation: do you have a sense for whether they're succeeding in simplifying things yet? (besides the benefit that Wikifunctions should bring)
      - starkparker 3y ago
        They aren't simplifying templates so much as moving them to a language better suited for the complexity. There are some similarly complex Lua modules that have rats' nests of dependencies and piles of undocumented code. Templates like {{Convert}} are also by and large unrelated to Wikifunctions, which are initially and primarily more concerned about lower-language natural language generation problems, like cross-language conjugation, that are relevant to Abstract Wikipedia. Or, in the Google.org evaluation I mention in a sibling: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation#Scribunto https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... > Instead of attempting to invent so much from scratch, Wikifunctions could have been based on Scribunto. We think this is a huge missed opportunity. One of the longest-standing requests from the Wikimedia community has been to provide a means of reusing code across different projects by centralizing common code on a global wiki. This is a tractable, well-scoped problem (at least in comparison with Wikifunctions as currently conceived). > “V1” of Wikifunctions could have been a new, central wiki for common Lua code. Structuring the development plan for Wikifunctions in this way would have meant a much shorter, surer path to delivering value to Wikimedians. It would have satisfied a long-standing community request, and (we suspect) earned the team quite a lot of goodwill. The team could then innovate on top of this initial version by building new graphical interfaces for producing and interacting with code, and by extending the capabilities of querying Wikidata. > We are not sure why this was overlooked, particularly in light of the project requirements described above. In conversations with the Abstract Wikipedia team, team members suggested to us that this option was considered and rejected because Scribunto is an expert system and that Lua is based on English, and that these properties made it incompatible with the Wikifunction goal of broad accessibility. We think this is a bad argument, for the following reasons: > - Lua has been successfully adopted by many projects whose primary language (as in, the language spoken by the users and developers) is not English. It is used successfully in hundreds of different language editions of Wikipedia. > - Wikifunctions can provide localized, graphical user interfaces that generate Lua code behind the scenes (see Blockly, for example). Lua can also be used as a “transpilation” target for text-based programming languages (there is precedent for this on Wikimedia wikis). As an intermediate representation, Lua would be far more efficient and robust than an ad-hoc function composition language. Which are all great points explicitly disregarded by Abstract Wikipedia for Wikifunctions.
- jessamyn 3y ago
  Thank you so much for this. I'm pretty Wiki-literate and was having a hard time understanding this from their initial pages.
- starkparker 3y ago
  It'd be an even better example if there was an equivalent conversion function on Wikifunctions to link to. There doesn't appear to be one yet. The project's response to the funding Google.org Fellows' evaluation is a deeper explanation that explicitly compares the goals of Wikifunctions to Scribunto (those Lua functions) — and is especially notable in the context of replacing templates and Lua modules since Google.org made a bunch of recommendations to further decouple Abstract Wikipedia's goals from Wikifunctions that Wikimedia thanked them for and subsequently ignored: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation_-_Answer https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... The catalogue of functions useful to Wikimedia is much more prosaic than the {{Convert}} template and in line with the Google.org examination — it's more about Abstract Wikipedia's goals of natural language generation (NLG): https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue The list of all functions is also similarly lower level; language transformations, form validation, sub/string operations: https://www.wikifunctions.org/wiki/Special:ListObjectsByType/Z8 https://www.wikifunctions.org/wiki/Special:ListObjectsByType...
- dragonwriter 3y ago
  > The description is a little vague and hand-wavey. Here's a concrete example: Much better description of what wikifunctions (the Wikimedia blog post seems more about where it fits in a broader strategy) is on wikifunctions about page: https://www.wikifunctions.org/wiki/Wikifunctions:About https://www.wikifunctions.org/wiki/Wikifunctions:About
haltist 3y ago
It is a formalization of knowledge with formal expressions. One could say it is a formalization of knowledge with executable/computational semantics. WikiMedia is actually an AI company and that's why they are always asking for donations.
dark-star 3y ago
Adding to what others have said, it contains functions that can be used to do grammatical transformations of nouns/verbs/... as in some languages those rules can be quite complicated. So having a central function that converts singular->plural for example is a huge win for maintainability

The description is a little vague and hand-wavey. Here's a concrete example: A lot of Wikipedia sites have scripts embedded in the wikitext which automatically generate or transform information on a page, e.g. automatically performing unit conversions to generate text like "I would walk 500 miles (804.67 km)", performing date math to automatically generate and update a person's age based on their birthdate, or querying structured data from Wikidata [1] to display in an infobox. One example of these scripts is the {{convert}} [2] template on the English Wikipedia. Initially, these scripts were written in MediaWiki template logic [3], and were maintained individually on each wiki. This quickly proved unmaintainable, and some of those scripts were rewritten in Lua using the Scribunto extension [4], but these were still per-wiki, and there were frequently issues where different wikis would copy scripts from each other and introduce their own incompatible features. The WikiFunctions project is an attempt to centralize development of these scripts, much like how Wikimedia Commons [5] centralizes hosting of freely licensed images and other media. [1]: https://wikidata.org/ https://wikidata.org/ [2]: https://en.wikipedia.org/wiki/Template:Convert https://en.wikipedia.org/wiki/Template:Convert [3]: https://www.mediawiki.org/wiki/Help:Extension:ParserFunctions https://www.mediawiki.org/wiki/Help:Extension:ParserFunction... [4]: https://www.mediawiki.org/wiki/Extension:Scribunto https://www.mediawiki.org/wiki/Extension:Scribunto [5]: https://commons.wikimedia.org/ https://commons.wikimedia.org/

hiAndrewQuinn 3y ago
That makes way more sense to me, and my handful of edits and additions to Wiktionary's Finnish lexicon has definitely had me wondering how exactly those {{shortcode looking things}} actually worked. I'm seriously considering getting involved in this project now thanks to your explanation, that seems like it could be a very good thing indeed.
- duskwuff 3y ago
  For a fairly horrifying example of how parser function templates looked, here's an old version of the Convert template from the English Wikipedia: https://en.wikipedia.org/w/index.php?title=Template:Convert&action=edit&oldid=577834328 https://en.wikipedia.org/w/index.php?title=Template:Convert&... This isn't even all of the code for the template, either; there were a bunch of subtemplates for individual operations. And now you probably have a decent idea of why Wikimedia wants to move away from that. :)
  - nmstoker 3y ago
    I've not kept up with the template complexity situation: do you have a sense for whether they're succeeding in simplifying things yet? (besides the benefit that Wikifunctions should bring)
    - starkparker 3y ago
      They aren't simplifying templates so much as moving them to a language better suited for the complexity. There are some similarly complex Lua modules that have rats' nests of dependencies and piles of undocumented code. Templates like {{Convert}} are also by and large unrelated to Wikifunctions, which are initially and primarily more concerned about lower-language natural language generation problems, like cross-language conjugation, that are relevant to Abstract Wikipedia. Or, in the Google.org evaluation I mention in a sibling: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation#Scribunto https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... > Instead of attempting to invent so much from scratch, Wikifunctions could have been based on Scribunto. We think this is a huge missed opportunity. One of the longest-standing requests from the Wikimedia community has been to provide a means of reusing code across different projects by centralizing common code on a global wiki. This is a tractable, well-scoped problem (at least in comparison with Wikifunctions as currently conceived). > “V1” of Wikifunctions could have been a new, central wiki for common Lua code. Structuring the development plan for Wikifunctions in this way would have meant a much shorter, surer path to delivering value to Wikimedians. It would have satisfied a long-standing community request, and (we suspect) earned the team quite a lot of goodwill. The team could then innovate on top of this initial version by building new graphical interfaces for producing and interacting with code, and by extending the capabilities of querying Wikidata. > We are not sure why this was overlooked, particularly in light of the project requirements described above. In conversations with the Abstract Wikipedia team, team members suggested to us that this option was considered and rejected because Scribunto is an expert system and that Lua is based on English, and that these properties made it incompatible with the Wikifunction goal of broad accessibility. We think this is a bad argument, for the following reasons: > - Lua has been successfully adopted by many projects whose primary language (as in, the language spoken by the users and developers) is not English. It is used successfully in hundreds of different language editions of Wikipedia. > - Wikifunctions can provide localized, graphical user interfaces that generate Lua code behind the scenes (see Blockly, for example). Lua can also be used as a “transpilation” target for text-based programming languages (there is precedent for this on Wikimedia wikis). As an intermediate representation, Lua would be far more efficient and robust than an ad-hoc function composition language. Which are all great points explicitly disregarded by Abstract Wikipedia for Wikifunctions.
jessamyn 3y ago
Thank you so much for this. I'm pretty Wiki-literate and was having a hard time understanding this from their initial pages.
starkparker 3y ago
It'd be an even better example if there was an equivalent conversion function on Wikifunctions to link to. There doesn't appear to be one yet. The project's response to the funding Google.org Fellows' evaluation is a deeper explanation that explicitly compares the goals of Wikifunctions to Scribunto (those Lua functions) — and is especially notable in the context of replacing templates and Lua modules since Google.org made a bunch of recommendations to further decouple Abstract Wikipedia's goals from Wikifunctions that Wikimedia thanked them for and subsequently ignored: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation_-_Answer https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... The catalogue of functions useful to Wikimedia is much more prosaic than the {{Convert}} template and in line with the Google.org examination — it's more about Abstract Wikipedia's goals of natural language generation (NLG): https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue The list of all functions is also similarly lower level; language transformations, form validation, sub/string operations: https://www.wikifunctions.org/wiki/Special:ListObjectsByType/Z8 https://www.wikifunctions.org/wiki/Special:ListObjectsByType...
dragonwriter 3y ago
> The description is a little vague and hand-wavey. Here's a concrete example: Much better description of what wikifunctions (the Wikimedia blog post seems more about where it fits in a broader strategy) is on wikifunctions about page: https://www.wikifunctions.org/wiki/Wikifunctions:About https://www.wikifunctions.org/wiki/Wikifunctions:About

haltist 3y ago

It is a formalization of knowledge with formal expressions. One could say it is a formalization of knowledge with executable/computational semantics. WikiMedia is actually an AI company and that's why they are always asking for donations.

dark-star 3y ago

Adding to what others have said, it contains functions that can be used to do grammatical transformations of nouns/verbs/... as in some languages those rules can be quite complicated. So having a central function that converts singular->plural for example is a huge win for maintainability