14 ms·

Wikifunctions

296 points by edward 3y ago

hiAndrewQuinn 3y ago
I think I need an explainer for this. Is this like the number sequences library but for pure functions?
- duskwuff 3y ago
  The description is a little vague and hand-wavey. Here's a concrete example: A lot of Wikipedia sites have scripts embedded in the wikitext which automatically generate or transform information on a page, e.g. automatically performing unit conversions to generate text like "I would walk 500 miles (804.67 km)", performing date math to automatically generate and update a person's age based on their birthdate, or querying structured data from Wikidata [1] to display in an infobox. One example of these scripts is the {{convert}} [2] template on the English Wikipedia. Initially, these scripts were written in MediaWiki template logic [3], and were maintained individually on each wiki. This quickly proved unmaintainable, and some of those scripts were rewritten in Lua using the Scribunto extension [4], but these were still per-wiki, and there were frequently issues where different wikis would copy scripts from each other and introduce their own incompatible features. The WikiFunctions project is an attempt to centralize development of these scripts, much like how Wikimedia Commons [5] centralizes hosting of freely licensed images and other media. [1]: https://wikidata.org/ https://wikidata.org/ [2]: https://en.wikipedia.org/wiki/Template:Convert https://en.wikipedia.org/wiki/Template:Convert [3]: https://www.mediawiki.org/wiki/Help:Extension:ParserFunctions https://www.mediawiki.org/wiki/Help:Extension:ParserFunction... [4]: https://www.mediawiki.org/wiki/Extension:Scribunto https://www.mediawiki.org/wiki/Extension:Scribunto [5]: https://commons.wikimedia.org/ https://commons.wikimedia.org/
  - hiAndrewQuinn 3y ago
    That makes way more sense to me, and my handful of edits and additions to Wiktionary's Finnish lexicon has definitely had me wondering how exactly those {{shortcode looking things}} actually worked. I'm seriously considering getting involved in this project now thanks to your explanation, that seems like it could be a very good thing indeed.
    - duskwuff 3y ago
      For a fairly horrifying example of how parser function templates looked, here's an old version of the Convert template from the English Wikipedia: https://en.wikipedia.org/w/index.php?title=Template:Convert&action=edit&oldid=577834328 https://en.wikipedia.org/w/index.php?title=Template:Convert&... This isn't even all of the code for the template, either; there were a bunch of subtemplates for individual operations. And now you probably have a decent idea of why Wikimedia wants to move away from that. :)
      - nmstoker 3y ago
        I've not kept up with the template complexity situation: do you have a sense for whether they're succeeding in simplifying things yet? (besides the benefit that Wikifunctions should bring)
        starkparker 3y ago
        They aren't simplifying templates so much as moving them to a language better suited for the complexity. There are some similarly complex Lua modules that have rats' nests of dependencies and piles of undocumented code. Templates like {{Convert}} are also by and large unrelated to Wikifunctions, which are initially and primarily more concerned about lower-language natural language generation problems, like cross-language conjugation, that are relevant to Abstract Wikipedia. Or, in the Google.org evaluation I mention in a sibling: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation#Scribunto https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... > Instead of attempting to invent so much from scratch, Wikifunctions could have been based on Scribunto. We think this is a huge missed opportunity. One of the longest-standing requests from the Wikimedia community has been to provide a means of reusing code across different projects by centralizing common code on a global wiki. This is a tractable, well-scoped problem (at least in comparison with Wikifunctions as currently conceived). > “V1” of Wikifunctions could have been a new, central wiki for common Lua code. Structuring the development plan for Wikifunctions in this way would have meant a much shorter, surer path to delivering value to Wikimedians. It would have satisfied a long-standing community request, and (we suspect) earned the team quite a lot of goodwill. The team could then innovate on top of this initial version by building new graphical interfaces for producing and interacting with code, and by extending the capabilities of querying Wikidata. > We are not sure why this was overlooked, particularly in light of the project requirements described above. In conversations with the Abstract Wikipedia team, team members suggested to us that this option was considered and rejected because Scribunto is an expert system and that Lua is based on English, and that these properties made it incompatible with the Wikifunction goal of broad accessibility. We think this is a bad argument, for the following reasons: > - Lua has been successfully adopted by many projects whose primary language (as in, the language spoken by the users and developers) is not English. It is used successfully in hundreds of different language editions of Wikipedia. > - Wikifunctions can provide localized, graphical user interfaces that generate Lua code behind the scenes (see Blockly, for example). Lua can also be used as a “transpilation” target for text-based programming languages (there is precedent for this on Wikimedia wikis). As an intermediate representation, Lua would be far more efficient and robust than an ad-hoc function composition language. Which are all great points explicitly disregarded by Abstract Wikipedia for Wikifunctions.
  - jessamyn 3y ago
    Thank you so much for this. I'm pretty Wiki-literate and was having a hard time understanding this from their initial pages.
  - starkparker 3y ago
    It'd be an even better example if there was an equivalent conversion function on Wikifunctions to link to. There doesn't appear to be one yet. The project's response to the funding Google.org Fellows' evaluation is a deeper explanation that explicitly compares the goals of Wikifunctions to Scribunto (those Lua functions) — and is especially notable in the context of replacing templates and Lua modules since Google.org made a bunch of recommendations to further decouple Abstract Wikipedia's goals from Wikifunctions that Wikimedia thanked them for and subsequently ignored: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation_-_Answer https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... The catalogue of functions useful to Wikimedia is much more prosaic than the {{Convert}} template and in line with the Google.org examination — it's more about Abstract Wikipedia's goals of natural language generation (NLG): https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue The list of all functions is also similarly lower level; language transformations, form validation, sub/string operations: https://www.wikifunctions.org/wiki/Special:ListObjectsByType/Z8 https://www.wikifunctions.org/wiki/Special:ListObjectsByType...
  - dragonwriter 3y ago
    > The description is a little vague and hand-wavey. Here's a concrete example: Much better description of what wikifunctions (the Wikimedia blog post seems more about where it fits in a broader strategy) is on wikifunctions about page: https://www.wikifunctions.org/wiki/Wikifunctions:About https://www.wikifunctions.org/wiki/Wikifunctions:About
- haltist 3y ago
  It is a formalization of knowledge with formal expressions. One could say it is a formalization of knowledge with executable/computational semantics. WikiMedia is actually an AI company and that's why they are always asking for donations.
- dark-star 3y ago
  Adding to what others have said, it contains functions that can be used to do grammatical transformations of nouns/verbs/... as in some languages those rules can be quite complicated. So having a central function that converts singular->plural for example is a huge win for maintainability
hlandau 3y ago
[flagged]
- duskwuff 3y ago
  Then there's something wrong with your browser. This is a fairly standard Wordpress site and the content is all present as perfectly standard HTML.
  - hlandau 3y ago
    No, I'm referring to each Wikifunction page on the Wikifunctions site itself.
    - themikesanto 3y ago
      How do you propose the functions should run if you don't have JavaScript enabled?
      - hlandau 3y ago
        I don't want to run the functions, I want to view them. Also the functions execute on the server anyway (plus JavaScript isn't even the only supported language), so execution should also be possible without a browser having JavaScript enabled.
      - simonw 3y ago
        They are server side functions, not client side.
- gngoo 3y ago
  C’mon it’s 2023.
  - MrStonedOne 3y ago
    [dead]
  - 3y ago
solarpunk 3y ago
Gonna dig deeper, but I get the sense Wikipedia is preparing a native dataformat for LLM ingestion.
- duskwuff 3y ago
  You're probably thinking of something more along the lines of Wikidata [1], which is over 10 years old. [1]: https://www.wikidata.org/wiki/Wikidata:Main_Page https://www.wikidata.org/wiki/Wikidata:Main_Page
  - starkparker 3y ago
    It's very much designed for and around Wikidata. > Wikifunctions will allow easy access to large knowledge bases such as Wikidata, but also to binary input and output files https://diff.wikimedia.org/2023/08/07/wikifunctions-is-starting-up/ https://diff.wikimedia.org/2023/08/07/wikifunctions-is-start... > Using the simple facts housed in Wikidata, you will be able to write functions that make calculations, provide a person’s age, estimate population densities, and more, and integrate the results into Wikipedia. https://www.wikifunctions.org/wiki/Wikifunctions:FAQ https://www.wikifunctions.org/wiki/Wikifunctions:FAQ > In the future: > - It will be possible to call Wikifunctions functions from other Wikimedia projects, and integrate their results in the output of the page. > - It will be possible to use data from Wikidata in functions.
    - zozbot234 3y ago
      Wikidata cannot express the "semantic" content of average encyclopedic text, which is an express goal of Wikifunctions. So they will have to expand the data model quite a bit compared to what Wikidata has today. (This can be done quite cleanly though, since the whole point of RDF is to be able to express general graphs, and this aspect has been made even stronger with RDF* which aligns fairly well with the "frame" semantics Wikifunctions plans to use for the above purpose.)
      - starkparker 3y ago
starkparker 3y ago
Previously: https://news.ycombinator.com/item?id=36927695 https://news.ycombinator.com/item?id=36927695 cf. https://www.wikifunctions.org/wiki/Wikifunctions:About https://www.wikifunctions.org/wiki/Wikifunctions:About > Wikifunctions is a Wikimedia project for everyone to collaboratively create and maintain a library of code functions to support the Wikimedia projects and beyond, for everyone to call and re-use in the world's natural and programming languages. It's a support project of the Abstract Wikipedia initiative to model facts with data not specific to any spoken or written language, building on and supporting things like Wikidata.
- smsm42 3y ago
  It sounds as some kind of anti-Sapir-Whorf hypothesis - that we can express things entirely without using any language at all. I am very curious how it turns out - while I think Wikidata is taking it as far as it practically possible, and Abstract Wikipedia takes it way further - I'd be very curious to know how it turns out. Sometimes people just need to do things that look crazy to see what happens.
  - jmopp 3y ago
    You don't need language to do maths — the Greeks went pretty far with geometry alone. While language makes it easier to understand things, it's not strictly necessary: you can learn a lot from mimicry, for instance.
- dang 3y ago
  Thanks! Macroexpanded: Welcome to Wikifunctions - https://news.ycombinator.com/item?id=36927695 https://news.ycombinator.com/item?id=36927695 - July 2023 (163 comments)
- account42 3y ago
  I don't really understand how this is different from https://rosettacode.org/ https://rosettacode.org/
simonw 3y ago
Here's an example function which shows that they can have multiple implementations (click the Details tab) - in this case there's an implementation in Python and one in JavaScript: https://www.wikifunctions.org/view/en/Z10070 https://www.wikifunctions.org/view/en/Z10070 And a much more interesting example which helps illustrate why Wikipedia built this: "genitive case of Bangla word in Python": https://www.wikifunctions.org/view/en/Z10594 https://www.wikifunctions.org/view/en/Z10594
- pantalaimon 3y ago
  All the functions are numbered and call each other. substring_exists(haystack, needle) is implemented as function Z10070( Z10070K1, Z10070K2 ) { return Z10070K1.includes(Z10070K2); } This is atrocious! Might as well come straight out of a decompiler.
  - starkparker 3y ago
    And an explicit goal of the project: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation_-_Answer https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... > In particular, according to Keet, Grammatical Framework notably lacks the flexibility to handle certain aspects of Niger-Congo B languages’ morphology. The traditional answer to this kind of critique would be to say, “Let’s just organize an effort to work on Grammatical Framework.” The design philosophy behind Abstract Wikipedia is to make as few assumptions about a contributor’s facility with English and programming experience as possible. Organized efforts around Grammatical Framework (or otherwise) are not a bad idea at all. They may well solve some problems for some languages. However, the contributor pools for under-resourced languages are already small. Demanding expertise with specific natural languages like English and also specific programming paradigms contracts those pools still further. and > A solution designed by a small group of Westerners is likely to produce a system that replicates the trends of an imperialist English-focused Western-thinking industry. Existing tools tell a one-voice story; they are built by the same people (in a socio-cultural sense) and produce the same outcome, which is crafted to the needs of their creators (or by the limits of their understanding). Tools are then used to build tools, which will be again used to build more tools; step by step, every level of architectural decision-making limits more and more the space that can be benefitted by these efforts. and > The evaluation acknowledges little to no value to the effort that Wikifunctions is investing in being multilingual in terms of natural languages, even for implementations. It is unclear as to how a solution that is built on top of (the existing Lua-based Scribunto scripting system) would succeed in being multilingual, and not just be an English project which the other languages may be invited to use, if they understand it, but would be mostly blocked from contributing to.
anigbrowl 3y ago
I like the idea but the catalog has the look of something that will quickly grow unmanageable. I hope it doesn't end up like other Wiki projects that die from lack of momentum/direction like wikinews. To be clear, the latter still runs but hardly anyone uses it. https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue Does this have the potential to become some standard reference for code proving/transpilation purposes, or is it likely to end up like past attempts to unify all programming languages?
shpx 3y ago
Awful interface. I have to click three times and scroll to see code in a tiny font, the name of the function is repeated on the page like 7 times.
slugiscool99 3y ago
Needs some UI work but this is a really great idea. Hope it gains traction
noirscape 3y ago
Yet another Wikimedia project with way too lofty ambitions... this is what Wikimedia donation money gets spent on rather than the operation of Wikipedia. For reference, the greater project this is a part of has been in development since 2013[0] and seems to be an attempt to basically remove the editor from Wikipedia. For the uninitiated - this is part of an attempt to basically turn WikiData[1] into article text. From what I understand, the idea behind this is to create a bunch of text rendering functions to make it easier to provide automatic translations of rendered text. Don't get me wrong, the idea here is neat, but from what I understand the Wikipedia community has been having far greater needs not being met rather than some lofty goal to drive even more editors out of the site and replacing them with data entry drones. It's a bit too often that you hear about some ailing extension and the WMF response being "no budget for it", while a lot of it gets spend on projects like this. [0]: https://en.wikipedia.org/wiki/Abstract_Wikipedia https://en.wikipedia.org/wiki/Abstract_Wikipedia [1]: https://www.wikidata.org/wiki/Wikidata:Main_Page https://www.wikidata.org/wiki/Wikidata:Main_Page - basically, Wikipedia data meant to be in a machine-readable format.
- simonw 3y ago
  What are some of those greater needs that are not being met?
  - noirscape 3y ago
    It's been a hot minute since I last looked at the most recent list, but the short of it is that the WMF likes to release extensions onto Wikipedias that ship with a ton of technical bugs, aren't tested beforehand by the Wikipedia community for usability (which is ironic given the WMFs obsession with making tools as accessible as possible) and when the complaints get leveled, the response from the WMF basically ends up being "yeah so there's no budget for that". Rinse and repeat for a ton of tiny technical issues that affect all their extensions on the regular. It's a bit odd to be putting so many engineers and money onto lofty projects when the maintenance of Wikimedia extensions is in such an extremely precarious state already. The most evergreen example of this is and probably always will be the rollout of VisualEditor, their WYSIWG editor. It was completely unwanted, was proven to be not ready for general use (due to all the problems that plague "combining a WYSIWG editor with a code view"), yet forcibly deployed on all Wikipedia languages without much room for warning or feedback. After that, the WMF promised communities the option to opt-out if they didn't want it... before violating it on the German Wikipedia because they voted to not roll it out. To my knowledge, VisualEditor hasn't improved over the years (beyond "rewrite Parsoid into PHP"), but editors have just gritted their teeth. That's the extreme example, but the WMFs relationship with building features for Wikipedia is... pretty much in a vacuum from what users want. A lot of their projects seem to be conceived by WMF managers first, without consultation to their existing community of volunteers. Which is I guess fine for moonshot projects like Wikifunctions (since they don't involve the volunteers before release) but results in a lot of friction when a managers pet project is forcibly rolled out on all language Wikipedias. Very few of their big extension deployments have gone smoothly with the community. I distinctly remember the Minerva skin (mobile Wikipedia) needing tons of changes and improvements before it was usable as well, after it was already rolled out and caused a ton of negative fedback.
deleted 3y ago
[deleted]
primitivesuave 3y ago
Terrible UI. I found two example functions (call them `f` and `g`), found the dropdown to change the input type of `f` from "string" to "function call", and even manage to get `g` suggested when I enter it's unique identifier - but cannot set up any configuration to compute `f(g(x))` (even after exhaustively trying all options). It seems like it should be a core functionality of such a project to easily set up complex computations as function compositions.
- zeroimpl 3y ago
  Probably gotta define h(x) = f(g(x)) and then invoke h. Or they should build a REPL or Fiddle interface to let you test arbitrary expressions.
faresahmed 3y ago
Amazing to see RustPython being used in the wild. I wonder why they are using it. Is it easier to sandbox? smaller memory footprint than CPytohn and PyPy? just because it's cool?
- dhx 3y ago
  I gather this might be changing per [1] as Wikifunctions is investigating the possibility of implementing interpreters such as CPython with WebAssembly. [1] https://phabricator.wikimedia.org/T308250 https://phabricator.wikimedia.org/T308250
- deleted 3y ago
  [deleted]
13415 3y ago
I'm kind of disappointed. Based on the name, I thought it's a searchable collection of functions, a mixture of OEIS, searchonmath, and a classic formula encyclopedia. But it seems to be only about little algorithmic snippets without explanation or definition?
- extraduder_ire 3y ago
  The details page for each function seems to let you look at the implementations of that function. I expect more interesting ones to show up in the coming months or years.
tropsis 3y ago
Cool project! I hope they improve their UI.
culi 3y ago
Is it me or does it not seem very well thought out? Every example I've seen only has implementations in JavaScript and/or Python (without any mention of which version of this language the code is meant to run in. A critical piece of info for certain languages). I haven't seen any other languages nor a way to search by language. What a "string" means in one language can be completely different in another language. The primitive data types that the project assumes are not really supported across all programming languages. Also if anyone hasn't already seen them, similar projects already exist and are more complete. E.g. * https://rosettacode.org/ https://rosettacode.org/ * https://programming-idioms.org/ https://programming-idioms.org/ * https://the-algorithms.com/ https://the-algorithms.com/ Not to mention LeetCode, CodeWars, Project Euler, Exercism can kinda serve the same role.
- willdr 3y ago
  Wikimedia Foundation and letting their main product languish while they pursue badly-thought-out side projects? Name a more iconic duo.
  - windwindow 3y ago
    I'd argue that sister projects such as Wikidata, Wikiversity, etc, are greatly useful and that the solution to improving them is not to neglect them, but to provide more resources to them.
    - willdr 3y ago
      Maybe so, but Wikipedia itself is fraying at the seams these days. Less and less Administrators every year, less and less committed editors, and more and more tech rot. See this comment https://news.ycombinator.com/item?id=38550441 https://news.ycombinator.com/item?id=38550441 for more context.
    - mardifoufs 3y ago
      Im not super active in the wiki community anymore but last time I heard it being discussed, wikiuniversity seemed like something that caused more issues than it solved for wiki admins. Is that not the case anymore? Did it get better?
extraduder_ire 3y ago
Very cool. How long until someone writes PyPi/npm packages to import/use these functions by Z number? Speaking of which, are there already packages to access wikidata by objects' Q numbers? The two could work together quite well, with a bit of caching.
aaron695 3y ago
[dead]
giovannibonetti 3y ago
Related: Hoogle [1] a function search index for the Haskell language. It searches functions by signature. [1] https://hoogle.haskell.org/ https://hoogle.haskell.org/
CJefferson 3y ago
Python is an awful choice of language for this (although I understand why they used it, it's so popular). What do they mean by 'Python'? Most programming languages make considerable effort in supporting old versions (gcc has c90 and c++98 for example, fortran goes even earlier). With Java, they make very strong efforts to support older code (as long as it uses the library which are labelled as stable). Old javascript code also tends to work well, as there are lots of old websites that don't want breaking. Python on the other hand has gone out of it's way to nuke Python 2, and many new version of Python 3 deletes something from the standard library.
- JCharante 3y ago
  Python 3 came out 15 years ago. Are we still really having this discussion?
  - CJefferson 3y ago
    New Python versions do continue to remove features from the standard library, which can break programs. Most other languages I'm familiar with make efforts to only do this in extreme cases (like c removed gets, but basically nothing else). Also, I have Python 2 programs I'm keeping running, and expect to continue doing so for several years to come. Many libraries never made the jump to Python 3.
- deleted 3y ago
  [deleted]
pyromaker 3y ago
Will be cool if these functions were available via API endpoints :)
laserbeam 3y ago
Mixed feelings about the longevity of this. It feels like it wants to be an open source alternative to WolframAlpha, but without the UX of "just typing a question". One has to navigate and search for functions that do the thing they want done. Wanna convert a string to the nato phonetic alphabet? Sure, that function is there... You gotta search for it first though. Does it accept lowercase inputs? No, not at the time of writing this comment... But you know, someone might fix that. No problem. It sounds like a project that would be dead on arrival if it were launched by and for any other community. If wiki maintainers like this? Then great! I do see value if people could integrate functions within wikipedia article. Maybe it makes sense for certain articles to include some dynamic widgets in them. Unsure I'd like that but it may work.
trenchgun 3y ago
So this is something like Rosetta stone, but also has features or Unison language

Wikifunctions

296 points by edward 3y ago

hiAndrewQuinn 3y ago

I think I need an explainer for this. Is this like the number sequences library but for pure functions?

duskwuff 3y ago
The description is a little vague and hand-wavey. Here's a concrete example: A lot of Wikipedia sites have scripts embedded in the wikitext which automatically generate or transform information on a page, e.g. automatically performing unit conversions to generate text like "I would walk 500 miles (804.67 km)", performing date math to automatically generate and update a person's age based on their birthdate, or querying structured data from Wikidata [1] to display in an infobox. One example of these scripts is the {{convert}} [2] template on the English Wikipedia. Initially, these scripts were written in MediaWiki template logic [3], and were maintained individually on each wiki. This quickly proved unmaintainable, and some of those scripts were rewritten in Lua using the Scribunto extension [4], but these were still per-wiki, and there were frequently issues where different wikis would copy scripts from each other and introduce their own incompatible features. The WikiFunctions project is an attempt to centralize development of these scripts, much like how Wikimedia Commons [5] centralizes hosting of freely licensed images and other media. [1]: https://wikidata.org/ https://wikidata.org/ [2]: https://en.wikipedia.org/wiki/Template:Convert https://en.wikipedia.org/wiki/Template:Convert [3]: https://www.mediawiki.org/wiki/Help:Extension:ParserFunctions https://www.mediawiki.org/wiki/Help:Extension:ParserFunction... [4]: https://www.mediawiki.org/wiki/Extension:Scribunto https://www.mediawiki.org/wiki/Extension:Scribunto [5]: https://commons.wikimedia.org/ https://commons.wikimedia.org/
- hiAndrewQuinn 3y ago
  That makes way more sense to me, and my handful of edits and additions to Wiktionary's Finnish lexicon has definitely had me wondering how exactly those {{shortcode looking things}} actually worked. I'm seriously considering getting involved in this project now thanks to your explanation, that seems like it could be a very good thing indeed.
  - duskwuff 3y ago
    For a fairly horrifying example of how parser function templates looked, here's an old version of the Convert template from the English Wikipedia: https://en.wikipedia.org/w/index.php?title=Template:Convert&action=edit&oldid=577834328 https://en.wikipedia.org/w/index.php?title=Template:Convert&... This isn't even all of the code for the template, either; there were a bunch of subtemplates for individual operations. And now you probably have a decent idea of why Wikimedia wants to move away from that. :)
    - nmstoker 3y ago
      I've not kept up with the template complexity situation: do you have a sense for whether they're succeeding in simplifying things yet? (besides the benefit that Wikifunctions should bring)
      - starkparker 3y ago
        They aren't simplifying templates so much as moving them to a language better suited for the complexity. There are some similarly complex Lua modules that have rats' nests of dependencies and piles of undocumented code. Templates like {{Convert}} are also by and large unrelated to Wikifunctions, which are initially and primarily more concerned about lower-language natural language generation problems, like cross-language conjugation, that are relevant to Abstract Wikipedia. Or, in the Google.org evaluation I mention in a sibling: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation#Scribunto https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... > Instead of attempting to invent so much from scratch, Wikifunctions could have been based on Scribunto. We think this is a huge missed opportunity. One of the longest-standing requests from the Wikimedia community has been to provide a means of reusing code across different projects by centralizing common code on a global wiki. This is a tractable, well-scoped problem (at least in comparison with Wikifunctions as currently conceived). > “V1” of Wikifunctions could have been a new, central wiki for common Lua code. Structuring the development plan for Wikifunctions in this way would have meant a much shorter, surer path to delivering value to Wikimedians. It would have satisfied a long-standing community request, and (we suspect) earned the team quite a lot of goodwill. The team could then innovate on top of this initial version by building new graphical interfaces for producing and interacting with code, and by extending the capabilities of querying Wikidata. > We are not sure why this was overlooked, particularly in light of the project requirements described above. In conversations with the Abstract Wikipedia team, team members suggested to us that this option was considered and rejected because Scribunto is an expert system and that Lua is based on English, and that these properties made it incompatible with the Wikifunction goal of broad accessibility. We think this is a bad argument, for the following reasons: > - Lua has been successfully adopted by many projects whose primary language (as in, the language spoken by the users and developers) is not English. It is used successfully in hundreds of different language editions of Wikipedia. > - Wikifunctions can provide localized, graphical user interfaces that generate Lua code behind the scenes (see Blockly, for example). Lua can also be used as a “transpilation” target for text-based programming languages (there is precedent for this on Wikimedia wikis). As an intermediate representation, Lua would be far more efficient and robust than an ad-hoc function composition language. Which are all great points explicitly disregarded by Abstract Wikipedia for Wikifunctions.
- jessamyn 3y ago
  Thank you so much for this. I'm pretty Wiki-literate and was having a hard time understanding this from their initial pages.
- starkparker 3y ago
  It'd be an even better example if there was an equivalent conversion function on Wikifunctions to link to. There doesn't appear to be one yet. The project's response to the funding Google.org Fellows' evaluation is a deeper explanation that explicitly compares the goals of Wikifunctions to Scribunto (those Lua functions) — and is especially notable in the context of replacing templates and Lua modules since Google.org made a bunch of recommendations to further decouple Abstract Wikipedia's goals from Wikifunctions that Wikimedia thanked them for and subsequently ignored: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation_-_Answer https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... The catalogue of functions useful to Wikimedia is much more prosaic than the {{Convert}} template and in line with the Google.org examination — it's more about Abstract Wikipedia's goals of natural language generation (NLG): https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue The list of all functions is also similarly lower level; language transformations, form validation, sub/string operations: https://www.wikifunctions.org/wiki/Special:ListObjectsByType/Z8 https://www.wikifunctions.org/wiki/Special:ListObjectsByType...
- dragonwriter 3y ago
  > The description is a little vague and hand-wavey. Here's a concrete example: Much better description of what wikifunctions (the Wikimedia blog post seems more about where it fits in a broader strategy) is on wikifunctions about page: https://www.wikifunctions.org/wiki/Wikifunctions:About https://www.wikifunctions.org/wiki/Wikifunctions:About
haltist 3y ago
It is a formalization of knowledge with formal expressions. One could say it is a formalization of knowledge with executable/computational semantics. WikiMedia is actually an AI company and that's why they are always asking for donations.
dark-star 3y ago
Adding to what others have said, it contains functions that can be used to do grammatical transformations of nouns/verbs/... as in some languages those rules can be quite complicated. So having a central function that converts singular->plural for example is a huge win for maintainability

hlandau 3y ago

[flagged]

duskwuff 3y ago
Then there's something wrong with your browser. This is a fairly standard Wordpress site and the content is all present as perfectly standard HTML.
- hlandau 3y ago
  No, I'm referring to each Wikifunction page on the Wikifunctions site itself.
  - themikesanto 3y ago
    How do you propose the functions should run if you don't have JavaScript enabled?
    - hlandau 3y ago
      I don't want to run the functions, I want to view them. Also the functions execute on the server anyway (plus JavaScript isn't even the only supported language), so execution should also be possible without a browser having JavaScript enabled.
    - simonw 3y ago
      They are server side functions, not client side.
gngoo 3y ago
C’mon it’s 2023.
- MrStonedOne 3y ago
  [dead]
- 3y ago

solarpunk 3y ago

Gonna dig deeper, but I get the sense Wikipedia is preparing a native dataformat for LLM ingestion.

duskwuff 3y ago
You're probably thinking of something more along the lines of Wikidata [1], which is over 10 years old. [1]: https://www.wikidata.org/wiki/Wikidata:Main_Page https://www.wikidata.org/wiki/Wikidata:Main_Page
- starkparker 3y ago
  It's very much designed for and around Wikidata. > Wikifunctions will allow easy access to large knowledge bases such as Wikidata, but also to binary input and output files https://diff.wikimedia.org/2023/08/07/wikifunctions-is-starting-up/ https://diff.wikimedia.org/2023/08/07/wikifunctions-is-start... > Using the simple facts housed in Wikidata, you will be able to write functions that make calculations, provide a person’s age, estimate population densities, and more, and integrate the results into Wikipedia. https://www.wikifunctions.org/wiki/Wikifunctions:FAQ https://www.wikifunctions.org/wiki/Wikifunctions:FAQ > In the future: > - It will be possible to call Wikifunctions functions from other Wikimedia projects, and integrate their results in the output of the page. > - It will be possible to use data from Wikidata in functions.
  - zozbot234 3y ago
    Wikidata cannot express the "semantic" content of average encyclopedic text, which is an express goal of Wikifunctions. So they will have to expand the data model quite a bit compared to what Wikidata has today. (This can be done quite cleanly though, since the whole point of RDF is to be able to express general graphs, and this aspect has been made even stronger with RDF* which aligns fairly well with the "frame" semantics Wikifunctions plans to use for the above purpose.)
    - starkparker 3y ago

starkparker 3y ago

Previously: https://news.ycombinator.com/item?id=36927695 https://news.ycombinator.com/item?id=36927695 cf. https://www.wikifunctions.org/wiki/Wikifunctions:About https://www.wikifunctions.org/wiki/Wikifunctions:About > Wikifunctions is a Wikimedia project for everyone to collaboratively create and maintain a library of code functions to support the Wikimedia projects and beyond, for everyone to call and re-use in the world's natural and programming languages. It's a support project of the Abstract Wikipedia initiative to model facts with data not specific to any spoken or written language, building on and supporting things like Wikidata.

smsm42 3y ago
It sounds as some kind of anti-Sapir-Whorf hypothesis - that we can express things entirely without using any language at all. I am very curious how it turns out - while I think Wikidata is taking it as far as it practically possible, and Abstract Wikipedia takes it way further - I'd be very curious to know how it turns out. Sometimes people just need to do things that look crazy to see what happens.
- jmopp 3y ago
  You don't need language to do maths — the Greeks went pretty far with geometry alone. While language makes it easier to understand things, it's not strictly necessary: you can learn a lot from mimicry, for instance.
dang 3y ago
Thanks! Macroexpanded: Welcome to Wikifunctions - https://news.ycombinator.com/item?id=36927695 https://news.ycombinator.com/item?id=36927695 - July 2023 (163 comments)
account42 3y ago
I don't really understand how this is different from https://rosettacode.org/ https://rosettacode.org/

simonw 3y ago

Here's an example function which shows that they can have multiple implementations (click the Details tab) - in this case there's an implementation in Python and one in JavaScript: https://www.wikifunctions.org/view/en/Z10070 https://www.wikifunctions.org/view/en/Z10070 And a much more interesting example which helps illustrate why Wikipedia built this: "genitive case of Bangla word in Python": https://www.wikifunctions.org/view/en/Z10594 https://www.wikifunctions.org/view/en/Z10594

pantalaimon 3y ago
All the functions are numbered and call each other. substring_exists(haystack, needle) is implemented as function Z10070( Z10070K1, Z10070K2 ) { return Z10070K1.includes(Z10070K2); } This is atrocious! Might as well come straight out of a decompiler.
- starkparker 3y ago
  And an explicit goal of the project: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evaluation_-_Answer https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.or... > In particular, according to Keet, Grammatical Framework notably lacks the flexibility to handle certain aspects of Niger-Congo B languages’ morphology. The traditional answer to this kind of critique would be to say, “Let’s just organize an effort to work on Grammatical Framework.” The design philosophy behind Abstract Wikipedia is to make as few assumptions about a contributor’s facility with English and programming experience as possible. Organized efforts around Grammatical Framework (or otherwise) are not a bad idea at all. They may well solve some problems for some languages. However, the contributor pools for under-resourced languages are already small. Demanding expertise with specific natural languages like English and also specific programming paradigms contracts those pools still further. and > A solution designed by a small group of Westerners is likely to produce a system that replicates the trends of an imperialist English-focused Western-thinking industry. Existing tools tell a one-voice story; they are built by the same people (in a socio-cultural sense) and produce the same outcome, which is crafted to the needs of their creators (or by the limits of their understanding). Tools are then used to build tools, which will be again used to build more tools; step by step, every level of architectural decision-making limits more and more the space that can be benefitted by these efforts. and > The evaluation acknowledges little to no value to the effort that Wikifunctions is investing in being multilingual in terms of natural languages, even for implementations. It is unclear as to how a solution that is built on top of (the existing Lua-based Scribunto scripting system) would succeed in being multilingual, and not just be an English project which the other languages may be invited to use, if they understand it, but would be mostly blocked from contributing to.

anigbrowl 3y ago

I like the idea but the catalog has the look of something that will quickly grow unmanageable. I hope it doesn't end up like other Wiki projects that die from lack of momentum/direction like wikinews. To be clear, the latter still runs but hardly anyone uses it. https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue Does this have the potential to become some standard reference for code proving/transpilation purposes, or is it likely to end up like past attempts to unify all programming languages?

shpx 3y ago

Awful interface. I have to click three times and scroll to see code in a tiny font, the name of the function is repeated on the page like 7 times.

slugiscool99 3y ago

Needs some UI work but this is a really great idea. Hope it gains traction

noirscape 3y ago

Yet another Wikimedia project with way too lofty ambitions... this is what Wikimedia donation money gets spent on rather than the operation of Wikipedia. For reference, the greater project this is a part of has been in development since 2013[0] and seems to be an attempt to basically remove the editor from Wikipedia. For the uninitiated - this is part of an attempt to basically turn WikiData[1] into article text. From what I understand, the idea behind this is to create a bunch of text rendering functions to make it easier to provide automatic translations of rendered text. Don't get me wrong, the idea here is neat, but from what I understand the Wikipedia community has been having far greater needs not being met rather than some lofty goal to drive even more editors out of the site and replacing them with data entry drones. It's a bit too often that you hear about some ailing extension and the WMF response being "no budget for it", while a lot of it gets spend on projects like this. [0]: https://en.wikipedia.org/wiki/Abstract_Wikipedia https://en.wikipedia.org/wiki/Abstract_Wikipedia [1]: https://www.wikidata.org/wiki/Wikidata:Main_Page https://www.wikidata.org/wiki/Wikidata:Main_Page - basically, Wikipedia data meant to be in a machine-readable format.

simonw 3y ago
What are some of those greater needs that are not being met?
- noirscape 3y ago
  It's been a hot minute since I last looked at the most recent list, but the short of it is that the WMF likes to release extensions onto Wikipedias that ship with a ton of technical bugs, aren't tested beforehand by the Wikipedia community for usability (which is ironic given the WMFs obsession with making tools as accessible as possible) and when the complaints get leveled, the response from the WMF basically ends up being "yeah so there's no budget for that". Rinse and repeat for a ton of tiny technical issues that affect all their extensions on the regular. It's a bit odd to be putting so many engineers and money onto lofty projects when the maintenance of Wikimedia extensions is in such an extremely precarious state already. The most evergreen example of this is and probably always will be the rollout of VisualEditor, their WYSIWG editor. It was completely unwanted, was proven to be not ready for general use (due to all the problems that plague "combining a WYSIWG editor with a code view"), yet forcibly deployed on all Wikipedia languages without much room for warning or feedback. After that, the WMF promised communities the option to opt-out if they didn't want it... before violating it on the German Wikipedia because they voted to not roll it out. To my knowledge, VisualEditor hasn't improved over the years (beyond "rewrite Parsoid into PHP"), but editors have just gritted their teeth. That's the extreme example, but the WMFs relationship with building features for Wikipedia is... pretty much in a vacuum from what users want. A lot of their projects seem to be conceived by WMF managers first, without consultation to their existing community of volunteers. Which is I guess fine for moonshot projects like Wikifunctions (since they don't involve the volunteers before release) but results in a lot of friction when a managers pet project is forcibly rolled out on all language Wikipedias. Very few of their big extension deployments have gone smoothly with the community. I distinctly remember the Minerva skin (mobile Wikipedia) needing tons of changes and improvements before it was usable as well, after it was already rolled out and caused a ton of negative fedback.

deleted 3y ago

[deleted]

primitivesuave 3y ago

Terrible UI. I found two example functions (call them `f` and `g`), found the dropdown to change the input type of `f` from "string" to "function call", and even manage to get `g` suggested when I enter it's unique identifier - but cannot set up any configuration to compute `f(g(x))` (even after exhaustively trying all options). It seems like it should be a core functionality of such a project to easily set up complex computations as function compositions.

zeroimpl 3y ago
Probably gotta define h(x) = f(g(x)) and then invoke h. Or they should build a REPL or Fiddle interface to let you test arbitrary expressions.

faresahmed 3y ago

Amazing to see RustPython being used in the wild. I wonder why they are using it. Is it easier to sandbox? smaller memory footprint than CPytohn and PyPy? just because it's cool?

dhx 3y ago
I gather this might be changing per [1] as Wikifunctions is investigating the possibility of implementing interpreters such as CPython with WebAssembly. [1] https://phabricator.wikimedia.org/T308250 https://phabricator.wikimedia.org/T308250
deleted 3y ago
[deleted]

13415 3y ago

I'm kind of disappointed. Based on the name, I thought it's a searchable collection of functions, a mixture of OEIS, searchonmath, and a classic formula encyclopedia. But it seems to be only about little algorithmic snippets without explanation or definition?

extraduder_ire 3y ago
The details page for each function seems to let you look at the implementations of that function. I expect more interesting ones to show up in the coming months or years.

tropsis 3y ago

Cool project! I hope they improve their UI.

culi 3y ago

Is it me or does it not seem very well thought out? Every example I've seen only has implementations in JavaScript and/or Python (without any mention of which version of this language the code is meant to run in. A critical piece of info for certain languages). I haven't seen any other languages nor a way to search by language. What a "string" means in one language can be completely different in another language. The primitive data types that the project assumes are not really supported across all programming languages. Also if anyone hasn't already seen them, similar projects already exist and are more complete. E.g. * https://rosettacode.org/ https://rosettacode.org/ * https://programming-idioms.org/ https://programming-idioms.org/ * https://the-algorithms.com/ https://the-algorithms.com/ Not to mention LeetCode, CodeWars, Project Euler, Exercism can kinda serve the same role.

willdr 3y ago
Wikimedia Foundation and letting their main product languish while they pursue badly-thought-out side projects? Name a more iconic duo.
- windwindow 3y ago
  I'd argue that sister projects such as Wikidata, Wikiversity, etc, are greatly useful and that the solution to improving them is not to neglect them, but to provide more resources to them.
  - willdr 3y ago
    Maybe so, but Wikipedia itself is fraying at the seams these days. Less and less Administrators every year, less and less committed editors, and more and more tech rot. See this comment https://news.ycombinator.com/item?id=38550441 https://news.ycombinator.com/item?id=38550441 for more context.
  - mardifoufs 3y ago
    Im not super active in the wiki community anymore but last time I heard it being discussed, wikiuniversity seemed like something that caused more issues than it solved for wiki admins. Is that not the case anymore? Did it get better?

extraduder_ire 3y ago

Very cool. How long until someone writes PyPi/npm packages to import/use these functions by Z number? Speaking of which, are there already packages to access wikidata by objects' Q numbers? The two could work together quite well, with a bit of caching.

aaron695 3y ago

[dead]

giovannibonetti 3y ago

Related: Hoogle [1] a function search index for the Haskell language. It searches functions by signature. [1] https://hoogle.haskell.org/ https://hoogle.haskell.org/

CJefferson 3y ago

Python is an awful choice of language for this (although I understand why they used it, it's so popular). What do they mean by 'Python'? Most programming languages make considerable effort in supporting old versions (gcc has c90 and c++98 for example, fortran goes even earlier). With Java, they make very strong efforts to support older code (as long as it uses the library which are labelled as stable). Old javascript code also tends to work well, as there are lots of old websites that don't want breaking. Python on the other hand has gone out of it's way to nuke Python 2, and many new version of Python 3 deletes something from the standard library.

JCharante 3y ago
Python 3 came out 15 years ago. Are we still really having this discussion?
- CJefferson 3y ago
  New Python versions do continue to remove features from the standard library, which can break programs. Most other languages I'm familiar with make efforts to only do this in extreme cases (like c removed gets, but basically nothing else). Also, I have Python 2 programs I'm keeping running, and expect to continue doing so for several years to come. Many libraries never made the jump to Python 3.
deleted 3y ago
[deleted]

pyromaker 3y ago

Will be cool if these functions were available via API endpoints :)

laserbeam 3y ago

Mixed feelings about the longevity of this. It feels like it wants to be an open source alternative to WolframAlpha, but without the UX of "just typing a question". One has to navigate and search for functions that do the thing they want done. Wanna convert a string to the nato phonetic alphabet? Sure, that function is there... You gotta search for it first though. Does it accept lowercase inputs? No, not at the time of writing this comment... But you know, someone might fix that. No problem. It sounds like a project that would be dead on arrival if it were launched by and for any other community. If wiki maintainers like this? Then great! I do see value if people could integrate functions within wikipedia article. Maybe it makes sense for certain articles to include some dynamic widgets in them. Unsure I'd like that but it may work.

trenchgun 3y ago

So this is something like Rosetta stone, but also has features or Unison language