9 ms·
What Lua scripting means for Wikimedia and open source
- youngerdryas 14y agoFrom the comments: "People who have programming skills should enjoy better productivity and lower frustration while editing Wikipedia. Perhaps most exciting is how many people will have a gentle and practical introduction to programming because of this." A lot of people on HN get down on Wikipedia because of deletionists etc. and support a forked version but the fact is Wikipedia is the most salient expression of the spirit of the internet. Google is great but is a business with all the downsides that entails, project abandonment etc. Maybe I am gushing but Wikipedia is without peer.
- kibwen 14y agoAlso of note is that (AFAICT) the ultimate decision came down to a choice between Lua and Javascript: http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/57769/ http://thread.gmane.org/gmane.science.linguistics.wikipedia....
- cs702 14y agoThis is a big deal: Wikipedia is allowing contributors to extend the functionality of Wikipedia with new code (written in a powerful language), with minimal or no supervision. It's hard to predict what kinds of improvements we will see over time. Every Wikipedia article is now, effectively, an interactive application. As far as I know, something like this has not been tried by anyone else at this scale before. Quoting from the original post: "Anyone can write a chunk of code to be included in an article that will be seen by millions of people, often without much review. We are taking our 'anyone can edit' maxim one big step forward."
- StavrosK 14y agoThe good thing about Lua is that you can sandbox it entirely and set hard limits on memory/CPU time, after which the process exits. You can just give an error message if RAM grows over a limit or if it takes longer than a few secs to execute. It's pretty safe.
- pekk 14y agoIs it possible to run a program in 'steps,' between which the state is paused?
- StavrosK 14y agoI don't think it's possible with the stock interpreter, but I'm guessing it's pretty simple to modify the interpreter to do it. Why do you ask?
- pekk 14y agoSimple time sharing between low-priority, potentially long-running programs submitted by untrusted users.
- StavrosK 14y agoRun a process per script and let the OS handle it, probably.
- gngeal 14y agoYou don't even need to do that, you can run Lua in a multithreaded program, as long as no lua_State is accessed from multiple threads. You can spawn independent interpreter states, have them do their job and terminate. The whole Lua codebase as such is completely stateless (apart from the state that you keep for your code). The shorter the scripts actually run, the smaller overhead you get by this.
- pekk 14y agoThat isn't what I asked about and isn't really helpful
- StavrosK 14y agoDon't read it, I guess? I don't know any other way to do it, unfortunately. I'm sorry I wasted your time.
- mjn 14y agoIt's sort of been the case on Wikipedia already, but until now via a pretty crufty, home-grown templating system. Most people thought of it as a mere templating system, but it had #if/#else constructs for conditional inclusion, and some similar features, so you could program in it, if you tried hard enough (and some people did).
- Mindless2112 14y ago> Every Wikipedia article is now, effectively, an interactive application. I don't think that's quite true. MediaWiki templates are now more powerful, but I'm not sure even that counts as interactive. This makes editing easier, but the reader isn't going to interact with a Wikipedia page any more than they did last week.
- metacontent 14y agoI'm not quite sure if this applies, but can't anyone currently write a Lua script for World of Warcraft? In which case the script could possibly be connecting to and interacting with the WoW servers? If one thing can be said about Lua, it is that embedding Lua is a solved problem, and not something dangerous to be worried about.
- kemayo 14y agoIt's similar, I suppose. Like the Wikimedia example, the Lua you write for WoW is tightly limited -- there's an API available [1], and you can request some extra data from the server sometimes. However, there's absolutely no loading of external modules, contacting any sort of outside server, or any direct file access. There's also large swaths of "what the player can do" which are blocked off from the API, in the name of not allowing people to automate core game elements. [1]: http://wowprogramming.com/docs/api http://wowprogramming.com/docs/api
- Zash 14y agoVideo presentation and demo: http://www.youtube.com/watch?v=PrhzAtC8fCc http://www.youtube.com/watch?v=PrhzAtC8fCc
- TazeTSchnitzel 14y agoSome of the examples were shocking. The inefficiency of str len astounds me.
- ksec 14y agoFinally, i hope this makes Lua much more widespread.
- scribu 14y agoTemplates were already pretty complex a few years ago when I was a casual contributor. I am so glad they did this! Now, if we could only go back in time and replace Vimscript with Lua or Lisp or anything...
- simonw 14y agoCan anyone link to a good example of a simply lua template that's live on Wikipedia right now? The linked article wasn't pretty great for examples.
- acqq 14y agohttps://en.wikipedia.org/wiki/Module:Citation/CS1 https://en.wikipedia.org/wiki/Module:Citation/CS1
- deleted 14y ago[deleted]
- acqq 14y agoPerformance change after some wikipedia templates were rewritten in Lua: https://en.wikipedia.org/wiki/User:Dragons_flight/Lua_performance https://en.wikipedia.org/wiki/User:Dragons_flight/Lua_perfor...
- vor_ 14y agoI'm glad they replaced the old template nightmare, but I wish Lua was a little more sane since it might be some people's first exposure to a real programming language. For example, it silently returns nil when accessing undefined variables, returns true for (1) but false for (1 == true), has 1-based arrays, can have array "holes" that confuse the length operator, and so on. I can't help but think that as Lua gains more mindshare, its quirks might become major criticisms in the same way PHP's weirdness was ignored until it became widely used.
- greggman 14y agoThis makes me both happy and sad. I'm happy that Wikimedia is adding a code path. I'm sad that it's not JavaScript. Wikimedia needs to move beyond 1990s internet and allow interactive articles. Articles on 3D that have interactive 3D. Articles on Physics that have interactive simulations. If you asked me in 1980 what an encyclopedia would look like in 2013 I would not have guessed static text + static pictures. I know it's not an "either or" thing. They can have both. But I think once they get around to adding JavaScript they'll regret having added Lua now and having 2 code bases with 2 sets of libraries and code to synchronize the 2 when they need to communicate with each other.
- SomeCallMeTim 14y agoI completely disagree. Lua is SUCH a cleaner, faster, more consistent language than JavaScript, ESPECIALLY when considering how easy it is to embed in another app (like binding it to PHP), that it's a no-brainer to use Lua. They were looking for speed, and LuaJIT is FAST; approaching C speed under the right circumstances. What I'm sad about is that JavaScript is so locked-in on the browser that I can't use Lua there as well.
- azakai 14y agoI would agree that Lua is cleaner. But it is not perfect either, for example indexing starting at 1 is definitely an oddity. As for fast, I would love to see more comparisons on real-world code, there are too few on the web. It is easier to embed Lua than JavaScript, however, that's say a week of work, then you're done, so it isn't a reason for something like Wikipedia to prefer Lua. With all that said, Lua and JavaScript are both good languages, adding either one to Wikipedia is going to be a huge improvement. edit: looks like my phrasing has offended mikemike, who I have the utmost respect for, so I removed some stuff he disliked.
- bsg75 14y agoThe Lua index from 1 complaint is becoming the new Python whitespace gripe.
- sigil 14y agoWhile more efficient templating sounds like a good thing and is clearly a pain point, I'm not convinced Wikimedia is really thinking this one through. 1. Every page is now a "real" program. Will separation of code and content result from this? The article hints that structured data is on the way, but I have a hard time imagining the average contributor will restrain themselves given this awesome new power. Instead I foresee bigger and badder messes. 2. Does this get us any closer to a parseable format for Wikipedia content? This is the world's knowledge. The fact that it's currently trapped in an ad hoc, PHP-inspired template "language" (there is no formal grammar for it, it's whatever some PHP code accepts as input) should be very concerning. 3. Why Lua? Javascript seemed like the obvious choice. Rendering templates server-side at Wikipedia's scale sounds incredibly challenging; it boggles the mind really. Surely most page views are from js-enabled browsers -- why not offload all that rendering to the client?
- capnrefsmmat 14y agoLua was chosen over JavaScript for its ease of embedding. It's easy to limit how much memory the Lua interpreter uses to prevent users from DoSing Wikimedia servers with templates. V8 and SpiderMonkey do not make this easy. More details here: https://www.mediawiki.org/wiki/User:Sumanah/Lua_vs_Javascript https://www.mediawiki.org/wiki/User:Sumanah/Lua_vs_Javascrip... Wikipedia pages are massively cached: rendered pages are cached in Squid, and I believe parsed page contents are stored in memcached. Templates are only re-rendered when pages are edited, if all goes well.
- gregholmberg 14y agoWhy Lua? I have been running Node.js and Lua at fairly large scale on EC2 recently. I like the V8/Lua/Luajit platforms just fine. Each has some nice strengths. With Lua, I see significantly less memory use, and moderately increased performance. As long as RAM is the expensive bottleneck for hosted servers, it looks like I will be able to get better performance (in operations per dollar) by using a smaller runtime. Why not client-side rendering? Accessibility? Cross-platform uniformity? (better answer from 'capnrefsmmat: render once, then cache massively)
- neilk 14y ago
- gngeal 14y ago"But, because we’d never planned for wikitext to become a programming language, these templates were terribly inefficient and hacky — they didn’t even have recursion or loops — and were terrible for performance." That sounds like the life story of PHP! And many other similar solutions.
- DannoHung 14y agoTo people grousing about it no being in JS: The hell is wrong with you? JS being the only language your browser will run is the problem, not people choosing to use something besides JS for something they do on the web.
- reirob 14y agoThis news made look closer into Lua. I discovered that it actually does not provide any support for unicode. I am wondering now how this will impact the goals for Lua as scripting language? Or is there a special module for Unicode support that will be used?