9 ms·

The Lua VM, on the Web

132 points by vmorgulis 10y ago

scrame 10y ago
So, I understand that asm.js is a strict subset of JS that's made to run very efficiently. I understand emscripten can transpile C to asm.js, so C programs to run in a browser. I guess what I don't understand is how the higher-level bindings work. Something like the neo-geo emulators running in asm.js can read controllers and output graphics. How much custom code is required going from system C to browser JS. Is it closer to a 'gimme' if it targets GL and can just use WebGL? Is there something like sdl.js? And if it has those external library dependencies, does the whole dependency tree have to be compiled to asm.js? More specifically, if the lua VM can be put into the browser, and interpret lua code, then how much custom code would it take to make Love2D work? Edit: Just re-reading this question I realized it'd most likely be like a static build, so it would pull in the dependency source, and then most likely something that would use the io/display stuff API, but with a browser back-end. I've dug around a couple times trying to find a basic overview of that, but to not much avail. I'd love any references though.
- cygx 10y ago
  Emscripten[1] comes with SDL support (cf its test suite [2]). Not sure how complete it is. [1] http://kripken.github.io/emscripten-site/ http://kripken.github.io/emscripten-site/ [2] https://github.com/kripken/emscripten/blob/master/tests/hello_image_sdl.c https://github.com/kripken/emscripten/blob/master/tests/hell...
  - vitovito 10y ago
    The original SDL support was a JavaScript implementation of just enough of 1.x to get things compiling. SDL2 cross-compiles to JavaScript properly and should be used by most things now.
    - scrame 10y ago
      so emscripten provides the sdl binding? I was assuming it'd cover libc, but does that mean it's part of the ecosystem or the compiler? I feel like I'm just asking dumb toolchain questions because I'm missing something fundamental about targeting an executable build to the browser.
      - corysama 10y ago
        I think this is what you are missing. https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html https://kripken.github.io/emscripten-site/docs/porting/conne...
  - dividuum 10y ago
    Emscripten also allows access to WebGL, which is pretty neat. I'm writing (and selling, you should check out https://info-beamer.com/pi https://info-beamer.com/pi) a software to make it easy to build animated information screens. I'm also embedding Lua to make everything scriptable. As a proof of concept I cross-compiled the existing open source software (https://github.com/dividuum/info-beamer https://github.com/dividuum/info-beamer) to JS with emscripten two years ago. Here's the demo site: https://dividuum.de/experiments/info-beamer-js/ https://dividuum.de/experiments/info-beamer-js/. What you see is a C program with an embedded Lua VM running very simple Lua code (https://github.com/dividuum/info-beamer/blob/master/samples/shader/node.lua https://github.com/dividuum/info-beamer/blob/master/samples/...) to transform a loaded PNG image through a GLSL in a WebGL context. It's pretty amazing that these things are possible. I'm also pretty sure a lot has improved in emscripten since then. Maybe I should give that another try...
- fiatjaf 10y ago
  I have naïver questions: taking this port of GNUnet[1] to Javascript through emscripten, where does the GNUnet code ends and where does JS starts? What functions are exported by the GNUnet code? In this case, Clojurescript is used, but I suppose I could reuse the "ported C" part and write my own pure-Javascript app on top of it, right? How difficult is that? [1]: https://github.com/amatus/gnunet-web https://github.com/amatus/gnunet-web
- vmorgulis 10y ago
  > .. how much custom code would it take to make Love2D work? Someone is working on it: http://tannerrogalsky.com/mrrescue/ http://tannerrogalsky.com/mrrescue/ https://github.com/TannerRogalsky/love.js https://github.com/TannerRogalsky/love.js
- flohofwoe 10y ago
  There are several approaches in emscripten to talk to the JS/HTML5 side from the C/C++ side: 1) wrappers for popular native APIs and frameworks to simplify porting existing code: apart from the 'expected' C and C++ runtime wrappers, emscripten also offers wrappers for GL, GLES 1..3, EGL, GLFW, GLU, GLUT, GLEW, SDL and OpenAL, these generally translate (and sometimes emulate) the behaviour of the native APIs to the underlying HTML5 APIs (e.g. GL to WebGL, SDL Audio and OpenAL to WebAudio, etc...) 2) there are also smaller emscripten-specific APIs that are essentially minimal shims around HTML5 APIs, for instance to initialize a WebGL context, one could either use the EGL or SDL wrappers which add some code bloat and don't directly map to WebGL initialization attributes, or one can use the emscripten_webgl_*() functions which map directly to the WebGL context initialization process, there are also similar small shims for input, XMLHttpRequest, WebWorkers and others, the advantage of these direct HTML5 shims is that they introduce less code bloat, but they don't exist anywhere outside the emscripten platform 3) emscripten also provides a number of different ways to easily write your own shims to call from C/C++ into JS, and the other way around, even with marshalling complex C++ data types (see: https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/embind.html https://kripken.github.io/emscripten-site/docs/porting/conne...) I really like that emscripten is much more than just a compiler toolchain with C/C++ runtime, there has been a tremendous amount of work to make it as simple as possible to port existing code over without or just minimal changes!
  - scrame 10y ago
    Thank you! That clears it up. I wasn't getting that it had the whole run-time with it, and that it was so extensive.
- IshKebab 10y ago
  Basically emscripten provides a set of libraries that emulate C libraries in Javascript. For example you can use the standard Berkeley socket API, but it actually translates TCP and UDP into SCTP (which can support TCP-like and UDP-like modes) via WebRTC's DataChannel. Similarly, emscripten provides a standard C OpenGL API, but it actually translates those calls to WebGL Javascript calls.
scardine 10y ago
Lua and Elixir are among the few relevant contributions to the field coming from Brazil.
vmorgulis 10y ago
Few running example: http://mangelmaxime.github.io/Lua.vm.js-Samples/pages/www/ http://mangelmaxime.github.io/Lua.vm.js-Samples/pages/www/
lopatin 10y ago
This is really cool. I know Lua is a commonly embedded language for plugins and stuff. So now that we can support Lua powered extensions for web apps, I'd like to add to the list of naive questions here, coming from someone who's never used the language. What benefits does Lua provide over other langs for such a purpose? Is it the ecosystem of libraries that makes it unique in that respect? Or the actual language lends itself better for user scripting over, say, running JS code through JS-interpreter or just eval-ing validated JS code directly.
- vvanders 10y ago
  It's an incredibly composable language through metatables, closures and coroutines. You can implement anything from component based design, traditional inheritance and other paradigms with just those 3 components.
- meric 10y ago
  The syntax is very simple. I wrote a hybrid lua/lisp language with it https://github.com/meric/l2l https://github.com/meric/l2l, where Lua expressions can be quoted and quasi-quoted seamlessly. Excuse me for the plug. :)
- john_reel 10y ago
  Lua is perfectly multiparadigm. Anyone who hasn’t tried it should.
  - sprobertson 10y ago
    I only recently realized how much lua I'm using day to day: * nginx modules (proxying rules) * Redis modules (special triggers) * Machine learning with Torch * Embedded systems (ESP8266)
qwertyuiop924 10y ago
Now I want to see how the performance compares to that of Moonshine.
nitemice 10y ago
The future is coming! https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...
- omginternets 10y ago
  This really was a fantastic talk. Where can I read more about this kernel-level sandboxing that's discussed towards the end? Additionally: as I understand things, the methodology behind asm.js could (in principle) be applied to any JIT-ed language. Is the choice of doing this in JS only because JS is the language of browsers (i.e.: kernel-level sandboxing can easily use existing browser sandboxing code) or are there other technical considerations behind this decision?
- bogomipz 10y ago
  I have watched this talk on two occasions now. Gary Bernhardt is really a great speaker. I am curious has anyone subscribed to the screencasts at: https://www.destroyallsoftware.com/screencasts https://www.destroyallsoftware.com/screencasts It's a bit pricey at $29 dollars a month but if all of the content is as interesting, entertaining and thought provoking as this talk I could see it being worth the price. I would be curious to hear anyone's feedback.
ww520 10y ago
This is very cool. The Lua VM can run in Node.js backend in this case.
- ufo 10y ago
  If you are in the server why not just run Lua directly?
s369610 10y ago
I Ported the MOAI engine of Broken Age fame to html/emscripten, It runs full lua along with a bunch of other libraries (libpng, freetype etc) You can see it running here (and edit the code and run in browser) http://moaiforge.com/samples/sample-browser/player/index.html#hello-moai http://moaiforge.com/samples/sample-browser/player/index.htm...
- Vaskivo 10y ago
  I absolutely love MOAI! Thank you for your work. It is a really well designed engine. And it's community has a lot of talented people. I'm deeply saddened that it appears to be dying. And that I don't have more time to work with it and contribute to it.
- Tanner 10y ago
  Hey, we interacted a little over 2 years ago when I was talking about a hand-coded port of Love2D to the web using Moonshine. Eventually the dependencies for Love did get to the point where I could do an emscripten port. Just wanted you to know that your project and our little interaction has stuck with me. The emscripten port of Love.js: https://github.com/TannerRogalsky/love.js https://github.com/TannerRogalsky/love.js
  - s369610 10y ago
    Wow, glad to see it come together! Love2d is so clean, it is great that it works in browser now.
d33 10y ago
I'm surprised nobody said it here - it would be even cooler with webassembly! Imagine writing Lua code instead of JS... EDIT: though now that I look at it, it's already possible: https://daurnimator.github.io/lua.vm.js/script_example.html https://daurnimator.github.io/lua.vm.js/script_example.html
- etiene 10y ago
  Im waiting for this natively, anxiously
- john_reel 10y ago
  Yes please! Any ETAs on webassembly landing?
- daurnimator 10y ago
  webassembly gains lua.vm.js nothing: it's just a bytecode form of asm.js The big thing lua.vm.js needs is weakrefs https://github.com/tc39/proposal-weakrefs https://github.com/tc39/proposal-weakrefs
marktangotango 10y ago
So, a bit off topic, but is anyone using Lua for web application backends? For example Lua with nginx or mod_lua for apache? I've used Lua with mod_lua, and have been very happy with the result. There are actually a lot of libraries available on luarocks.org, although not at the 'batteries included' level of Python.
- etiene 10y ago
  I use my own framework. It's MVC and it works on the top of nginx(openresty), apache(mod_lua) and xavante (a pure Lua web server). It's called Sailor http://sailorproject.org http://sailorproject.org I also wrote a blog post with a comparison of different frameworks: http://lua.space/webdev/the-best-lua-web-frameworks http://lua.space/webdev/the-best-lua-web-frameworks
- sprobertson 10y ago
  I use it between nginx and backend servers, to do routing/proxying based on hostnames in Redis: https://github.com/spro/simon https://github.com/spro/simon I have been slowly working towards a full express clone, but don't have a compelling reason to complete it.
etiene 10y ago
This is another Lua VM worth taking a look: http://starlight.paulcuth.me.uk/ http://starlight.paulcuth.me.uk/ I use both of them with Sailor, but I tend to prefer starlight recently

132 points by vmorgulis 10y ago

So, I understand that asm.js is a strict subset of JS that's made to run very efficiently. I understand emscripten can transpile C to asm.js, so C programs to run in a browser. I guess what I don't understand is how the higher-level bindings work. Something like the neo-geo emulators running in asm.js can read controllers and output graphics. How much custom code is required going from system C to browser JS. Is it closer to a 'gimme' if it targets GL and can just use WebGL? Is there something like sdl.js? And if it has those external library dependencies, does the whole dependency tree have to be compiled to asm.js? More specifically, if the lua VM can be put into the browser, and interpret lua code, then how much custom code would it take to make Love2D work? Edit: Just re-reading this question I realized it'd most likely be like a static build, so it would pull in the dependency source, and then most likely something that would use the io/display stuff API, but with a browser back-end. I've dug around a couple times trying to find a basic overview of that, but to not much avail. I'd love any references though.

cygx 10y ago
Emscripten[1] comes with SDL support (cf its test suite [2]). Not sure how complete it is. [1] http://kripken.github.io/emscripten-site/ http://kripken.github.io/emscripten-site/ [2] https://github.com/kripken/emscripten/blob/master/tests/hello_image_sdl.c https://github.com/kripken/emscripten/blob/master/tests/hell...
- vitovito 10y ago
  The original SDL support was a JavaScript implementation of just enough of 1.x to get things compiling. SDL2 cross-compiles to JavaScript properly and should be used by most things now.
  - scrame 10y ago
    so emscripten provides the sdl binding? I was assuming it'd cover libc, but does that mean it's part of the ecosystem or the compiler? I feel like I'm just asking dumb toolchain questions because I'm missing something fundamental about targeting an executable build to the browser.
    - corysama 10y ago
      I think this is what you are missing. https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html https://kripken.github.io/emscripten-site/docs/porting/conne...
- dividuum 10y ago
  Emscripten also allows access to WebGL, which is pretty neat. I'm writing (and selling, you should check out https://info-beamer.com/pi https://info-beamer.com/pi) a software to make it easy to build animated information screens. I'm also embedding Lua to make everything scriptable. As a proof of concept I cross-compiled the existing open source software (https://github.com/dividuum/info-beamer https://github.com/dividuum/info-beamer) to JS with emscripten two years ago. Here's the demo site: https://dividuum.de/experiments/info-beamer-js/ https://dividuum.de/experiments/info-beamer-js/. What you see is a C program with an embedded Lua VM running very simple Lua code (https://github.com/dividuum/info-beamer/blob/master/samples/shader/node.lua https://github.com/dividuum/info-beamer/blob/master/samples/...) to transform a loaded PNG image through a GLSL in a WebGL context. It's pretty amazing that these things are possible. I'm also pretty sure a lot has improved in emscripten since then. Maybe I should give that another try...
fiatjaf 10y ago
I have naïver questions: taking this port of GNUnet[1] to Javascript through emscripten, where does the GNUnet code ends and where does JS starts? What functions are exported by the GNUnet code? In this case, Clojurescript is used, but I suppose I could reuse the "ported C" part and write my own pure-Javascript app on top of it, right? How difficult is that? [1]: https://github.com/amatus/gnunet-web https://github.com/amatus/gnunet-web
vmorgulis 10y ago
> .. how much custom code would it take to make Love2D work? Someone is working on it: http://tannerrogalsky.com/mrrescue/ http://tannerrogalsky.com/mrrescue/ https://github.com/TannerRogalsky/love.js https://github.com/TannerRogalsky/love.js
flohofwoe 10y ago
There are several approaches in emscripten to talk to the JS/HTML5 side from the C/C++ side: 1) wrappers for popular native APIs and frameworks to simplify porting existing code: apart from the 'expected' C and C++ runtime wrappers, emscripten also offers wrappers for GL, GLES 1..3, EGL, GLFW, GLU, GLUT, GLEW, SDL and OpenAL, these generally translate (and sometimes emulate) the behaviour of the native APIs to the underlying HTML5 APIs (e.g. GL to WebGL, SDL Audio and OpenAL to WebAudio, etc...) 2) there are also smaller emscripten-specific APIs that are essentially minimal shims around HTML5 APIs, for instance to initialize a WebGL context, one could either use the EGL or SDL wrappers which add some code bloat and don't directly map to WebGL initialization attributes, or one can use the emscripten_webgl_*() functions which map directly to the WebGL context initialization process, there are also similar small shims for input, XMLHttpRequest, WebWorkers and others, the advantage of these direct HTML5 shims is that they introduce less code bloat, but they don't exist anywhere outside the emscripten platform 3) emscripten also provides a number of different ways to easily write your own shims to call from C/C++ into JS, and the other way around, even with marshalling complex C++ data types (see: https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/embind.html https://kripken.github.io/emscripten-site/docs/porting/conne...) I really like that emscripten is much more than just a compiler toolchain with C/C++ runtime, there has been a tremendous amount of work to make it as simple as possible to port existing code over without or just minimal changes!
- scrame 10y ago
  Thank you! That clears it up. I wasn't getting that it had the whole run-time with it, and that it was so extensive.
IshKebab 10y ago
Basically emscripten provides a set of libraries that emulate C libraries in Javascript. For example you can use the standard Berkeley socket API, but it actually translates TCP and UDP into SCTP (which can support TCP-like and UDP-like modes) via WebRTC's DataChannel. Similarly, emscripten provides a standard C OpenGL API, but it actually translates those calls to WebGL Javascript calls.

scardine 10y ago

Lua and Elixir are among the few relevant contributions to the field coming from Brazil.

vmorgulis 10y ago

Few running example: http://mangelmaxime.github.io/Lua.vm.js-Samples/pages/www/ http://mangelmaxime.github.io/Lua.vm.js-Samples/pages/www/

lopatin 10y ago

This is really cool. I know Lua is a commonly embedded language for plugins and stuff. So now that we can support Lua powered extensions for web apps, I'd like to add to the list of naive questions here, coming from someone who's never used the language. What benefits does Lua provide over other langs for such a purpose? Is it the ecosystem of libraries that makes it unique in that respect? Or the actual language lends itself better for user scripting over, say, running JS code through JS-interpreter or just eval-ing validated JS code directly.

vvanders 10y ago
It's an incredibly composable language through metatables, closures and coroutines. You can implement anything from component based design, traditional inheritance and other paradigms with just those 3 components.
meric 10y ago
The syntax is very simple. I wrote a hybrid lua/lisp language with it https://github.com/meric/l2l https://github.com/meric/l2l, where Lua expressions can be quoted and quasi-quoted seamlessly. Excuse me for the plug. :)
john_reel 10y ago
Lua is perfectly multiparadigm. Anyone who hasn’t tried it should.
- sprobertson 10y ago
  I only recently realized how much lua I'm using day to day: * nginx modules (proxying rules) * Redis modules (special triggers) * Machine learning with Torch * Embedded systems (ESP8266)

qwertyuiop924 10y ago

Now I want to see how the performance compares to that of Moonshine.

nitemice 10y ago

The future is coming! https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...

omginternets 10y ago
This really was a fantastic talk. Where can I read more about this kernel-level sandboxing that's discussed towards the end? Additionally: as I understand things, the methodology behind asm.js could (in principle) be applied to any JIT-ed language. Is the choice of doing this in JS only because JS is the language of browsers (i.e.: kernel-level sandboxing can easily use existing browser sandboxing code) or are there other technical considerations behind this decision?
bogomipz 10y ago
I have watched this talk on two occasions now. Gary Bernhardt is really a great speaker. I am curious has anyone subscribed to the screencasts at: https://www.destroyallsoftware.com/screencasts https://www.destroyallsoftware.com/screencasts It's a bit pricey at $29 dollars a month but if all of the content is as interesting, entertaining and thought provoking as this talk I could see it being worth the price. I would be curious to hear anyone's feedback.

ww520 10y ago

This is very cool. The Lua VM can run in Node.js backend in this case.

ufo 10y ago
If you are in the server why not just run Lua directly?

s369610 10y ago

I Ported the MOAI engine of Broken Age fame to html/emscripten, It runs full lua along with a bunch of other libraries (libpng, freetype etc) You can see it running here (and edit the code and run in browser) http://moaiforge.com/samples/sample-browser/player/index.html#hello-moai http://moaiforge.com/samples/sample-browser/player/index.htm...

Vaskivo 10y ago
I absolutely love MOAI! Thank you for your work. It is a really well designed engine. And it's community has a lot of talented people. I'm deeply saddened that it appears to be dying. And that I don't have more time to work with it and contribute to it.
Tanner 10y ago
Hey, we interacted a little over 2 years ago when I was talking about a hand-coded port of Love2D to the web using Moonshine. Eventually the dependencies for Love did get to the point where I could do an emscripten port. Just wanted you to know that your project and our little interaction has stuck with me. The emscripten port of Love.js: https://github.com/TannerRogalsky/love.js https://github.com/TannerRogalsky/love.js
- s369610 10y ago
  Wow, glad to see it come together! Love2d is so clean, it is great that it works in browser now.

d33 10y ago

I'm surprised nobody said it here - it would be even cooler with webassembly! Imagine writing Lua code instead of JS... EDIT: though now that I look at it, it's already possible: https://daurnimator.github.io/lua.vm.js/script_example.html https://daurnimator.github.io/lua.vm.js/script_example.html

etiene 10y ago
Im waiting for this natively, anxiously
john_reel 10y ago
Yes please! Any ETAs on webassembly landing?
daurnimator 10y ago
webassembly gains lua.vm.js nothing: it's just a bytecode form of asm.js The big thing lua.vm.js needs is weakrefs https://github.com/tc39/proposal-weakrefs https://github.com/tc39/proposal-weakrefs

marktangotango 10y ago

So, a bit off topic, but is anyone using Lua for web application backends? For example Lua with nginx or mod_lua for apache? I've used Lua with mod_lua, and have been very happy with the result. There are actually a lot of libraries available on luarocks.org, although not at the 'batteries included' level of Python.

etiene 10y ago
I use my own framework. It's MVC and it works on the top of nginx(openresty), apache(mod_lua) and xavante (a pure Lua web server). It's called Sailor http://sailorproject.org http://sailorproject.org I also wrote a blog post with a comparison of different frameworks: http://lua.space/webdev/the-best-lua-web-frameworks http://lua.space/webdev/the-best-lua-web-frameworks
sprobertson 10y ago
I use it between nginx and backend servers, to do routing/proxying based on hostnames in Redis: https://github.com/spro/simon https://github.com/spro/simon I have been slowly working towards a full express clone, but don't have a compelling reason to complete it.

etiene 10y ago

This is another Lua VM worth taking a look: http://starlight.paulcuth.me.uk/ http://starlight.paulcuth.me.uk/ I use both of them with Sailor, but I tend to prefer starlight recently