19 ms·

Luje 0.1 – a pure Lua JVM

160 points by leafo 13y ago

tinco 13y ago
The article makes a rather hefty claim, that it is comparable or even in some ways faster performancewise than Sun's hotspot JVM. From what I've heard the JVM is one of the most heavily researched and optimized bytecode VM out there. Could it really be that some guy can write a competitor in a scripting language in his spare time? Not saying it's impossible, just curious if there's anything to back that claim up, besides the very specific microbenchmarks. edit: I was a bit too quick to comment, reading again he doesn't really claim to be faster at anything other than small loops. Still interested if it's fast at any real world things though.
- seiji 13y ago
  It's using LuaJIT which regularly advances the state of the art in JIT'd language performance optimizations.
- lucian1900 13y ago
  Afaict this project isn't actually a JVM, it translates JVM bytecode to Lua and then runs that. It's perfectly reasonable to expect that to be very fast, assuming Lua(JIT) can compete with JVM in speed (which it does).
  - tinco 13y ago
    There's usually a problem with that approach and that is when the virtual machine models don't match perfectly. This leads to any feature mismatches to be implemented in a roundabout way often leading to performance problems. An example of this is the projects that implement ruby virtual machines on the JVM and the CLR. Both VM's actually had features added later to better match Ruby's features. If LuaJIT can really execute Java bytecode so efficiently that says a lot for the genericity of the compiler, which makes it pretty awesome tech indeed :)
    - lucian1900 13y ago
      I don't think those two cases are comparable. Ruby is a fairly large and very generic language. It's possible to override pretty much anything, which makes running it efficiently on a runtime that doesn't is hard. On the other hand, JVM bytecode is a tiny, concrete language, pretty much just saver assembly. Few operations are generic, and when they are they're generic on the type of something, not much else. Most languages that are already fast could probably run translated JVM bytecode fast. (And of course LuaJIT is amazing, but for being that fast in the first place)
- MrBuddyCasino 13y ago
  Well he didn't write a fully jitted VM, this is just sort of translating the JVM opcodes into Lua scripts if I got that right, and then runs those with LuaJit 2. LuaJIT 2 is known to be fast, but I too was impressed by the numbers, outperforming HotSpot is pretty good, even if its just a tiny microbenchmark.
  - copx 13y ago
    Seems it could be even faster: http://www.freelists.org/post/luajit/ANN-luje-01 http://www.freelists.org/post/luajit/ANN-luje-01
- zeckalpha 13y ago
  LuaJIT is heavily researched and optimized. http://article.gmane.org/gmane.comp.lang.lua.general/58908 http://article.gmane.org/gmane.comp.lang.lua.general/58908
- pron 13y ago
  HotSpot is indeed one of the most optimized runtimes around. It is, however, optimized for large, complex applications. It's quite possible to beat in a carefully designed microbenchmark. Beating it in a complex application is a whole other matter (in fact, it's pretty hard to beat even in C++).
  - haberman 13y ago
    > in fact, it's pretty hard to beat even in C++ If there's one thing I've learned, it's that the wishful thinking will never stop.
    - dman 13y ago
      What?
      - deleted 13y ago
        [deleted]
    - peterashford 13y ago
      That's pretty snarky
      - haberman 13y ago
        It's really irritating, as someone who works in C and C++ with good reason, to hear people continually deny the very real performance benefits of working at a lower level. We (C/C++ guys) write the OS's, VMs, browsers, codecs, etc that power your software stacks, and then you (managed language fans) turn around and tell us we're wasting our time by using C and C++, while giving us stuff like Eclipse. So yes, when someone says it's hard to beat Java with C++, it inspires some snark. Write the next JVM in Java if you think it's that easy.
        hermanhermitage 13y ago
        This is what happens when an important variable (the specific problem domain of performance comparison) is left as a free variable :-)
- aaronblohowiak 13y ago
  > In any non-trivial benchmark it usually manages about 50% to 75% of Hotspot.
jacques_chester 13y ago
David Given is one of those prolific thinkers and doers who seem to be overrepresented in the Lua community. He's probably best known to HNers for his critical article "On Go" and Objective Lua. http://cowlark.com/2009-11-15-go/ http://cowlark.com/2009-11-15-go/ https://cowlark.com/objective-lua/index.html https://cowlark.com/objective-lua/index.html
- seabrookmx 13y ago
  Thanks for the link! That's a really interesting article. Though out of date, I still think his point holds water. It's shocking how little difference there is between Algol-68 and modern Go or C. Go has changed quite a bit since his article, but not all of these changes have been well received (a great example being automatic semicolon insertion).
doublec 13y ago
There's an interesting paper about a similar project that was done using Self [1]. "Design and Implementation of Pep, a Java Just-In-Time Translator written in Self" It goes through details with benchmarks. The PEP code is available with the Self distribution. They were able to get faster than the Java implementation at the time. Back then Java was interpreted though. [1] http://bluishcoder.co.nz/self/97-pep.pdf http://bluishcoder.co.nz/self/97-pep.pdf
- pjmlp 13y ago
  Languages are nor interpreted or compiled. It is all about implementations.
taranka231 13y ago
is there any java/lua interop?
deleted 13y ago
[deleted]
hbbio 13y ago
It's an impressive project that shows the extraordinary work of a single developer, Mike Pall. LuaJIT is not only an incredibly beautiful JIT, the current project also demonstrates it can run generated code efficiently, which is probably the hardest thing to do for a language. When implementing Opa in OCaml, we run into tons of problem since the OCaml runtime was unable to run generated code efficiently. One of its author, Pierre Weis told us at the time: "it's normal, OCaml is not meant to run generated code at all". As a side note regarding Luje source code: Fossil must be great and all that but having the code on a publicly available Git repository would probably bolster contributions.
- jsnell 13y ago
  Luajit is of course awesome, but let's not get too carried away. It doesn't sound like this JVM has been used for running anything beyond extremely simple benchmarks. Of course you're not going to run into difficulties with the translation of a 15 line method. I saw a talk Mike gave on Luajit last week, and one thing he credited Luajit's success on was that it was working on a rather high abstraction layer. Stuff like the loop optimizations and alias analysis work because the optimizer can essentially understand the code at a fairly high level, and is somewhat specific to Lua. When dealing with java translated to jvm bytecode naively translated to lua source you lose much of that context. (And at the time he didn't seem too convinced about using luajit as the backend for other languages.) But even if you ignore the loss of context, I'm sure that various luajit limits would get hit with real life Java code. E.g. I'd guess there's a fundamental 65k limit for the number of constants and bytecode instructions in a single function.
  - justincormack 13y ago
    JVM bytecode is quite nasty from this point of view; one of Mike's comments in the mailing list was to try Dalvik instead. It loses a lot of loop structure.
    - jonasb 13y ago
      http://www.freelists.org/post/luajit/ANN-luje-01 http://www.freelists.org/post/luajit/ANN-luje-01
JulianMorrison 13y ago
Isn't this going to be intrinsically restricted to one numeric type? Lua numbers are all the same type (double by default).
- justincormack 13y ago
  No, LuaJIT has an ffi interface with support for any type, and you can just use them as if they are native types.
  - derleth 13y ago
    Seems a bit like cheating if you do it that way, though.
    - deleted 13y ago
      [deleted]
    - justincormack 13y ago
      Its not really, they are just native types, but defined with a C syntax so they are compatible with C. The code generation knows about them natively and optimises them.
- derleth 13y ago
  Eh, programs compiled to assembly language aren't limited to machine-word-sized integers and whatever the floating-point hardware (if any) natively supports. I'd be interested to know what its performance on integer code is, though.
swah 13y ago
Is LuaJIT bytecode a good target for toy languages?

160 points by leafo 13y ago

tinco 13y ago

The article makes a rather hefty claim, that it is comparable or even in some ways faster performancewise than Sun's hotspot JVM. From what I've heard the JVM is one of the most heavily researched and optimized bytecode VM out there. Could it really be that some guy can write a competitor in a scripting language in his spare time? Not saying it's impossible, just curious if there's anything to back that claim up, besides the very specific microbenchmarks. edit: I was a bit too quick to comment, reading again he doesn't really claim to be faster at anything other than small loops. Still interested if it's fast at any real world things though.

seiji 13y ago
It's using LuaJIT which regularly advances the state of the art in JIT'd language performance optimizations.
lucian1900 13y ago
Afaict this project isn't actually a JVM, it translates JVM bytecode to Lua and then runs that. It's perfectly reasonable to expect that to be very fast, assuming Lua(JIT) can compete with JVM in speed (which it does).
- tinco 13y ago
  There's usually a problem with that approach and that is when the virtual machine models don't match perfectly. This leads to any feature mismatches to be implemented in a roundabout way often leading to performance problems. An example of this is the projects that implement ruby virtual machines on the JVM and the CLR. Both VM's actually had features added later to better match Ruby's features. If LuaJIT can really execute Java bytecode so efficiently that says a lot for the genericity of the compiler, which makes it pretty awesome tech indeed :)
  - lucian1900 13y ago
    I don't think those two cases are comparable. Ruby is a fairly large and very generic language. It's possible to override pretty much anything, which makes running it efficiently on a runtime that doesn't is hard. On the other hand, JVM bytecode is a tiny, concrete language, pretty much just saver assembly. Few operations are generic, and when they are they're generic on the type of something, not much else. Most languages that are already fast could probably run translated JVM bytecode fast. (And of course LuaJIT is amazing, but for being that fast in the first place)
MrBuddyCasino 13y ago
Well he didn't write a fully jitted VM, this is just sort of translating the JVM opcodes into Lua scripts if I got that right, and then runs those with LuaJit 2. LuaJIT 2 is known to be fast, but I too was impressed by the numbers, outperforming HotSpot is pretty good, even if its just a tiny microbenchmark.
- copx 13y ago
  Seems it could be even faster: http://www.freelists.org/post/luajit/ANN-luje-01 http://www.freelists.org/post/luajit/ANN-luje-01
zeckalpha 13y ago
LuaJIT is heavily researched and optimized. http://article.gmane.org/gmane.comp.lang.lua.general/58908 http://article.gmane.org/gmane.comp.lang.lua.general/58908
pron 13y ago
HotSpot is indeed one of the most optimized runtimes around. It is, however, optimized for large, complex applications. It's quite possible to beat in a carefully designed microbenchmark. Beating it in a complex application is a whole other matter (in fact, it's pretty hard to beat even in C++).
- haberman 13y ago
  > in fact, it's pretty hard to beat even in C++ If there's one thing I've learned, it's that the wishful thinking will never stop.
  - dman 13y ago
    What?
    - deleted 13y ago
      [deleted]
  - peterashford 13y ago
    That's pretty snarky
    - haberman 13y ago
      It's really irritating, as someone who works in C and C++ with good reason, to hear people continually deny the very real performance benefits of working at a lower level. We (C/C++ guys) write the OS's, VMs, browsers, codecs, etc that power your software stacks, and then you (managed language fans) turn around and tell us we're wasting our time by using C and C++, while giving us stuff like Eclipse. So yes, when someone says it's hard to beat Java with C++, it inspires some snark. Write the next JVM in Java if you think it's that easy.
      - hermanhermitage 13y ago
        This is what happens when an important variable (the specific problem domain of performance comparison) is left as a free variable :-)
aaronblohowiak 13y ago
> In any non-trivial benchmark it usually manages about 50% to 75% of Hotspot.

jacques_chester 13y ago

David Given is one of those prolific thinkers and doers who seem to be overrepresented in the Lua community. He's probably best known to HNers for his critical article "On Go" and Objective Lua. http://cowlark.com/2009-11-15-go/ http://cowlark.com/2009-11-15-go/ https://cowlark.com/objective-lua/index.html https://cowlark.com/objective-lua/index.html

seabrookmx 13y ago
Thanks for the link! That's a really interesting article. Though out of date, I still think his point holds water. It's shocking how little difference there is between Algol-68 and modern Go or C. Go has changed quite a bit since his article, but not all of these changes have been well received (a great example being automatic semicolon insertion).

doublec 13y ago

There's an interesting paper about a similar project that was done using Self [1]. "Design and Implementation of Pep, a Java Just-In-Time Translator written in Self" It goes through details with benchmarks. The PEP code is available with the Self distribution. They were able to get faster than the Java implementation at the time. Back then Java was interpreted though. [1] http://bluishcoder.co.nz/self/97-pep.pdf http://bluishcoder.co.nz/self/97-pep.pdf

pjmlp 13y ago
Languages are nor interpreted or compiled. It is all about implementations.

taranka231 13y ago

is there any java/lua interop?

deleted 13y ago

[deleted]

hbbio 13y ago

It's an impressive project that shows the extraordinary work of a single developer, Mike Pall. LuaJIT is not only an incredibly beautiful JIT, the current project also demonstrates it can run generated code efficiently, which is probably the hardest thing to do for a language. When implementing Opa in OCaml, we run into tons of problem since the OCaml runtime was unable to run generated code efficiently. One of its author, Pierre Weis told us at the time: "it's normal, OCaml is not meant to run generated code at all". As a side note regarding Luje source code: Fossil must be great and all that but having the code on a publicly available Git repository would probably bolster contributions.

jsnell 13y ago
Luajit is of course awesome, but let's not get too carried away. It doesn't sound like this JVM has been used for running anything beyond extremely simple benchmarks. Of course you're not going to run into difficulties with the translation of a 15 line method. I saw a talk Mike gave on Luajit last week, and one thing he credited Luajit's success on was that it was working on a rather high abstraction layer. Stuff like the loop optimizations and alias analysis work because the optimizer can essentially understand the code at a fairly high level, and is somewhat specific to Lua. When dealing with java translated to jvm bytecode naively translated to lua source you lose much of that context. (And at the time he didn't seem too convinced about using luajit as the backend for other languages.) But even if you ignore the loss of context, I'm sure that various luajit limits would get hit with real life Java code. E.g. I'd guess there's a fundamental 65k limit for the number of constants and bytecode instructions in a single function.
- justincormack 13y ago
  JVM bytecode is quite nasty from this point of view; one of Mike's comments in the mailing list was to try Dalvik instead. It loses a lot of loop structure.
  - jonasb 13y ago
    http://www.freelists.org/post/luajit/ANN-luje-01 http://www.freelists.org/post/luajit/ANN-luje-01

JulianMorrison 13y ago

Isn't this going to be intrinsically restricted to one numeric type? Lua numbers are all the same type (double by default).

justincormack 13y ago
No, LuaJIT has an ffi interface with support for any type, and you can just use them as if they are native types.
- derleth 13y ago
  Seems a bit like cheating if you do it that way, though.
  - deleted 13y ago
    [deleted]
  - justincormack 13y ago
    Its not really, they are just native types, but defined with a C syntax so they are compatible with C. The code generation knows about them natively and optimises them.
derleth 13y ago
Eh, programs compiled to assembly language aren't limited to machine-word-sized integers and whatever the floating-point hardware (if any) natively supports. I'd be interested to know what its performance on integer code is, though.

swah 13y ago

Is LuaJIT bytecode a good target for toy languages?