7 ms·

LuaJIT 2.0.3

110 points by kul_ 13y ago

deutronium 13y ago
I really wish I could use LuaJIT, but unfortunately the memory limitations of it means I can't (I'm aware of FFI).
- copx 13y ago
  Well if manually managed memory allocated through the FFI is unacceptable to you for some reason you are out of luck indeed. Well, the limit is per process but I guess distributing the task across multiple processes is not acceptable either. By the way, it is worth pointing out that Mike Pall (LuaJIT's author) has said that the current Lua garbage collector is not designed to handle such big heaps efficiently anyway.
  - snogglethorpe 13y ago
    There are real applications where the GC limitations aren't a problem, but LuaJIT's addressing limitations are. The one I've run into is using extremely large strings (hundreds of megabytes per string). I've encountered cases where LuaJIT runs out of memory, but standard Lua works fine (and is quite fast). So while Mike Pall may correct about the behavior of the GC, the implication that this means LuaJIT's addressing limitations aren't a problem in practice is not correct. Given LuaJIT's limitations (not just this, but also the more fundamental issue of portability), the defacto forking of the language is worrying...
- cauterize 13y ago
  Could you expand on the memory limitations? I'm not familiar with them.
  - deutronium 13y ago
    Sure, It's limited to using a maximum of 2GB of memory if I recall correctly, as far as I'm aware it's mainly due to the 32 bit pointers that are used along with the garbage collector.
    - agumonkey 13y ago
      With lua I always assume people are talking about downward limitations not upward.
      - deutronium 13y ago
        Yes, that is a good point, I'm planning on using lua though on a wide range of hardware ranging from MCUs through to Servers.
    - thinkpad20 13y ago
      Is that because they're using one of the 32 bits as a boolean to mark if it should be collected? Just curious.
- justincormack 13y ago
  You really need more than 2G of heap? And can't use ffi? There were some posts to the mailing list that seemed to have a genuine issue, but most use cases do not.
  - deutronium 13y ago
    Yes, I've posted on the mailing list regarding it before.
    - justincormack 13y ago
      Ah ok that was you...
  - malkia 13y ago
    2GB (for 64-bit apps) by itself can be mitigated by using ffi-allocated memory. The problem is really if you write a plugin (OSX) using luajit, and you can't change the main app to not use any memory below 4GB (where the heap should be). This may be a valid technical concern for certain products (plugins), but not a real limitation for others. And so far it's been mostly on OSX.
pavanky 13y ago
Curios how many people here actually use Lua ? What are the applications you are using it for ? I know it is used a bit in the gaming industry for scripting purposes, but I am not aware of anything else. We use it internally to map some of the functions in our library to OpenGL.
- kolev 13y ago
  I use it within Nginx.
- OP_JMP 13y ago
  Please do not confuse Lua with LuaJIT. Whilst LuaJIT claims to be ABI compatible with Lua 5.1, 5.1 is not the latest release of Lua. There are some in the Lua community that consider these two different languages or at least that JIT is a fork. https://sites.google.com/site/marbux/home/where-lua-is-used https://sites.google.com/site/marbux/home/where-lua-is-used
  - malkia 13y ago
    I'm not following. LuaJIT is 5.1 compatible, with some of the 5.2 features in (compile enabled with LUAJIT_ENABLE_COMPAT52). Just because it's not the latest release of Lua, it does not make it less Lua language. Yes, bytecode generated by lua reference implementation can't be loaded by luajit (or maybe things have changed since), but that's just like saying Microsoft C++ libs can't be used by MINGW.
    - OP_JMP 13y ago
      You may want to inform Mike Pall that it is 5.1 compatible then. Every time someone on the list says something to the effect that they have just replaced the dll/lib, he informs them that they are doing it wrong and instead they want to do it this way or that way with LuaJIT. The FFI has become such a core feature of LuaJIT now that as soon as you start using it you have already discounted ever using Lua again. So let us be honest, LuaJIT has split from Lua and is now a separate language which Mike says he will not update to support the latest version and keeps adding none compatible features.
just2n 13y ago
Do most games that use Lua for scripting (addons, UI, possibly game logic) actually use LuaJIT? As far as I could tell, 100% of the AAA titles that use it that I tested (3 in total) did not. And in profiling, Lua was actually consuming a lot of CPU relatively. That confuses me given how easy it is to use LuaJIT over the reference, and how vital performance in these environments is. Games, contrary to the beliefs of dense hiring managers, disconnected producers, and even designers, are performance critical applications -- achieving a mere 30 FPS and > XX+ms input latency on hardware 5 years in the future in the enthusiast category, on the order of $4000+, is an absolute failure. Every gamer I've ever met (thousands, thousands upon thousands, hundreds that I know well) is extremely sensitive to performance. Performance is king. If a game looks a little shitty but runs AMAZING, it is better than a game that looks AMAZING but runs a little shitty. Hell, it's even better than a game that looks WTFAMAZING and runs pretty well. We only start caring about the looks, typically, when performance is sufficient. So if we can dial the game's graphics up while maintaining performance, we get happy. Enthusiasts love it when they can dial all the way to max settings and still play the game at 60+ fps and unnoticeable input lag. So running your render loop and lua on the same thread (a WTF in itself) and not using LuaJIT is a huge WTF.
- justincormack 13y ago
  I think people are gradually switching over. There may be some porting involved, people may have lots of Lua code for older versions. Plus depending what you are doing the JIT may not speed things up (and you can't use it on many consoles); the 2.1 has a lot of string functions accelerated that would break jitting in 2.0. But someone has just paid for a PS4 port, so clearly many people are using it...
  - tomlu 13y ago
    It has worked on PS3 + Xbox360 for some years now and is significantly faster than vanilla Lua. I've no idea why more games haven't picked it up.
    - pjmlp 13y ago
mjcohen 13y ago
And then there is LuaTex: "LuaTeX is an extended version of pdfTeX using Lua as an embedded scripting language." See luatex.org.

LuaJIT 2.0.3

110 points by kul_ 13y ago

deutronium 13y ago

I really wish I could use LuaJIT, but unfortunately the memory limitations of it means I can't (I'm aware of FFI).

copx 13y ago
Well if manually managed memory allocated through the FFI is unacceptable to you for some reason you are out of luck indeed. Well, the limit is per process but I guess distributing the task across multiple processes is not acceptable either. By the way, it is worth pointing out that Mike Pall (LuaJIT's author) has said that the current Lua garbage collector is not designed to handle such big heaps efficiently anyway.
- snogglethorpe 13y ago
  There are real applications where the GC limitations aren't a problem, but LuaJIT's addressing limitations are. The one I've run into is using extremely large strings (hundreds of megabytes per string). I've encountered cases where LuaJIT runs out of memory, but standard Lua works fine (and is quite fast). So while Mike Pall may correct about the behavior of the GC, the implication that this means LuaJIT's addressing limitations aren't a problem in practice is not correct. Given LuaJIT's limitations (not just this, but also the more fundamental issue of portability), the defacto forking of the language is worrying...
cauterize 13y ago
Could you expand on the memory limitations? I'm not familiar with them.
- deutronium 13y ago
  Sure, It's limited to using a maximum of 2GB of memory if I recall correctly, as far as I'm aware it's mainly due to the 32 bit pointers that are used along with the garbage collector.
  - agumonkey 13y ago
    With lua I always assume people are talking about downward limitations not upward.
    - deutronium 13y ago
      Yes, that is a good point, I'm planning on using lua though on a wide range of hardware ranging from MCUs through to Servers.
  - thinkpad20 13y ago
    Is that because they're using one of the 32 bits as a boolean to mark if it should be collected? Just curious.
justincormack 13y ago
You really need more than 2G of heap? And can't use ffi? There were some posts to the mailing list that seemed to have a genuine issue, but most use cases do not.
- deutronium 13y ago
  Yes, I've posted on the mailing list regarding it before.
  - justincormack 13y ago
    Ah ok that was you...
- malkia 13y ago
  2GB (for 64-bit apps) by itself can be mitigated by using ffi-allocated memory. The problem is really if you write a plugin (OSX) using luajit, and you can't change the main app to not use any memory below 4GB (where the heap should be). This may be a valid technical concern for certain products (plugins), but not a real limitation for others. And so far it's been mostly on OSX.

pavanky 13y ago

Curios how many people here actually use Lua ? What are the applications you are using it for ? I know it is used a bit in the gaming industry for scripting purposes, but I am not aware of anything else. We use it internally to map some of the functions in our library to OpenGL.

kolev 13y ago
I use it within Nginx.
OP_JMP 13y ago
Please do not confuse Lua with LuaJIT. Whilst LuaJIT claims to be ABI compatible with Lua 5.1, 5.1 is not the latest release of Lua. There are some in the Lua community that consider these two different languages or at least that JIT is a fork. https://sites.google.com/site/marbux/home/where-lua-is-used https://sites.google.com/site/marbux/home/where-lua-is-used
- malkia 13y ago
  I'm not following. LuaJIT is 5.1 compatible, with some of the 5.2 features in (compile enabled with LUAJIT_ENABLE_COMPAT52). Just because it's not the latest release of Lua, it does not make it less Lua language. Yes, bytecode generated by lua reference implementation can't be loaded by luajit (or maybe things have changed since), but that's just like saying Microsoft C++ libs can't be used by MINGW.
  - OP_JMP 13y ago
    You may want to inform Mike Pall that it is 5.1 compatible then. Every time someone on the list says something to the effect that they have just replaced the dll/lib, he informs them that they are doing it wrong and instead they want to do it this way or that way with LuaJIT. The FFI has become such a core feature of LuaJIT now that as soon as you start using it you have already discounted ever using Lua again. So let us be honest, LuaJIT has split from Lua and is now a separate language which Mike says he will not update to support the latest version and keeps adding none compatible features.

just2n 13y ago

Do most games that use Lua for scripting (addons, UI, possibly game logic) actually use LuaJIT? As far as I could tell, 100% of the AAA titles that use it that I tested (3 in total) did not. And in profiling, Lua was actually consuming a lot of CPU relatively. That confuses me given how easy it is to use LuaJIT over the reference, and how vital performance in these environments is. Games, contrary to the beliefs of dense hiring managers, disconnected producers, and even designers, are performance critical applications -- achieving a mere 30 FPS and > XX+ms input latency on hardware 5 years in the future in the enthusiast category, on the order of $4000+, is an absolute failure. Every gamer I've ever met (thousands, thousands upon thousands, hundreds that I know well) is extremely sensitive to performance. Performance is king. If a game looks a little shitty but runs AMAZING, it is better than a game that looks AMAZING but runs a little shitty. Hell, it's even better than a game that looks WTFAMAZING and runs pretty well. We only start caring about the looks, typically, when performance is sufficient. So if we can dial the game's graphics up while maintaining performance, we get happy. Enthusiasts love it when they can dial all the way to max settings and still play the game at 60+ fps and unnoticeable input lag. So running your render loop and lua on the same thread (a WTF in itself) and not using LuaJIT is a huge WTF.

justincormack 13y ago
I think people are gradually switching over. There may be some porting involved, people may have lots of Lua code for older versions. Plus depending what you are doing the JIT may not speed things up (and you can't use it on many consoles); the 2.1 has a lot of string functions accelerated that would break jitting in 2.0. But someone has just paid for a PS4 port, so clearly many people are using it...
- tomlu 13y ago
  It has worked on PS3 + Xbox360 for some years now and is significantly faster than vanilla Lua. I've no idea why more games haven't picked it up.
  - pjmlp 13y ago

mjcohen 13y ago

And then there is LuaTex: "LuaTeX is an extended version of pdfTeX using Lua as an embedded scripting language." See luatex.org.