7 ms·

I really wish I could use LuaJIT, but unfortunately the memory limitations of it means I can't (I'm aware of FFI).

copx 13y ago
Well if manually managed memory allocated through the FFI is unacceptable to you for some reason you are out of luck indeed. Well, the limit is per process but I guess distributing the task across multiple processes is not acceptable either. By the way, it is worth pointing out that Mike Pall (LuaJIT's author) has said that the current Lua garbage collector is not designed to handle such big heaps efficiently anyway.
- snogglethorpe 13y ago
  There are real applications where the GC limitations aren't a problem, but LuaJIT's addressing limitations are. The one I've run into is using extremely large strings (hundreds of megabytes per string). I've encountered cases where LuaJIT runs out of memory, but standard Lua works fine (and is quite fast). So while Mike Pall may correct about the behavior of the GC, the implication that this means LuaJIT's addressing limitations aren't a problem in practice is not correct. Given LuaJIT's limitations (not just this, but also the more fundamental issue of portability), the defacto forking of the language is worrying...
cauterize 13y ago
Could you expand on the memory limitations? I'm not familiar with them.
- deutronium 13y ago
  Sure, It's limited to using a maximum of 2GB of memory if I recall correctly, as far as I'm aware it's mainly due to the 32 bit pointers that are used along with the garbage collector.
  - agumonkey 13y ago
    With lua I always assume people are talking about downward limitations not upward.
    - deutronium 13y ago
      Yes, that is a good point, I'm planning on using lua though on a wide range of hardware ranging from MCUs through to Servers.
  - thinkpad20 13y ago
    Is that because they're using one of the 32 bits as a boolean to mark if it should be collected? Just curious.
justincormack 13y ago
You really need more than 2G of heap? And can't use ffi? There were some posts to the mailing list that seemed to have a genuine issue, but most use cases do not.
- deutronium 13y ago
  Yes, I've posted on the mailing list regarding it before.
  - justincormack 13y ago
    Ah ok that was you...
- malkia 13y ago
  2GB (for 64-bit apps) by itself can be mitigated by using ffi-allocated memory. The problem is really if you write a plugin (OSX) using luajit, and you can't change the main app to not use any memory below 4GB (where the heap should be). This may be a valid technical concern for certain products (plugins), but not a real limitation for others. And so far it's been mostly on OSX.

copx 13y ago

Well if manually managed memory allocated through the FFI is unacceptable to you for some reason you are out of luck indeed. Well, the limit is per process but I guess distributing the task across multiple processes is not acceptable either. By the way, it is worth pointing out that Mike Pall (LuaJIT's author) has said that the current Lua garbage collector is not designed to handle such big heaps efficiently anyway.

snogglethorpe 13y ago
There are real applications where the GC limitations aren't a problem, but LuaJIT's addressing limitations are. The one I've run into is using extremely large strings (hundreds of megabytes per string). I've encountered cases where LuaJIT runs out of memory, but standard Lua works fine (and is quite fast). So while Mike Pall may correct about the behavior of the GC, the implication that this means LuaJIT's addressing limitations aren't a problem in practice is not correct. Given LuaJIT's limitations (not just this, but also the more fundamental issue of portability), the defacto forking of the language is worrying...

cauterize 13y ago

Could you expand on the memory limitations? I'm not familiar with them.

deutronium 13y ago
Sure, It's limited to using a maximum of 2GB of memory if I recall correctly, as far as I'm aware it's mainly due to the 32 bit pointers that are used along with the garbage collector.
- agumonkey 13y ago
  With lua I always assume people are talking about downward limitations not upward.
  - deutronium 13y ago
    Yes, that is a good point, I'm planning on using lua though on a wide range of hardware ranging from MCUs through to Servers.
- thinkpad20 13y ago
  Is that because they're using one of the 32 bits as a boolean to mark if it should be collected? Just curious.

justincormack 13y ago

You really need more than 2G of heap? And can't use ffi? There were some posts to the mailing list that seemed to have a genuine issue, but most use cases do not.

deutronium 13y ago
Yes, I've posted on the mailing list regarding it before.
- justincormack 13y ago
  Ah ok that was you...
malkia 13y ago
2GB (for 64-bit apps) by itself can be mitigated by using ffi-allocated memory. The problem is really if you write a plugin (OSX) using luajit, and you can't change the main app to not use any memory below 4GB (where the heap should be). This may be a valid technical concern for certain products (plugins), but not a real limitation for others. And so far it's been mostly on OSX.