9 ms·

Lua 5.4.0

256 points by smlckz 6y ago

ufo 6y ago
This release brings many performance improvements, including a new garbage collector. In terms of language features, the biggest new feature are the "to be closed" variables. They can be used to ensure that cleanup routines are always called, similarly to RAII in C++ or try-finally blocks.
- anonymoushn 6y ago
  <close> is equivalent to golang's defer (either can be implemented in terms of the other) except at the block level. imo calling it RAII mostly leads to confusion. One mailing list user recently asked a bunch of questions around using <close> for RAII. They expected RAII to work even for resources that are never bound to a particular type of local variable, for example if one writes do_thing(io.open("foo.txt")), where do_thing is not responsible for closing files because sometimes it is used with files that will continue to be used. They eventually concluded that the closest thing to RAII available was __gc. Some users presented separate complaints about resources kept in to-be-closed locals in coroutines not necessarily being closed. You can do something to try to solve this, like setting the __gc and __close metamethods of coroutines to call coroutine.close. A "real" solution would look more like dynamic-wind. Notably golang doesn't attempt to solve this one, so maybe not solving it is fine.
  - pansa2 6y ago
    > <close> is equivalent to golang's defer [...] except at the block level. Does that mean it’s the same as C#'s `using` and Python's `with`?
    - anonymoushn 6y ago
      I wasn't deeply familiar with python's `with`, so I looked it up[0] <close> differs from `with` in at least the sense that it doesn't have any equivalent to __enter__ and doesn't create its own block. It creates a local variable whose __close metamethod will be called when it goes out of scope. Since Lua has lexical scope at the block level rather than the function level, this works similarly to the way Python calls __exit__. These snippets of Python and Lua are roughly equivalent. They both open a file, read one line, and close it, or do nothing if there was some error opening the file. try: with open("foo.txt") as f: print(f.readline()) # f is now closed. except: pass local f,err = io.open("foo.txt") if not err then local f <close> = f print(f:read("*l")) end -- f is now closed. C#'s `using`[1] seems much closer, except that it handles nulls by not trying to close them and lua's <close> does not include any such handling. [0]: https://docs.python.org/3/reference/compound_stmts.html#the-with-statement https://docs.python.org/3/reference/compound_stmts.html#the-... [1]: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/statements#the-using-statement https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...
      - frabert 6y ago
        The reference manual https://www.lua.org/manual/5.4/manual.html#3.3.8 https://www.lua.org/manual/5.4/manual.html#3.3.8 seems to specify that "nil" and "false" values are ignored, so it behaves similarly to C#'s using.
        anonymoushn 6y ago
        Oh, that's great!
      - pull_my_finger 6y ago
        Do file descriptors come with a default __close metamethod or do you have to create your own?
        anonymoushn 6y ago
        They come with one.
catalogia 6y ago
[Unfortunately, IMHO] Lua "minor versions" tend to break compatibility with older versions, making it problematic to upgrade to new versions without breaking scripts written for older versions. I believe mpv is stuck on 5.2 due to this. Here's the list of incompatibilities for 5.4: https://www.lua.org/manual/5.4/manual.html#8 https://www.lua.org/manual/5.4/manual.html#8
- sitkack 6y ago
  Yeah, Lua doesn't use semver, which is ok. Just multiply by 10 and pretend it does. So this would be Lua 54.
  - pansa2 6y ago
    Yes, Lua doesn’t do “minor versions”. You can think of 5.4.0 as 54.0.0, and 5.3.5 as 53.0.5.
- topspin 6y ago
  Likewise OpenResty is "stuck" on 5.1.
  - Rochus 6y ago
    And doesn't seem to suffer from it.
  - bungle 6y ago
    "Stuck" might be a wrong word. OpenResty actually dropped support for Lua and is now LuaJIT only. LuaJIT is a fork of Lua 5.1, but it has features from 5.2 [1] (and from later versions where it sees fit). Because of this, Lua 5.1 is a golden standard of Lua world at the moment. [1] http://luajit.org/extensions.html#lua52 http://luajit.org/extensions.html#lua52
    - xvilka 6y ago
      There is more modern alternative to LuaJIT though - MoonJIT [1]. Unlike the former it supports modern Lua standards. [1] https://github.com/moonjit/moonjit https://github.com/moonjit/moonjit
shakna 6y ago
The new random library implementation should stop me dragging in a Mersenne Twister for every other thing. It was libc's random, which is... Trash. Now it use's xoshiro256 [a], which I hadn't heard of. (Not my area). [0] It seems somewhat similar to the MT, as it isn't cryptographically safe, but should give you a fast and fairly random set of integers back. It's also significantly faster than the MT. [a] Followed by two asterisks. I give up trying to make them appear in HN's formatting. [0] http://prng.di.unimi.it/ http://prng.di.unimi.it/
- nestorD 6y ago
  To give a very short summary, xoshiro256 is much faster but also has a much smaller period (however the period of the Mersenne Twister generator is way too large for most applications anyway).
- JoshuaRLi 6y ago
  Yeah, I was happy to see the switch away from C random/srandom or rand/srand. Not my area either, but a little surprised why a PCG-family non-CSPRNG algorithm wasn’t used: https://www.pcg-random.org/posts/a-quick-look-at-xoshiro256.html https://www.pcg-random.org/posts/a-quick-look-at-xoshiro256.... PCG XSL RR 128/64 (MCG) could be used for 64-bit values (albeit with period 2^126 compared to xoroshiro256’s 2^256 − 1) with 128 bits. Half the state size than xoshiro256, and the performance might be comparable (would need benchmarking). LCG version if you want to trade PRNG quality for even more speed.
anonymoushn 6y ago
One interesting change in this release is the move to a stackful VM. This creates a pretty small (edit: actually huge, 1978 stack frames in a lua 5.4 repl installed with luaver just now) limit on the depth of the Lua call stack. I wonder what motivated this decision.
- moonchild 6y ago
  Was it not a stack machine already? The embedding API explicitly exposes a stack, so I assumed that it was implemented that way.
  - anonymoushn 6y ago
    it had a stack, but that stack was on the heap. In lua 5.4 under normal circumstances, 20 frames of Lua call stack corresponds to 20ish frames of C call stack. In lua 5.3 the number of C stack frames was some smaller number unrelated to the depth of the Lua call stack.
    - moonchild 6y ago
      Ah - so it uses the C native stack.
- wahern 6y ago
  Citation? I don't think anything significant changed in that regard. Coroutines are still "stackless" in terms of the C stack, but "stackful" in terms of the language semantics. The virtual data stack is still clearly on the heap. Lua 5.4 adds a new function, lua_setcstacklimit, but this merely exposes what was a compile-time constant, LUAI_MAXCCALLS, in earlier versions. Lua can't avoid making some use of the C stack as it supports intermingling of lua_call's from C code with in-language VM calls, which necessarily will use some amount of C stack. Lua has lua_callk for yield/resuming across C code invocations, but not everybody makes use of this and in any event it's only for coroutine semantics, not for recursion.
  - shakna 6y ago
deleted 6y ago
[deleted]
amirmasoudabdol 6y ago
I just recently started using Lua through the sol2 library. I'm basically providing some flexibility through my input file to my users. I'm very surprised by both library and Lua itself. For my use, I don't feel a lot of performance hit, and gets a lot of flexibility.
loustak 6y ago
I really like lua. Especially when it comes to game with Love2d beeing so wonderful. I wish more projects adopted it as a scripting interface...
- slmjkdbtl 6y ago
  in my knowledge most tools with scripting support uses lua, there's not a lot of options (i'm also a big fan of love2d)
- donquichotte 6y ago
  Love2D is indeed excellent, and integrates well with the Tiled map editor. It makes making games almost as easy as Macromedia Flash did!
- pkukkapalli 6y ago
  I wanted to like Lua and Love2D, but struggled with writing code quickly because there's no editor autocompletion that I know of, so I spent a lot of time having to reference the docs. Do others just use it enough that they know the API by heart now?
pansa2 6y ago
It would be interesting to know how many projects are using regular Lua vs LuaJIT, as the two implementations have diverged. When both were on version 5.1, there didn't seem to be any downsides to LuaJIT. Many Lua projects could benefit from it as long as they weren't trying to run on a platform that disallows JIT compilation (e.g. iOS or a game console). However, now that LuaJIT is 2-3 language versions out-of-date, and has different best-practices (e.g. around FFI), it's almost like the two are different languages.
- kzrdude 6y ago
  Every major release of Lua has always been a different language, and embedding in general encourages not upgrading. If you ship something based on Lua 5.2, how to handle all existing user scripts if you upgrade Lua?
- otabdeveloper4 6y ago
  > When both were on version 5.1, there didn't seem to be any downsides to LuaJIT. LuaJIT has (had?) weird memory limits due to its GC implementation. This matters if you're running scripts on a server.
- upofadown 6y ago
  I think we do have to call this a legitimate language fork at this point. Prosody (XMPP server) runs on 5.2 and does not require LuaJIT. The LuaJIT thing is more of a symptom than a cause.
- fwsgonzo 6y ago
  Performance-wise Lua is not too far behind LuaJIT that you should just "never" use Lua. I used Lua for a long time without problems, and in the cases where you need that extra performance you are just one "system call" away from native performance for that specific operation. I used to benchmark these things (for my use cases) for a long time, and Lua was always somewhere between 50-100% slower than LuaJIT, which is NOT a lot. It's a mistake to think that people implement Mandelbrot as a script callback. :) It's unfortunate that LuaJIT fell behind so much, but as far as game engines who need a scripting language goes: You are either safe with Lua, or you have other alternatives: JavaScript, WASM, dynamic libraries and so on.
grillorafael 6y ago
"const variables" makes me giggle a little
pansa2 6y ago
There are many lightweight, embeddable scripting languages available, yet Lua remains dominant. This is despite the language being "quirky" at best (and "dreaded" at worst [0]). Is this just inertia, or is there a technical reason for new projects to embed Lua rather than a different language? [0] https://insights.stackoverflow.com/survey/2018#technology-_-most-loved-dreaded-and-wanted-languages https://insights.stackoverflow.com/survey/2018#technology-_-...
- IshKebab 6y ago
  It's very easy to embed, and LuaJit is pretty fast.
- TazeTSchnitzel 6y ago
  What better languages are there which are friendly to embedding? A significant part of Lua's popularity, AIUI, comes from it being easy to embed.
  - pansa2 6y ago
    Squirrel [0] seems to have been used in a few places [1], but not many compared to Lua. There are also embeddable JavaScript implementations such as Duktape [2], and a lightweight Ruby in mruby [3]. [0] http://www.squirrel-lang.org/ http://www.squirrel-lang.org/ [1] https://en.wikipedia.org/wiki/Squirrel_(programming_language)#Applications https://en.wikipedia.org/wiki/Squirrel_(programming_language... [2] https://duktape.org/ https://duktape.org/ [3] https://github.com/mruby/mruby https://github.com/mruby/mruby
  - bjoli 6y ago
    Guile scheme, if you have users that don't have the knee-jerk "ewww parentheses!" reaction. The threading model is vastly different from lua, but compared to regular lua you get a lot faster execution.
    - srean 6y ago
      Guile is great and I enjoy how active it has been. However, it is no more that small embeddable lisp. Its embeddable of course, but not small by any means.
ausjke 6y ago
many projects I am aware of stuck with 5.1 and luajit still and so do I. I still use lua5.1 myself these days.

256 points by smlckz 6y ago

ufo 6y ago

This release brings many performance improvements, including a new garbage collector. In terms of language features, the biggest new feature are the "to be closed" variables. They can be used to ensure that cleanup routines are always called, similarly to RAII in C++ or try-finally blocks.

anonymoushn 6y ago
<close> is equivalent to golang's defer (either can be implemented in terms of the other) except at the block level. imo calling it RAII mostly leads to confusion. One mailing list user recently asked a bunch of questions around using <close> for RAII. They expected RAII to work even for resources that are never bound to a particular type of local variable, for example if one writes do_thing(io.open("foo.txt")), where do_thing is not responsible for closing files because sometimes it is used with files that will continue to be used. They eventually concluded that the closest thing to RAII available was __gc. Some users presented separate complaints about resources kept in to-be-closed locals in coroutines not necessarily being closed. You can do something to try to solve this, like setting the __gc and __close metamethods of coroutines to call coroutine.close. A "real" solution would look more like dynamic-wind. Notably golang doesn't attempt to solve this one, so maybe not solving it is fine.
- pansa2 6y ago
  > <close> is equivalent to golang's defer [...] except at the block level. Does that mean it’s the same as C#'s `using` and Python's `with`?
  - anonymoushn 6y ago
    I wasn't deeply familiar with python's `with`, so I looked it up[0] <close> differs from `with` in at least the sense that it doesn't have any equivalent to __enter__ and doesn't create its own block. It creates a local variable whose __close metamethod will be called when it goes out of scope. Since Lua has lexical scope at the block level rather than the function level, this works similarly to the way Python calls __exit__. These snippets of Python and Lua are roughly equivalent. They both open a file, read one line, and close it, or do nothing if there was some error opening the file. try: with open("foo.txt") as f: print(f.readline()) # f is now closed. except: pass local f,err = io.open("foo.txt") if not err then local f <close> = f print(f:read("*l")) end -- f is now closed. C#'s `using`[1] seems much closer, except that it handles nulls by not trying to close them and lua's <close> does not include any such handling. [0]: https://docs.python.org/3/reference/compound_stmts.html#the-with-statement https://docs.python.org/3/reference/compound_stmts.html#the-... [1]: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/statements#the-using-statement https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...
    - frabert 6y ago
      The reference manual https://www.lua.org/manual/5.4/manual.html#3.3.8 https://www.lua.org/manual/5.4/manual.html#3.3.8 seems to specify that "nil" and "false" values are ignored, so it behaves similarly to C#'s using.
      - anonymoushn 6y ago
        Oh, that's great!
    - pull_my_finger 6y ago
      Do file descriptors come with a default __close metamethod or do you have to create your own?
      - anonymoushn 6y ago
        They come with one.

catalogia 6y ago

[Unfortunately, IMHO] Lua "minor versions" tend to break compatibility with older versions, making it problematic to upgrade to new versions without breaking scripts written for older versions. I believe mpv is stuck on 5.2 due to this. Here's the list of incompatibilities for 5.4: https://www.lua.org/manual/5.4/manual.html#8 https://www.lua.org/manual/5.4/manual.html#8

sitkack 6y ago
Yeah, Lua doesn't use semver, which is ok. Just multiply by 10 and pretend it does. So this would be Lua 54.
- pansa2 6y ago
  Yes, Lua doesn’t do “minor versions”. You can think of 5.4.0 as 54.0.0, and 5.3.5 as 53.0.5.
topspin 6y ago
Likewise OpenResty is "stuck" on 5.1.
- Rochus 6y ago
  And doesn't seem to suffer from it.
- bungle 6y ago
  "Stuck" might be a wrong word. OpenResty actually dropped support for Lua and is now LuaJIT only. LuaJIT is a fork of Lua 5.1, but it has features from 5.2 [1] (and from later versions where it sees fit). Because of this, Lua 5.1 is a golden standard of Lua world at the moment. [1] http://luajit.org/extensions.html#lua52 http://luajit.org/extensions.html#lua52
  - xvilka 6y ago
    There is more modern alternative to LuaJIT though - MoonJIT [1]. Unlike the former it supports modern Lua standards. [1] https://github.com/moonjit/moonjit https://github.com/moonjit/moonjit

shakna 6y ago

The new random library implementation should stop me dragging in a Mersenne Twister for every other thing. It was libc's random, which is... Trash. Now it use's xoshiro256 [a], which I hadn't heard of. (Not my area). [0] It seems somewhat similar to the MT, as it isn't cryptographically safe, but should give you a fast and fairly random set of integers back. It's also significantly faster than the MT. [a] Followed by two asterisks. I give up trying to make them appear in HN's formatting. [0] http://prng.di.unimi.it/ http://prng.di.unimi.it/

nestorD 6y ago
To give a very short summary, xoshiro256 is much faster but also has a much smaller period (however the period of the Mersenne Twister generator is way too large for most applications anyway).
JoshuaRLi 6y ago
Yeah, I was happy to see the switch away from C random/srandom or rand/srand. Not my area either, but a little surprised why a PCG-family non-CSPRNG algorithm wasn’t used: https://www.pcg-random.org/posts/a-quick-look-at-xoshiro256.html https://www.pcg-random.org/posts/a-quick-look-at-xoshiro256.... PCG XSL RR 128/64 (MCG) could be used for 64-bit values (albeit with period 2^126 compared to xoroshiro256’s 2^256 − 1) with 128 bits. Half the state size than xoshiro256, and the performance might be comparable (would need benchmarking). LCG version if you want to trade PRNG quality for even more speed.

anonymoushn 6y ago

One interesting change in this release is the move to a stackful VM. This creates a pretty small (edit: actually huge, 1978 stack frames in a lua 5.4 repl installed with luaver just now) limit on the depth of the Lua call stack. I wonder what motivated this decision.

moonchild 6y ago
Was it not a stack machine already? The embedding API explicitly exposes a stack, so I assumed that it was implemented that way.
- anonymoushn 6y ago
  it had a stack, but that stack was on the heap. In lua 5.4 under normal circumstances, 20 frames of Lua call stack corresponds to 20ish frames of C call stack. In lua 5.3 the number of C stack frames was some smaller number unrelated to the depth of the Lua call stack.
  - moonchild 6y ago
    Ah - so it uses the C native stack.
wahern 6y ago
Citation? I don't think anything significant changed in that regard. Coroutines are still "stackless" in terms of the C stack, but "stackful" in terms of the language semantics. The virtual data stack is still clearly on the heap. Lua 5.4 adds a new function, lua_setcstacklimit, but this merely exposes what was a compile-time constant, LUAI_MAXCCALLS, in earlier versions. Lua can't avoid making some use of the C stack as it supports intermingling of lua_call's from C code with in-language VM calls, which necessarily will use some amount of C stack. Lua has lua_callk for yield/resuming across C code invocations, but not everybody makes use of this and in any event it's only for coroutine semantics, not for recursion.
- shakna 6y ago

deleted 6y ago

[deleted]

amirmasoudabdol 6y ago

I just recently started using Lua through the sol2 library. I'm basically providing some flexibility through my input file to my users. I'm very surprised by both library and Lua itself. For my use, I don't feel a lot of performance hit, and gets a lot of flexibility.

loustak 6y ago

I really like lua. Especially when it comes to game with Love2d beeing so wonderful. I wish more projects adopted it as a scripting interface...

slmjkdbtl 6y ago
in my knowledge most tools with scripting support uses lua, there's not a lot of options (i'm also a big fan of love2d)
donquichotte 6y ago
Love2D is indeed excellent, and integrates well with the Tiled map editor. It makes making games almost as easy as Macromedia Flash did!
pkukkapalli 6y ago
I wanted to like Lua and Love2D, but struggled with writing code quickly because there's no editor autocompletion that I know of, so I spent a lot of time having to reference the docs. Do others just use it enough that they know the API by heart now?

pansa2 6y ago

It would be interesting to know how many projects are using regular Lua vs LuaJIT, as the two implementations have diverged. When both were on version 5.1, there didn't seem to be any downsides to LuaJIT. Many Lua projects could benefit from it as long as they weren't trying to run on a platform that disallows JIT compilation (e.g. iOS or a game console). However, now that LuaJIT is 2-3 language versions out-of-date, and has different best-practices (e.g. around FFI), it's almost like the two are different languages.

kzrdude 6y ago
Every major release of Lua has always been a different language, and embedding in general encourages not upgrading. If you ship something based on Lua 5.2, how to handle all existing user scripts if you upgrade Lua?
otabdeveloper4 6y ago
> When both were on version 5.1, there didn't seem to be any downsides to LuaJIT. LuaJIT has (had?) weird memory limits due to its GC implementation. This matters if you're running scripts on a server.
upofadown 6y ago
I think we do have to call this a legitimate language fork at this point. Prosody (XMPP server) runs on 5.2 and does not require LuaJIT. The LuaJIT thing is more of a symptom than a cause.
fwsgonzo 6y ago
Performance-wise Lua is not too far behind LuaJIT that you should just "never" use Lua. I used Lua for a long time without problems, and in the cases where you need that extra performance you are just one "system call" away from native performance for that specific operation. I used to benchmark these things (for my use cases) for a long time, and Lua was always somewhere between 50-100% slower than LuaJIT, which is NOT a lot. It's a mistake to think that people implement Mandelbrot as a script callback. :) It's unfortunate that LuaJIT fell behind so much, but as far as game engines who need a scripting language goes: You are either safe with Lua, or you have other alternatives: JavaScript, WASM, dynamic libraries and so on.

grillorafael 6y ago

"const variables" makes me giggle a little

pansa2 6y ago

There are many lightweight, embeddable scripting languages available, yet Lua remains dominant. This is despite the language being "quirky" at best (and "dreaded" at worst [0]). Is this just inertia, or is there a technical reason for new projects to embed Lua rather than a different language? [0] https://insights.stackoverflow.com/survey/2018#technology-_-most-loved-dreaded-and-wanted-languages https://insights.stackoverflow.com/survey/2018#technology-_-...

IshKebab 6y ago
It's very easy to embed, and LuaJit is pretty fast.
TazeTSchnitzel 6y ago
What better languages are there which are friendly to embedding? A significant part of Lua's popularity, AIUI, comes from it being easy to embed.
- pansa2 6y ago
  Squirrel [0] seems to have been used in a few places [1], but not many compared to Lua. There are also embeddable JavaScript implementations such as Duktape [2], and a lightweight Ruby in mruby [3]. [0] http://www.squirrel-lang.org/ http://www.squirrel-lang.org/ [1] https://en.wikipedia.org/wiki/Squirrel_(programming_language)#Applications https://en.wikipedia.org/wiki/Squirrel_(programming_language... [2] https://duktape.org/ https://duktape.org/ [3] https://github.com/mruby/mruby https://github.com/mruby/mruby
- bjoli 6y ago
  Guile scheme, if you have users that don't have the knee-jerk "ewww parentheses!" reaction. The threading model is vastly different from lua, but compared to regular lua you get a lot faster execution.
  - srean 6y ago
    Guile is great and I enjoy how active it has been. However, it is no more that small embeddable lisp. Its embeddable of course, but not small by any means.

ausjke 6y ago

many projects I am aware of stuck with 5.1 and luajit still and so do I. I still use lua5.1 myself these days.