4 ms·

I've been diving into Lua (a little late to this party, but turns out it's a perfect language to rewrite some commandline scripts I had that were getting unwiel

I've been diving into Lua (a little late to this party, but turns out it's a perfect language to rewrite some commandline scripts I had that were getting unwieldy in Bash, especially with LLM assistance!) and it's really something of an eye-opener. LuaJITted Lua code runs at 80% (on average, sometimes faster!) of the compiled C version of the same algorithm, typically. Lua is embedded in a surprisingly massive number of products: https://en.wikipedia.org/wiki/List_of_applications_using_Lua https://en.wikipedia.org/wiki/List_of_applications_using_Lua The startup time of a script is in nanoseconds. An "echo" written in Lua runs faster than the native echo implementation. The only warts so far are 1-based indexing (you get used to it), and the fact that LuaJIT is stuck at Lua 5.1 while Lua itself is up to 5.3 or 5.4 and has added some niceties... with Lua proper running slower. And no real standard library to speak of (although some would argue that's a feature; there are a few options and different flavors out there if that's what you need, though- Such as functional-flavored ones...) Anyway, there's nothing else like it out there. Especially with its relative simplicity. There are also some neat languages that compile to (transpile to?) Lua, and deserve more attention, such as YueScript https://yuescript.org/ https://yuescript.org/, which is a still actively-updated enhanced dialect of MoonScript https://moonscript.org/ https://moonscript.org/ (described as "Coffeescript for Lua", although it hasn't been updated in 10 years) although neither of these are typed. HOWEVER... there IS this: TypescriptToLua https://typescripttolua.github.io/ https://typescripttolua.github.io/, which takes advantage of ALL the existing TypeScript tooling, it just outputs Lua instead of JS!

kanbankaren 1y ago
It is true that LuaJIT is stuck at 5.1, but you could write any performance critical sections in C/C++ and call it from Lua. Lack of LuaJIT for 5.1+ isn't that big of a deal for desktop apps. The embedded world is still stuck in 5.1, but for them, the benefits of the latest Lua is marginal.
- vrighter 1y ago
  and if you use luajit ffi, those calls actually get called just as fast as from a c program
- johnisgood 1y ago
  And despite it being stuck at 5.1, it still implements features from other versions. For example, there is the "LJ_52" macro, so you can compile "table.pack" and "table.unpack" into LuaJIT, which I do, because I use both at times. As someone else have pointed it out, they are cherry picked: https://luajit.org/extensions.html https://luajit.org/extensions.html.
  - hisham_hm 1y ago
    There is also the compat53 library which polyfills most of the missing parts. The Teal compiler has --gen-target and --gen-compat flags which adapts the generated Lua code for different Lua versions, and allows using the compat53 library behind the scenes if desired, so you can get a mostly Lua-5.3+ experience over LuaJIT using Teal.
jiehong 1y ago
Any recommendations going from bash to lua to watch out for except indexing?
- pmarreck 1y ago
  I haven’t found many downsides yet. I was already starting to rewrite some things in Awk (which I was drawn to for similar reasons- fast script startup time, simple easy language with good defaults), but Awk (while still also awesome) isn’t really designed for stuff beyond a certain size (no signal handling unless you use a fork, for example) LuaJIT is missing bignums, ints that aren’t floats, bit operations, and native utf8 handling, but it can pretty easily be extended with libraries, ffi and metatabling. (I actually made a working bignum library that integrates with gmp, but it has a memory leak somewhere and it’s a rabbit hole/bikeshedding project at this point…) LLM assistance helps hugely and I really like YueScript’s syntax additions. You can point any LLM at a syntax describing webpage and it will pretty much write that language for you…
  - nmz 1y ago
    For bignums, have you tried this? https://web.tecgraf.puc-rio.br/~lhf/ftp/lua/#limath https://web.tecgraf.puc-rio.br/~lhf/ftp/lua/#limath luajit does have integers (try 5ull+6), bit operations via the bit library (absorbed from lua5.2)
    - pmarreck 1y ago
      How does imath compare to gmp?
      - nmz 1y ago
        In lua? It works for one.
- nmz 1y ago
  I do think you're better off using ruby, but if you insist. Lots of default functionality missing so you MUST have these packages: inspect, luaposix, lrexlib-pcre, lrexlib-posix, lpeg, luastd/stdlib, luasocket, luahttp, luasec, luacheck, penlight * luajit is unnecessary in almost all cases, you don't need the speed. * use lsp or luacheck whenever you write something, entr -c luacheck file on everything. * patterns are not regex which means they do not support lookups, backtracking or |, so you must install lrexlib-pcre or lrexlib-posix (frankly I never need pcre so I stick to lrexlib-gnu or lrexlib-posix). * overload _ENV so it auto requires unknown things, I have a lua wrapper that does this and it makes it a joy not having all of my scripts with a bunch of require"posix" on all of them * install inspect to inspect tables * os.execute and io.popen only accepts strings as parameters which means you should overload it and make a function that accepts tables as well. * 5.4 is still lacking support for many libraries, 5.3 has most of the libraries. * assignments default to the global environment so you have to use local keyword or set _ENV to error on assignment (or better yet, don't care, just local _ENV = mymodule) Overall, Lua is just a mixture of C with a pascal syntax and garbage collection (and also tables which is a weird data structure)
Rochus 1y ago
> LuaJITted Lua code runs at 80% (on average, sometimes faster!) of the compiled C version of the same algorithm, typically Cannot confirm this. It might be true on selected micro benchmarks. Here are the results of the Are-we-fast-yet benchmark suite, which includes a decent set of benchmarks challenging CPU, cache and memory access: https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf https://github.com/rochus-keller/Oberon/blob/master/testcase.... On average, the C and C++ implementations are five times faster than LuaJIT. > There are also some neat languages that compile to (transpile to?) Lua Here is a comprehensive list: https://github.com/hengestone/lua-languages https://github.com/hengestone/lua-languages. Lanuages like Oberon or Luon directly compile to LuaJIT bytecode (i.e. not to Lua).
- Symmetry 1y ago
  The code in Are-we-fast-yet has been heavily optimized. It might still be true that naive LuaJIT can run almost as fast as Naive C.
  - Rochus 1y ago
    Have a look at the results and the code; there are benchmarks in the suite where the (ideomatic) C/C++ implementation is "only" twice as fast as the corresponding (idiomatic) Lua implementation, but on average (geomean of all factors) it's about five times as fast. The guidelines of the benchmark are pretty strict to enable fair comparisons (see https://github.com/smarr/are-we-fast-yet/blob/master/docs/guidelines.md https://github.com/smarr/are-we-fast-yet/blob/master/docs/gu...).
    - pmarreck 1y ago
      I looked at the docs. This seems to be comparing against Lua, which is why I specifically said LuaJIT This is quite a distinction to be made. Can you clarify? Directly from their guidelines page: Lua We write code compatible with Lua 5.1, 5.2 and 5.3. Smalltalk/Ruby symbols are represented as normal strings. We use Lua 1-based array and the length operator #. We use single object when a class is not required. Bitwise operators with various Lua versions is a nightmare. We use luacheck as a linter. If they are writing code compatible with Lua 5.2 or 5.3, then that cannot be LuaJIT, which is ONLY compatible with Lua 5.1. (Unless they mean that they JUST write 5.1 code, which due to backwards compatibility is runnable on 5.2 and 5.3? It's unclear from here.)
jwatte 1y ago
Another one of the biggest uses of Lua outside the hyperscaler-type software like nginx or redis: Roblox. Soooo many kids run games on Roblox every day! Roblox not only runs entirely on Lua, but they've been working on their own type inference version of Lua named Luau, and open sourced it, and it's still in very active development. https://github.com/luau-lang/luau https://github.com/luau-lang/luau
const_cast 1y ago
> but turns out it's a perfect language to rewrite some commandline scripts I had that were getting unwieldy in Bash IMO this is the perfect use case for Perl. I mean, it's why Perl was invented, but to this day it remains incredibly good at doing this specific task. It's incredibly easy to write and can be very easy to read, and it's much more robust than bash. But best of all - and the reason I think Perl still has a place in the modern world - it's available on practically every computer on Earth. And it will all work. It's so backwards-compatible that 25 year old scripts will run just fine. It's so portable that nothing even comes close. Also, it's shockingly fast. Faster than you would think. Regex is really powerful and it's been the template for so many regex implementations, but definitely be careful of runaway time complexity. Anyway, Lua is great too. Really nice for embedding into applications ala VBA. Great for config.
- pmarreck 1y ago
  Good point about Perl. Why don't I consider Perl? I don't know.
  - const_cast 1y ago
    Don’t worry, nobody considers Perl

kanbankaren 1y ago

It is true that LuaJIT is stuck at 5.1, but you could write any performance critical sections in C/C++ and call it from Lua. Lack of LuaJIT for 5.1+ isn't that big of a deal for desktop apps. The embedded world is still stuck in 5.1, but for them, the benefits of the latest Lua is marginal.

vrighter 1y ago
and if you use luajit ffi, those calls actually get called just as fast as from a c program
johnisgood 1y ago
And despite it being stuck at 5.1, it still implements features from other versions. For example, there is the "LJ_52" macro, so you can compile "table.pack" and "table.unpack" into LuaJIT, which I do, because I use both at times. As someone else have pointed it out, they are cherry picked: https://luajit.org/extensions.html https://luajit.org/extensions.html.
- hisham_hm 1y ago
  There is also the compat53 library which polyfills most of the missing parts. The Teal compiler has --gen-target and --gen-compat flags which adapts the generated Lua code for different Lua versions, and allows using the compat53 library behind the scenes if desired, so you can get a mostly Lua-5.3+ experience over LuaJIT using Teal.

jiehong 1y ago

Any recommendations going from bash to lua to watch out for except indexing?

pmarreck 1y ago
I haven’t found many downsides yet. I was already starting to rewrite some things in Awk (which I was drawn to for similar reasons- fast script startup time, simple easy language with good defaults), but Awk (while still also awesome) isn’t really designed for stuff beyond a certain size (no signal handling unless you use a fork, for example) LuaJIT is missing bignums, ints that aren’t floats, bit operations, and native utf8 handling, but it can pretty easily be extended with libraries, ffi and metatabling. (I actually made a working bignum library that integrates with gmp, but it has a memory leak somewhere and it’s a rabbit hole/bikeshedding project at this point…) LLM assistance helps hugely and I really like YueScript’s syntax additions. You can point any LLM at a syntax describing webpage and it will pretty much write that language for you…
- nmz 1y ago
  For bignums, have you tried this? https://web.tecgraf.puc-rio.br/~lhf/ftp/lua/#limath https://web.tecgraf.puc-rio.br/~lhf/ftp/lua/#limath luajit does have integers (try 5ull+6), bit operations via the bit library (absorbed from lua5.2)
  - pmarreck 1y ago
    How does imath compare to gmp?
    - nmz 1y ago
      In lua? It works for one.
nmz 1y ago
I do think you're better off using ruby, but if you insist. Lots of default functionality missing so you MUST have these packages: inspect, luaposix, lrexlib-pcre, lrexlib-posix, lpeg, luastd/stdlib, luasocket, luahttp, luasec, luacheck, penlight * luajit is unnecessary in almost all cases, you don't need the speed. * use lsp or luacheck whenever you write something, entr -c luacheck file on everything. * patterns are not regex which means they do not support lookups, backtracking or |, so you must install lrexlib-pcre or lrexlib-posix (frankly I never need pcre so I stick to lrexlib-gnu or lrexlib-posix). * overload _ENV so it auto requires unknown things, I have a lua wrapper that does this and it makes it a joy not having all of my scripts with a bunch of require"posix" on all of them * install inspect to inspect tables * os.execute and io.popen only accepts strings as parameters which means you should overload it and make a function that accepts tables as well. * 5.4 is still lacking support for many libraries, 5.3 has most of the libraries. * assignments default to the global environment so you have to use local keyword or set _ENV to error on assignment (or better yet, don't care, just local _ENV = mymodule) Overall, Lua is just a mixture of C with a pascal syntax and garbage collection (and also tables which is a weird data structure)

Rochus 1y ago

> LuaJITted Lua code runs at 80% (on average, sometimes faster!) of the compiled C version of the same algorithm, typically Cannot confirm this. It might be true on selected micro benchmarks. Here are the results of the Are-we-fast-yet benchmark suite, which includes a decent set of benchmarks challenging CPU, cache and memory access: https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf https://github.com/rochus-keller/Oberon/blob/master/testcase.... On average, the C and C++ implementations are five times faster than LuaJIT. > There are also some neat languages that compile to (transpile to?) Lua Here is a comprehensive list: https://github.com/hengestone/lua-languages https://github.com/hengestone/lua-languages. Lanuages like Oberon or Luon directly compile to LuaJIT bytecode (i.e. not to Lua).

Symmetry 1y ago
The code in Are-we-fast-yet has been heavily optimized. It might still be true that naive LuaJIT can run almost as fast as Naive C.
- Rochus 1y ago
  Have a look at the results and the code; there are benchmarks in the suite where the (ideomatic) C/C++ implementation is "only" twice as fast as the corresponding (idiomatic) Lua implementation, but on average (geomean of all factors) it's about five times as fast. The guidelines of the benchmark are pretty strict to enable fair comparisons (see https://github.com/smarr/are-we-fast-yet/blob/master/docs/guidelines.md https://github.com/smarr/are-we-fast-yet/blob/master/docs/gu...).
  - pmarreck 1y ago
    I looked at the docs. This seems to be comparing against Lua, which is why I specifically said LuaJIT This is quite a distinction to be made. Can you clarify? Directly from their guidelines page: Lua We write code compatible with Lua 5.1, 5.2 and 5.3. Smalltalk/Ruby symbols are represented as normal strings. We use Lua 1-based array and the length operator #. We use single object when a class is not required. Bitwise operators with various Lua versions is a nightmare. We use luacheck as a linter. If they are writing code compatible with Lua 5.2 or 5.3, then that cannot be LuaJIT, which is ONLY compatible with Lua 5.1. (Unless they mean that they JUST write 5.1 code, which due to backwards compatibility is runnable on 5.2 and 5.3? It's unclear from here.)

jwatte 1y ago

Another one of the biggest uses of Lua outside the hyperscaler-type software like nginx or redis: Roblox. Soooo many kids run games on Roblox every day! Roblox not only runs entirely on Lua, but they've been working on their own type inference version of Lua named Luau, and open sourced it, and it's still in very active development. https://github.com/luau-lang/luau https://github.com/luau-lang/luau

const_cast 1y ago

> but turns out it's a perfect language to rewrite some commandline scripts I had that were getting unwieldy in Bash IMO this is the perfect use case for Perl. I mean, it's why Perl was invented, but to this day it remains incredibly good at doing this specific task. It's incredibly easy to write and can be very easy to read, and it's much more robust than bash. But best of all - and the reason I think Perl still has a place in the modern world - it's available on practically every computer on Earth. And it will all work. It's so backwards-compatible that 25 year old scripts will run just fine. It's so portable that nothing even comes close. Also, it's shockingly fast. Faster than you would think. Regex is really powerful and it's been the template for so many regex implementations, but definitely be careful of runaway time complexity. Anyway, Lua is great too. Really nice for embedding into applications ala VBA. Great for config.

pmarreck 1y ago
Good point about Perl. Why don't I consider Perl? I don't know.
- const_cast 1y ago
  Don’t worry, nobody considers Perl