5 ms·

Lightweight Concurrency in Lua

128 points by kgwxd 8y ago

sitkack 8y ago
Previous discussion, https://news.ycombinator.com/item?id=14664150 https://news.ycombinator.com/item?id=14664150 One thing from the previous thread is that Andy has the time and the brains to dive deeply into this stuff. Andy is smart, but what I think sets him apart, is that he relentlessly drives to the root and he revisits the topic over long time scales. Here is a talk Andy gave back in 2017. https://www.youtube.com/watch?v=iskjbi0lNQY https://www.youtube.com/watch?v=iskjbi0lNQY Now that you know the secret, Go Forth! Update, if you find this stuff fascinating, I really recommend that you watch and follow along all of David Beazley's wonderful tutorial session, "A Curious Course on Coroutines and Concurrency" that ends in section 7 with making an OS scheduler loop. http://www.dabeaz.com/coroutines/ http://www.dabeaz.com/coroutines/ https://www.youtube.com/watch?v=Z_OAlIhXziw https://www.youtube.com/watch?v=Z_OAlIhXziw
- jdub 8y ago
  Wow, what a trove of good stuff. Thanks!
- acqq 8y ago
  > Previous discussion ... Warning! The one called the "previous discussion" is not about the same article as this now. This now is: "lightweight concurrency in lua, 16 May 2018" Whereas the one linked behind https://news.ycombinator.com/item?id=14664150 https://news.ycombinator.com/item?id=14664150 is "a new concurrent ml, 29 June 2017" so it can be just a "previous article by the same author which was discussed on HN" or something like that. This one, here now, is not a second go on the same topic, this one here is a brand new topic: it involves Lua, Luajit and Snabb https://github.com/snabbco/snabb https://github.com/snabbco/snabb the latest also something I've just learned that it even exists, and it's quite interesting to me.
- jpfr 8y ago
  In addition to this, Andy is performing wonders for Guile Scheme. It is rare to see such lucid communication and such a clean commit history for hard technical development. http://git.savannah.gnu.org/cgit/guile.git/log/ http://git.savannah.gnu.org/cgit/guile.git/log/ https://lists.gnu.org/archive/html/guile-devel/2018-05/msg00008.html https://lists.gnu.org/archive/html/guile-devel/2018-05/msg00... In a few years, all the cool kids will switch from Graal to Guile for language development.
bullen 8y ago
But with this you can't share memory directly between fibers right?
- ronsor 8y ago
  You can. These "fibers" are just built on the typical lua coroutines.
  - bullen 8y ago
    I need more proof than that. My guess is you can't because sharing memory between threads is hard. Co-routines usually can't, atleast not in rust/go etc. and those are more advanced than lua.
dontwantai 8y ago
As a high performance packet tool, I was expecting something less abstract, for example a special hash or structures to speed computations, or going to assembler in critical parts, but I only see general stuff, nothing that is specialized for high performance packet transformation / inspection.
- samatman 8y ago
  Welcome to the wild world of LuaJIT! It's normal here to write things you would think would need C (or asm!) only to discover that DynaASM and the JIT make them 'fast enough' Next time you're thinking "fast enough for what?": fast enough for real-time high-bandwidth packet inspection.

128 points by kgwxd 8y ago

Previous discussion, https://news.ycombinator.com/item?id=14664150 https://news.ycombinator.com/item?id=14664150 One thing from the previous thread is that Andy has the time and the brains to dive deeply into this stuff. Andy is smart, but what I think sets him apart, is that he relentlessly drives to the root and he revisits the topic over long time scales. Here is a talk Andy gave back in 2017. https://www.youtube.com/watch?v=iskjbi0lNQY https://www.youtube.com/watch?v=iskjbi0lNQY Now that you know the secret, Go Forth! Update, if you find this stuff fascinating, I really recommend that you watch and follow along all of David Beazley's wonderful tutorial session, "A Curious Course on Coroutines and Concurrency" that ends in section 7 with making an OS scheduler loop. http://www.dabeaz.com/coroutines/ http://www.dabeaz.com/coroutines/ https://www.youtube.com/watch?v=Z_OAlIhXziw https://www.youtube.com/watch?v=Z_OAlIhXziw

jdub 8y ago
Wow, what a trove of good stuff. Thanks!
acqq 8y ago
> Previous discussion ... Warning! The one called the "previous discussion" is not about the same article as this now. This now is: "lightweight concurrency in lua, 16 May 2018" Whereas the one linked behind https://news.ycombinator.com/item?id=14664150 https://news.ycombinator.com/item?id=14664150 is "a new concurrent ml, 29 June 2017" so it can be just a "previous article by the same author which was discussed on HN" or something like that. This one, here now, is not a second go on the same topic, this one here is a brand new topic: it involves Lua, Luajit and Snabb https://github.com/snabbco/snabb https://github.com/snabbco/snabb the latest also something I've just learned that it even exists, and it's quite interesting to me.
jpfr 8y ago
In addition to this, Andy is performing wonders for Guile Scheme. It is rare to see such lucid communication and such a clean commit history for hard technical development. http://git.savannah.gnu.org/cgit/guile.git/log/ http://git.savannah.gnu.org/cgit/guile.git/log/ https://lists.gnu.org/archive/html/guile-devel/2018-05/msg00008.html https://lists.gnu.org/archive/html/guile-devel/2018-05/msg00... In a few years, all the cool kids will switch from Graal to Guile for language development.

bullen 8y ago

But with this you can't share memory directly between fibers right?

ronsor 8y ago
You can. These "fibers" are just built on the typical lua coroutines.
- bullen 8y ago
  I need more proof than that. My guess is you can't because sharing memory between threads is hard. Co-routines usually can't, atleast not in rust/go etc. and those are more advanced than lua.

dontwantai 8y ago

As a high performance packet tool, I was expecting something less abstract, for example a special hash or structures to speed computations, or going to assembler in critical parts, but I only see general stuff, nothing that is specialized for high performance packet transformation / inspection.

samatman 8y ago
Welcome to the wild world of LuaJIT! It's normal here to write things you would think would need C (or asm!) only to discover that DynaASM and the JIT make them 'fast enough' Next time you're thinking "fast enough for what?": fast enough for real-time high-bandwidth packet inspection.