4 ms·

Does Lua also have a GIL?

by codexon 17y ago

silentbicycle 17y ago
No, it doesn't. The whole interpreter was designed from the beginning to be embedded in another (typically C or C++) project, and they didn't want to have any concurrency infrastructure in the core language that would clash with the surrounding project's design. (It has excellent support for coroutines, though.) Rather than having static variables and a GIL, the interpreter keeps all its state behind an opaque pointer to a VM context. The VM is quite lightweight (a few hundred K), and running in each thread or process is not a big deal. With forking, you even share the standard libraries. In practice, this just means that every call to the Lua C API starts with ("lua_State *L"). Not a big deal. Lua also has real tail-calls, closures, and lambdas. It's a great language. Rather like a smaller, cleaner, less opinionated Python.
- malkia 17y ago
  Also lua right now has the one of the best JIT's out there (luajit). The guy behind the project is simply genius. Really good stuff. And also kept small. It's x2-x3 faster than V8 on specific benchmarks that I've did (mainly dealing with 3d model meshes and modifying them). It's only for x86 right now, but he's working on x64 port.
  - silentbicycle 17y ago
    LuaJIT is really impressive, but honestly, the stock interpreter is fast enough for me in general, and I'm fine with just moving hotspots to C as needed. I'm more impressed with how concise Lua is. I've got the Lua 5.1 Reference Manual sitting in front of me. It's 103 pages for the whole language (roughly half of which is the C API), the entire syntax fits on page 95, etc. I can keep it all in my mental L1 cache, so to speak. Yet, it's expressive enough to have displaced Python as my go-to language. It's not perfect, but I'm very happy with the trade-offs it makes. I'll be psyched once it's ported to a couple more hardware platforms, though.
- codexon 17y ago
  I just read the Lua manually, and it appears as though Lua has a GIL. http://lua-users.org/wiki/ThreadsTutorial http://lua-users.org/wiki/ThreadsTutorial ANSI C, the standard to which Lua complies, has no mechanism for managing multiple threads of execution. The method above for implementing threading in Lua using a global mutex is inefficient on multiprocessor systems. As one thread holds the global mutex, other threads are waiting for it. Thus only one Lua thread may be running at a time, regardless of the number of processors in the system. Am I misinterpreting what's being written here?
  - silentbicycle 17y ago
    It's a bit confusing. Essentially, "the method above" (using multiple OS threads running on the same Lua VM) extends Lua, adding a GIL along the way. Lua itself does not use pre-emptive threads (the kind where you need to worry about locks, race conditions, "thread safe" libraries, etc.). It has co-routines, co-operative threads, in which independent light processes run and yield to each other. Only one is running in the interpreter at a time, so locking and race conditions are much less an issue. If something needs to happen atomically, you just don't yield and resume in the middle of it. (Coroutines are as versatile as one-shot continuations, incidentally - each can be implemented in terms of the other.) BUT, since Lua is designed for use in other projects, it may be running as a scripting engine inside of (say) a multi-threaded C++ game. In this case, Lua can have multiple threads running in the VM at once. There are no-op functions (lua_lock and lua_unlock) defined in the code around operations which need to be atomic. Such a project could recompile Lua with them calling appropriate locking operations. You can run Lua multithreaded, if you want, but then you have to worry about thread safety. (I have to admit some ignorance here, though - I think running multiple independent VMs is much simpler than having to deal with debugging multi-threaded code.) FWIW, the source for Lua 5.1.4 is ~626 KB. Adding a customized version of it to a project's source tree is nowhere near a big deal as including, say, all of Python (which is, what, 40+ MB?). In the Lua source, luaconf.h collects some items that are likely to be tweaked in project-specific ways. Anyway, Lua does not have any threading implementation of its own because it would make it far more difficult to run inside of other projects. The Lua VM itself is pretty light (a few hundred KB), and since all VM state is specific to each instance, you can run several without running into the sort of problems that people have with Python's GIL. There are several extensions for multitasking in Lua, beyond using coroutines. Here is a listing on the wiki (http://lua-users.org/wiki/MultiTasking http://lua-users.org/wiki/MultiTasking) and a comparison by the author of Lanes (http://kotisivu.dnainternet.net/askok/bin/lanes/comparison.html http://kotisivu.dnainternet.net/askok/bin/lanes/comparison.h...). This is one of the cases where keeping Lua small, portable, and easy to embed means that functionality people expect ends up in an extension rather than as part of the "official" standard library. I'm happy with it as a trade-off - it makes Lua as expressive as Python but keeps the core language as small and clean as SQLite.

Does Lua also have a GIL?

by codexon 17y ago

Does Lua also have a GIL?

No, it doesn't. The whole interpreter was designed from the beginning to be embedded in another (typically C or C++) project, and they didn't want to have any concurrency infrastructure in the core language that would clash with the surrounding project's design. (It has excellent support for coroutines, though.) Rather than having static variables and a GIL, the interpreter keeps all its state behind an opaque pointer to a VM context. The VM is quite lightweight (a few hundred K), and running in each thread or process is not a big deal. With forking, you even share the standard libraries. In practice, this just means that every call to the Lua C API starts with ("lua_State *L"). Not a big deal. Lua also has real tail-calls, closures, and lambdas. It's a great language. Rather like a smaller, cleaner, less opinionated Python.

malkia 17y ago
Also lua right now has the one of the best JIT's out there (luajit). The guy behind the project is simply genius. Really good stuff. And also kept small. It's x2-x3 faster than V8 on specific benchmarks that I've did (mainly dealing with 3d model meshes and modifying them). It's only for x86 right now, but he's working on x64 port.
- silentbicycle 17y ago
  LuaJIT is really impressive, but honestly, the stock interpreter is fast enough for me in general, and I'm fine with just moving hotspots to C as needed. I'm more impressed with how concise Lua is. I've got the Lua 5.1 Reference Manual sitting in front of me. It's 103 pages for the whole language (roughly half of which is the C API), the entire syntax fits on page 95, etc. I can keep it all in my mental L1 cache, so to speak. Yet, it's expressive enough to have displaced Python as my go-to language. It's not perfect, but I'm very happy with the trade-offs it makes. I'll be psyched once it's ported to a couple more hardware platforms, though.
codexon 17y ago
I just read the Lua manually, and it appears as though Lua has a GIL. http://lua-users.org/wiki/ThreadsTutorial http://lua-users.org/wiki/ThreadsTutorial ANSI C, the standard to which Lua complies, has no mechanism for managing multiple threads of execution. The method above for implementing threading in Lua using a global mutex is inefficient on multiprocessor systems. As one thread holds the global mutex, other threads are waiting for it. Thus only one Lua thread may be running at a time, regardless of the number of processors in the system. Am I misinterpreting what's being written here?
- silentbicycle 17y ago
  It's a bit confusing. Essentially, "the method above" (using multiple OS threads running on the same Lua VM) extends Lua, adding a GIL along the way. Lua itself does not use pre-emptive threads (the kind where you need to worry about locks, race conditions, "thread safe" libraries, etc.). It has co-routines, co-operative threads, in which independent light processes run and yield to each other. Only one is running in the interpreter at a time, so locking and race conditions are much less an issue. If something needs to happen atomically, you just don't yield and resume in the middle of it. (Coroutines are as versatile as one-shot continuations, incidentally - each can be implemented in terms of the other.) BUT, since Lua is designed for use in other projects, it may be running as a scripting engine inside of (say) a multi-threaded C++ game. In this case, Lua can have multiple threads running in the VM at once. There are no-op functions (lua_lock and lua_unlock) defined in the code around operations which need to be atomic. Such a project could recompile Lua with them calling appropriate locking operations. You can run Lua multithreaded, if you want, but then you have to worry about thread safety. (I have to admit some ignorance here, though - I think running multiple independent VMs is much simpler than having to deal with debugging multi-threaded code.) FWIW, the source for Lua 5.1.4 is ~626 KB. Adding a customized version of it to a project's source tree is nowhere near a big deal as including, say, all of Python (which is, what, 40+ MB?). In the Lua source, luaconf.h collects some items that are likely to be tweaked in project-specific ways. Anyway, Lua does not have any threading implementation of its own because it would make it far more difficult to run inside of other projects. The Lua VM itself is pretty light (a few hundred KB), and since all VM state is specific to each instance, you can run several without running into the sort of problems that people have with Python's GIL. There are several extensions for multitasking in Lua, beyond using coroutines. Here is a listing on the wiki (http://lua-users.org/wiki/MultiTasking http://lua-users.org/wiki/MultiTasking) and a comparison by the author of Lanes (http://kotisivu.dnainternet.net/askok/bin/lanes/comparison.html http://kotisivu.dnainternet.net/askok/bin/lanes/comparison.h...). This is one of the cases where keeping Lua small, portable, and easy to embed means that functionality people expect ends up in an extension rather than as part of the "official" standard library. I'm happy with it as a trade-off - it makes Lua as expressive as Python but keeps the core language as small and clean as SQLite.