4 ms·
The key to understanding Lua is that the C API is a first-class citizen, and the Lua authors strive to keep the Lua scripting and Lua C API semantics symmetric
by wahern 6y ago
The key to understanding Lua is that the C API is a first-class citizen, and the Lua authors strive to keep the Lua scripting and Lua C API semantics symmetric (my characterization). As a general rule, anything you can do in Lua script you can do from the C API--closures, coroutines, etc--with the same logical semantics. (And using the standard C-compliant Lua C API--no C compiler extensions required.) The VM implementation, language semantics, and Lua C API strongly reflect each other and channel language design and VM implementation. Finalizers are critical for a good C API, and whether the finalizer is a lua_CFunction or an in-language function should be and is irrelevant. And a rule that didn't permit leaking would be rather brittle and error prone from the perspective a C module author.
Of course, relying on ephemerons to merely detect GC collection (as opposed to their primary purpose of caching and memoizing data associated to a particular blackbox object instance) is a rather obtuse hack, and not something I've seen in practice for anything other than debugging.
I don't agree that JavaScript's semantics are what have permitted JavaScript optimizing compilers to work so well. Details like the above aren't necessarily difficult to implement efficiently; they're difficult, if not impossible, to implement if the implementation architecture didn't contemplate their existence, or if the architecture was predicated on their non-existence. For awhile LuaJIT outperformed mature JavaScript engines despite having to deal with more complex semantics and more abstract constructs (e.g. stackful coroutines that were independent of the C stack, fidelity to the Lua C API, goto, variable return lists, guaranteed TCO, etc), as well all the same dynamic typing headaches as JavaScript and Python. And it did so with an appreciably smaller VM. And I have no doubt it could do it again with the benefit of another 10 years of knowledge and experience.
What Lua has going in its favor is 1) the authors are only weakly committed to backwards compatibility--there are always breakages with each new version, usually small but they add up over time; and 2) a strong commitment to a symmetric C API, which means the authors have to think long and hard about language constructs and architectural details. #1 provides them the liberty to experiment with better semantics, including discarding constructs or behaviors that don't work out. #2 is a constraint that prevents them from taking certain shortcuts, such as intermingling the "C stack" with the logical language stack. The reason JavaScript only supports explicit "async" methods rather than a more seamless control flow construct is because all implementations (except DukTape?) mingle C stack semantics with the in-language stack, including especially the JIT compiler components. This chumminess isn't necessary--LuaJIT isn't particularly handicapped by avoiding it--but it's not something you can fix after the fact; it'd requiring starting an implementation from scratch, and that's probably not ever going to happen again, at least not merely for the purposes of providing better control flow semantics. VM authors tend not to appreciate how architectural details limit the range and scope of future language constructs. The Lua C API gives the Lua authors fewer degrees of freedom, which paradoxically results, I think, in them avoiding implementation dead-ends.
I do agree that it's problematic to rely on certain aspects of a GC, like absolute timing in the case of mark & sweep style GC. But other aspects, like finalizers and object revival, are deliberate guarantees focused on language semantics. Another guarantee that Lua provides is that objects are destroyed in reverse order of creation, which is a useful guarantee from the perspective of a C module writer. Again, because of the nature of the Lua C API and the need to provide more than just a blackbox GC, Lua is far more deliberate about which aspects of the GC can be relied upon (and how), and which aren't their problem. Other languages avoid this--the very thought of providing any GC-specific language features is anathema--but in time invariably find themselves providing ad hoc guarantees and interfaces, or locked into accidental semantics.