4 ms·

We do serious networking in Lua, but it's on top of C socket code (implemented using IOCP or epoll) that transparently serializes, compresses and encrypts Lua t

We do serious networking in Lua, but it's on top of C socket code (implemented using IOCP or epoll) that transparently serializes, compresses and encrypts Lua tuples to bytes on one and, and decrypts, decompresses and deserializes bytes to Lua tuples on the other. This is the backend of a real time game (multiple messages from each player per second), and we handle on the order of 10k connections per core. The "actual Lua language documentation" you refer to probably is the wiki? I agree it is crap. By all means, stay away from Lua and use Python and Node for new projects; we like that competitive advantage.

lifthrasiir 8y ago
> This is the backend of a real time game (multiple messages from each player per second), and we handle on the order of 10k connections per core. I pretty much agree that Lua can survive such throughput, but I question whether Lua can survive the code complexity. I have written about Lua previously on [1] with a vocal conclsion that Lua cannot really handle a large code base (in the order of 10^5 lines), so I like to hear otherwise (hint: I hadn't). That's why... > By all means, stay away from Lua and use Python and Node for new projects; [...] ...we have ultimately dropped (a heavy duty use of) Lua from all new projects. C# seems to suit our needs (or any other statically & strongly typed languages---obviously excluding C/C++---would probably do). [1] https://news.ycombinator.com/item?id=13902023 https://news.ycombinator.com/item?id=13902023
- unsigner 8y ago
  > I have written about Lua previously on [1] with a vocal conclsion that Lua cannot really handle a large code base (in the order of 10^5 lines), so I like to hear otherwise (hint: I hadn't). That's why... We have shipped and maintain a series of games built on progressive iterations of the same codebase; we have on the order of 200 KLOCs of Lua in the common "engine" code and tooling, and 200-400 KLOCs additional game-specific code per game. We tend to hit our milestones and performance targets on six platforms, with programmer quality of life way above the game industry "standards" (i.e. we do normal working hours). Can things be better? Of course, they always can. But om our experience, large-scale use of Lua is not the disaster it usually is assumed/predicted to be.
  - lifthrasiir 8y ago
    Thank you for sharing. But I'm not sure if your experience is not an outlier, that is, some people do produce great and large software in every programming language including annoying ones ;-) I claim that my experience is more typical because many annoyances pointed to Lua's core decisions. For the context, we had about 100 and 200 KLOCs of Lua code for the server and client respectively. We had to use Lua 5.1 because there were not much tooling nor library around 5.2 or later at that time (pain point #1). We of course needed 64-bit integers and sticked to special string representations with some metatable hack, which itself was quite usable. There were several, often critical, bugs that were found to be fixed in 5.2+ but left out in 5.1 even though everyone knows that Lua 5.1 won't die out any sooner (pain point #2): the most critical one for us was the use of thread-unsafe `localtime` which made my jaw dropping. Also there was no clear migration path from 5.1 (AFAIK this holds for 5.2 as well) to later versions (pain point #3); we had to build many system around Lua's dynamism to deal with the complexity and, as you can guess, every minor release of Lua breaks enough dynamism to prevent easy migration. Besides from this, every single library we have tried to use had a critical issue: luasocket plus luasec had spectacularily failed at correctly handling transport timeouts (pain point #4; we did use ASIO as a backend, we only needed them for some external services), luasocket alone had a very annoying and easy-to-mess API for every but simplest HTTP request (pain point #5; eventually we had to shell out to curl which was much, much better than any other Lua-based solution), dkjson had no control over empty arrays vs. empty objects which caused some interop problem; and so on and so on (I can't remember all of them). Ah, of course there are also shitty wikis but I dare not to describe them. We eventually had learned to avoid any external library or even snippet. I feel like that Lua actively encourages DIY mindset which is absurd for modern large-scale programming. Have I mentioned that we had used Lua's dynamism to deal with the complexity? Well, we could only try. We had built our own version of unit testing system because, well, obviously Lua is built for embedding and standalone unit testing does not make sense when you have a custom environment, right? This one is acceptable, but the lack of coverage calculation is not (pain point #6). I had built a custom one, but that was waaaay slow that we couldn't use all the time (and I'm pretty sure that a right debugging API will immersely change the situation, but there is still none as of 5.3). Debugging was also abysmal (pain point #7), again Lua is built for embedding and standalone debugging experience does not make sense, right? We already knew and used MobDebug [1] which kinda worked but was brittle. We had eventually built a serious debugging tool (open-sourced!) that integrates to Visual Studio Code via Language Server Protocol. The remaining story goes much like this: there are no or only partial tooling around embedded context (supposedly the biggest use case of Lua), and we had wasted too much time to build ours including a type checker (!!!!) [2]. I do agree that Lua has some upsides as well. In fact, Lua greatly helped us to quickly build a functioning product from the scratch, and the ability to safely discard the whole state massively simplified the error recovery process (alluding Erlang's approach). I really, really want to like Lua because it seems the only one viable programming language with those charactistics. But my own experience says otherwise. [1] https://github.com/pkulchenko/MobDebug https://github.com/pkulchenko/MobDebug [2] https://github.com/devcat-studio/kailua https://github.com/devcat-studio/kailua

lifthrasiir 8y ago

> This is the backend of a real time game (multiple messages from each player per second), and we handle on the order of 10k connections per core. I pretty much agree that Lua can survive such throughput, but I question whether Lua can survive the code complexity. I have written about Lua previously on [1] with a vocal conclsion that Lua cannot really handle a large code base (in the order of 10^5 lines), so I like to hear otherwise (hint: I hadn't). That's why... > By all means, stay away from Lua and use Python and Node for new projects; [...] ...we have ultimately dropped (a heavy duty use of) Lua from all new projects. C# seems to suit our needs (or any other statically & strongly typed languages---obviously excluding C/C++---would probably do). [1] https://news.ycombinator.com/item?id=13902023 https://news.ycombinator.com/item?id=13902023

unsigner 8y ago
> I have written about Lua previously on [1] with a vocal conclsion that Lua cannot really handle a large code base (in the order of 10^5 lines), so I like to hear otherwise (hint: I hadn't). That's why... We have shipped and maintain a series of games built on progressive iterations of the same codebase; we have on the order of 200 KLOCs of Lua in the common "engine" code and tooling, and 200-400 KLOCs additional game-specific code per game. We tend to hit our milestones and performance targets on six platforms, with programmer quality of life way above the game industry "standards" (i.e. we do normal working hours). Can things be better? Of course, they always can. But om our experience, large-scale use of Lua is not the disaster it usually is assumed/predicted to be.
- lifthrasiir 8y ago
  Thank you for sharing. But I'm not sure if your experience is not an outlier, that is, some people do produce great and large software in every programming language including annoying ones ;-) I claim that my experience is more typical because many annoyances pointed to Lua's core decisions. For the context, we had about 100 and 200 KLOCs of Lua code for the server and client respectively. We had to use Lua 5.1 because there were not much tooling nor library around 5.2 or later at that time (pain point #1). We of course needed 64-bit integers and sticked to special string representations with some metatable hack, which itself was quite usable. There were several, often critical, bugs that were found to be fixed in 5.2+ but left out in 5.1 even though everyone knows that Lua 5.1 won't die out any sooner (pain point #2): the most critical one for us was the use of thread-unsafe `localtime` which made my jaw dropping. Also there was no clear migration path from 5.1 (AFAIK this holds for 5.2 as well) to later versions (pain point #3); we had to build many system around Lua's dynamism to deal with the complexity and, as you can guess, every minor release of Lua breaks enough dynamism to prevent easy migration. Besides from this, every single library we have tried to use had a critical issue: luasocket plus luasec had spectacularily failed at correctly handling transport timeouts (pain point #4; we did use ASIO as a backend, we only needed them for some external services), luasocket alone had a very annoying and easy-to-mess API for every but simplest HTTP request (pain point #5; eventually we had to shell out to curl which was much, much better than any other Lua-based solution), dkjson had no control over empty arrays vs. empty objects which caused some interop problem; and so on and so on (I can't remember all of them). Ah, of course there are also shitty wikis but I dare not to describe them. We eventually had learned to avoid any external library or even snippet. I feel like that Lua actively encourages DIY mindset which is absurd for modern large-scale programming. Have I mentioned that we had used Lua's dynamism to deal with the complexity? Well, we could only try. We had built our own version of unit testing system because, well, obviously Lua is built for embedding and standalone unit testing does not make sense when you have a custom environment, right? This one is acceptable, but the lack of coverage calculation is not (pain point #6). I had built a custom one, but that was waaaay slow that we couldn't use all the time (and I'm pretty sure that a right debugging API will immersely change the situation, but there is still none as of 5.3). Debugging was also abysmal (pain point #7), again Lua is built for embedding and standalone debugging experience does not make sense, right? We already knew and used MobDebug [1] which kinda worked but was brittle. We had eventually built a serious debugging tool (open-sourced!) that integrates to Visual Studio Code via Language Server Protocol. The remaining story goes much like this: there are no or only partial tooling around embedded context (supposedly the biggest use case of Lua), and we had wasted too much time to build ours including a type checker (!!!!) [2]. I do agree that Lua has some upsides as well. In fact, Lua greatly helped us to quickly build a functioning product from the scratch, and the ability to safely discard the whole state massively simplified the error recovery process (alluding Erlang's approach). I really, really want to like Lua because it seems the only one viable programming language with those charactistics. But my own experience says otherwise. [1] https://github.com/pkulchenko/MobDebug https://github.com/pkulchenko/MobDebug [2] https://github.com/devcat-studio/kailua https://github.com/devcat-studio/kailua