4 ms·

An anecdote from a non-JS JIT, but similar: I once spent a summer working on a game engine with a couple others where the host language was LuaJIT. It started

An anecdote from a non-JS JIT, but similar: I once spent a summer working on a game engine with a couple others where the host language was LuaJIT. It started out great. The iteration cycles were incredibly short, performance was competitive, the code was portable, and we could do wild metaprogramming nonsense that was still fast. If you haven't worked with LuaJIT, its C FFI is also incredible! As we started scaling though, the wheels fell off the wagon. We'd add a new feature and suddenly the game wouldn't run at interactive framerates. One time, it was the 'unpack' function, which would trigger a JIT trace abort. We would drop from 12ms frames to 100ms frames. I wrote a length-specialized version that didn't abort and moved on. Another time, it was calling Lua's 'pairs' method (iterator over a map). Okay, so we can't do that, or a few other things that made Lua productive before. The other problem we hit was GC predictability being impossible. We tried to mitigate it by using native data structures through the C FFI, taking control over the GC cycle to run it once or twice per frame, etc. In the end, like the JIT problem, we weren't writing Lua at the end, we were writing... something else. It wasn't maintainable. That summer ruined dynamic languages for me. I didn't really want to be writing C or C++ at the time. I ended up picking up Rust, which was predictable and still felt high-level, and the Lua experience ended up getting me my current job.

samatman 7y ago
'pairs' was jitted recently by the RaptorJIT crew, fwiw. I've hit fewer snags with LuaJIT, but they're definitely there. Really wish Mike Pall had written that hyperblock scheduler before retiring...
- stephen82 7y ago
  He did not retire; he is still quite active and improving LuaJIT [1]. [1] https://github.com/LuaJIT/LuaJIT/tree/v2.1
  - samatman 7y ago
    Well, both are true: https://www.freelists.org/post/luajit/Looking-for-new-LuaJIT-maintainers https://www.freelists.org/post/luajit/Looking-for-new-LuaJIT... It's more of an emeritus situation than anything, he's still active on the list as well. I'd be ecstatic if hyperblock scheduling, or the quad-color garbage collector, were to drop onto the 2.1 trunk, but I'm not counting on it.
MrBuddyCasino 7y ago
Thanks for the story details, quite interesting. In the end this was unfortunately a case of picking the wrong tool for the job. Don’t use JITed / GC languages when you‘ve got hard realtime requirements. Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra). Don’t rely on auto-vectorisation in your inner loop if possible, one tiny change could bring that house of cards crashing down. I‘d be interested in how your team ended up picking that tech stack. Was it a „rational“ weighing of options with pros and cons? Was it „eh it‘ll be alright“? Was it personal preference and/or prior experience?
- pron 7y ago
  > Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra) OpenJDK's ZGC [1] is well on its way to have worst-case latencies of under 1ms (as soon as this year) on heaps up to 16TB in size. [1]: https://wiki.openjdk.java.net/display/zgc/Main https://wiki.openjdk.java.net/display/zgc/Main
  - MrBuddyCasino 7y ago
    That didn't exist yet when C* was started though. I'd be interested in how it compares to ScyllaDB, I suspect still not favourably.
    - naasking 7y ago
      Sub 1ms incremental GCs did exist, but they weren't common in production languages. They typically optimised for throughput rather than latency.
  - vbezhenar 7y ago
    This does not speak anything about end result. You might end up with 1000 GC collections per request. Each will be 1 ms, but total will be 1 second. It's still non-determenistic.
    - 7y ago
beetwenty 7y ago
I'm working with Lua right now(gopherlua) as a scripting option for real-time gaming. I've done similar things to your story in the past with trying to make Lua the host for everything and I'm well aware of the downsides, but I have a requirement of maintaining readable, compatible source(as in PICO-8's model) - and Lua is excellent at that, as are other dynamic languages, to the point where it's hard to consider anything else unless I build and maintain the entire implementation. So my mitigation strategy is to do everything possible to make the Lua code remain in the glue code space, which means that I have to add a lot of libaries. I'm also planning to add support for tl, which should make things easier on the in-the-large engineering side of things - something dynamic languages are also pretty awful at.
- masklinn 7y ago
  GP is talking about LuaJIT, you're talking about Lua. Lua has lower performances but should be completely predictable, GC aside (not sure what its GC scheme is), so it's a very different situation.
- yumaikas 7y ago
  You might still run into GC problems, but none of the Go-based Luas (built on Go, rather than binding to another Lua library) I am aware of have a JIT built in.

samatman 7y ago

'pairs' was jitted recently by the RaptorJIT crew, fwiw. I've hit fewer snags with LuaJIT, but they're definitely there. Really wish Mike Pall had written that hyperblock scheduler before retiring...

stephen82 7y ago
He did not retire; he is still quite active and improving LuaJIT [1]. [1] https://github.com/LuaJIT/LuaJIT/tree/v2.1
- samatman 7y ago
  Well, both are true: https://www.freelists.org/post/luajit/Looking-for-new-LuaJIT-maintainers https://www.freelists.org/post/luajit/Looking-for-new-LuaJIT... It's more of an emeritus situation than anything, he's still active on the list as well. I'd be ecstatic if hyperblock scheduling, or the quad-color garbage collector, were to drop onto the 2.1 trunk, but I'm not counting on it.

MrBuddyCasino 7y ago

Thanks for the story details, quite interesting. In the end this was unfortunately a case of picking the wrong tool for the job. Don’t use JITed / GC languages when you‘ve got hard realtime requirements. Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra). Don’t rely on auto-vectorisation in your inner loop if possible, one tiny change could bring that house of cards crashing down. I‘d be interested in how your team ended up picking that tech stack. Was it a „rational“ weighing of options with pros and cons? Was it „eh it‘ll be alright“? Was it personal preference and/or prior experience?

pron 7y ago
> Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra) OpenJDK's ZGC [1] is well on its way to have worst-case latencies of under 1ms (as soon as this year) on heaps up to 16TB in size. [1]: https://wiki.openjdk.java.net/display/zgc/Main https://wiki.openjdk.java.net/display/zgc/Main
- MrBuddyCasino 7y ago
  That didn't exist yet when C* was started though. I'd be interested in how it compares to ScyllaDB, I suspect still not favourably.
  - naasking 7y ago
    Sub 1ms incremental GCs did exist, but they weren't common in production languages. They typically optimised for throughput rather than latency.
- vbezhenar 7y ago
  This does not speak anything about end result. You might end up with 1000 GC collections per request. Each will be 1 ms, but total will be 1 second. It's still non-determenistic.
  - 7y ago

beetwenty 7y ago

I'm working with Lua right now(gopherlua) as a scripting option for real-time gaming. I've done similar things to your story in the past with trying to make Lua the host for everything and I'm well aware of the downsides, but I have a requirement of maintaining readable, compatible source(as in PICO-8's model) - and Lua is excellent at that, as are other dynamic languages, to the point where it's hard to consider anything else unless I build and maintain the entire implementation. So my mitigation strategy is to do everything possible to make the Lua code remain in the glue code space, which means that I have to add a lot of libaries. I'm also planning to add support for tl, which should make things easier on the in-the-large engineering side of things - something dynamic languages are also pretty awful at.

masklinn 7y ago
GP is talking about LuaJIT, you're talking about Lua. Lua has lower performances but should be completely predictable, GC aside (not sure what its GC scheme is), so it's a very different situation.
yumaikas 7y ago
You might still run into GC problems, but none of the Go-based Luas (built on Go, rather than binding to another Lua library) I am aware of have a JIT built in.