9 ms·
LLRT: A low-latency JavaScript runtime from AWS
- wmf 3y agoI wonder how this compares to QuickJS.
- readthenotes1 3y agoI wonder how popular "sever less" would be if it were called "depending on the kindness of strangers"
- politelemon 3y agoSince we're paying for it I'd call it more of "depending on the competence of service providers." AWS Lambda experience has been great for us in that sense, so I'd be looking forward to more general availability of this for Lambdas.
- cmsparks 3y agoI think this project runs on the QuickJS engine
- paulddraper 3y agoIt's built on QuickJS.
- syrusakbary 3y agoThis is awesome, I'm incredibly happy to see more alternatives on the Javascript Runtime Space! In a nutshell: LLRT aims to have incredibly fast startup times compared to Node, Bun or Deno (which is critical in Lambda environments). LLRT is built using Rust, Tokio and QuickJS under the hood. Here's their compatibility with Node.js APIs [1] We (at Wasmer), have been working on a very similar approach: WinterJS, which is using SpiderMonkey instead (but also built on top of Rust and Tokio) [2]. HN announcement was just three months ago [3] Really excited to see how the ecosystem is evolving! [1] https://github.com/awslabs/llrt?tab=readme-ov-file#compatibility-matrix https://github.com/awslabs/llrt?tab=readme-ov-file#compatibi... [2] https://github.com/wasmerio/winterjs https://github.com/wasmerio/winterjs [3] https://news.ycombinator.com/item?id=38047872 https://news.ycombinator.com/item?id=38047872
- mdaniel 3y agocurl https://github.com/wasmerio/winterjs | grep -i license # :-(
- syrusakbary 3y agoOops, will add the license right away! Edit: MIT license added! We had it on the Cargo.toml and wasmer.toml, but forgot to add the LICENSE file to the repo https://github.com/wasmerio/winterjs/blob/main/LICENSE https://github.com/wasmerio/winterjs/blob/main/LICENSE
- k__ 3y agoFaster than Bun? Not bad!
- geysersam 3y agoAre there any benchmarks comparing startup time of LLRT, Deno, and Bun? Edit: particularly curious about comparison with Deno deploy.
- ushakov 3y agoDeno’s and Bun’s Isolates should be spawning faster than a QuickJS process, they also utilize snapshots for more optimization. QuickJS is going to be faster at startup from 0 though
- mmastrac 3y agoInteresting project: they are trading lower raw throughput in compute-heavy applications for lower latency at startup and (likely) better behaviour w.r.t memory and "random GC jank". If you are writing an I/O bound JS service that doesn't do a lot of thinking, but does a lot of "moving stuff around", this will probably be a win.
- sfink 3y agoYes, though don't think of "traditional" JS engines as being all the way on the throughput side. One of the main distinguishing characteristics of JS engines, at least those that are used in web browsers, is that they have to care a lot about startup time and have to juggle a lot of tradeoffs that other managed runtimes typically don't need to bother with. A surprisingly high percentage of downloaded JS code is never run at all. (I can speculate: event handlers, unshaken trees, shaken trees with paths that happen to not be taken, ...) Of the rest, the vast majority is run once or a handful of times. That means startup time is a high percentage of total time, and therefore engines will JIT very little of that code, and the little that is JITted will use a baseline JIT that pretty much just translates opcodes to unoptimized machine code. The fancy optimization engineering that requires much cleverness and effort is actually applied to a very small percentage of incoming code. It will need to run something like 1000x before it is seen worth bothering with. Latency is more important than throughput for almost all of the JS that the engine sees. (Note that throughput-sensitivity can easily dominate most of the runtime, but that varies widely depending on workload.) So browser-embedded engines already work hard at optimizing latency. We're not talking Python or Ruby here. That still leaves a lot of potential for improvement by stripping the engine down and prioritizing latency above all else, though. There's definitely a niche for things like LLRT. And WinterJS, too. (Source: I work on SpiderMonkey)
- ushakov 3y agoHermes is a big one as well: low startup latency, low memory usage, pre-compiled and the newest version comes with static-optimization https://hermesengine.dev/ https://hermesengine.dev/
- hamoodhabibi 3y ago> offers up to over 10x faster startup and up to 2x overall lower cost i've been burned by Amazon with these type of claims can we get confirmation this is backed up actual real world testing from a non-amazon affiliated source?
- nextworddev 3y agoIt’s because they never get sued for saying stuff like this
- willsmith72 3y agoWell it does say "up to", that's pretty easy to prove with a niche contrived example
- incrudible 3y agoUp to 2x, or more.
- eminence32 3y agoThe full quote is this: > LLRT offers up to over 10x faster startup and up to 2x overall lower cost compared to other JavaScript runtimes running on AWS Lambda I read this to mean that AWS only cares about performance on AWS Lambda, and it's probably not this much faster outside of the Lambda contxt
- spankalee 3y agoI would love to see similarly low startup times with a real JS-to-WASM compiler (rather than today's approach of compiling a JS runtime to WASM). With WASM GC landed, and WASI preview 3 taking on async / event loops, it seems like this should be possible soon, except for the very large task of actually writing the compiler.
- whizzter 3y agoThere's a VERY good reason why JS runtimes use JIT's, it's more or less impossible to properly fully type a JS program w/o leaving runtime holes. If you restrict yourself to a subset or are willing to risk incompatiblities then it will work. There's some TypeScript to Wasm compilers that should work reasonably well (If you're breaking TS rules you've got yourself to blame a bit more than with JS). Ref: My thesis was on JS AOT and even if I had the idea of making it commercial I didn't pursue it in the end.
- ushakov 3y agoAssemblyScript works fine too! WASM and JS isn’t a good combination atm.
- ignoramous 3y ago> My thesis was on JS AOT Is it available online?
- deleted 3y ago[deleted]
- whizzter 3y agoYes, it was written for other students so it's not as clean as most of the referred research papers. Part of the work was implementing a compiler whilst part of was surveying how well real world JS developers would understand language subset limitations. Most relevant for my above comment re AOT vs JIT is that JS semantics (I think I touched upon it in the thesis) will have the same issues that made JIT's win out with BigInts in the Agesen&Hölzle paper (whilst the paper is on AOT's my goal was to find a solution for game developers so numerical performance was a non-moving goal for me). My thesis http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A823252&dswid=-3627 http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A8232... Agesen & Hölze on JIT vs AOT https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=e07752a69ee34b97659502efe528942fa395d7a3 https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...
- jitl 3y agoThis seems ideal for places you use Lambda as glue, like for routing requests or authorization policy decisions. But for application that doing a lot of thinking, keep in mind v8 jitless is about 3x faster than QuickJS, and with JIT it’s about 30x faster than quickjs. I’m curious to see where the break-even point is for various workloads comparing the two, and also numbers versus Bun, which starts quicker than Node but has comparable top-end JIT performance.
- hamoodhabibi 3y agoOne good thing Lambda provides is contractual guarantee, if you want 100000 code spun up you can put it in writing and hold AWS against it but then you realize you can achieve the same thing at 98% discount with traditional "monolith" setups with a load balancer soaking up all the requests Serverless is a major failure
- intelVISA 3y agoThe Cloud(tm) experience
- SahAssar 3y agoServerless has two things that I really like about it: 1. Scale to zero, which allows me to do things like automatically deploy a full environment for each branch in a project, with a full infra, frontend, backend, database, etc. 2. Good integration with IaC tools, so that I can define my infra and my runners in a single language/tool, be it terraform/cdk or something else. Most "monolithic" setups split configuring the infra and what runs in it into two tools/steps (please let me know of ones that don't!). But if I actually run a application for a long time with somewhat consistent load there are always cheaper, more performant and flexible solutions than a serverless setup.
- SaltyBackendGuy 3y agoWe've found it really nice for worker queues for specific workloads that fire infrequently'ish.
- 3y ago
- hendry 3y agoWonder how it compares to https://aws-lite.org/ https://aws-lite.org/
- rdtsc 3y agohttps://github.com/awslabs/llrt/blob/e2cc8237f4bd04e161d46b3266b040b81a3a66e6/src/main.rs#L36 https://github.com/awslabs/llrt/blob/e2cc8237f4bd04e161d46b3... This wraps QuickJS and there is not a single mention on their front page. At least a hello or shout-out to Fabrice Bellard would have been nice. But I guess it's AWS and they do this with other projects, so not too surprising.
- ignoramous 3y ago> At least a hello or shout-out to Fabrice Bellard would have been nice. Mentioned in the third-party file: https://github.com/awslabs/llrt/blob/630ec79/THIRD_PARTY_LICENSES#L31-L34 https://github.com/awslabs/llrt/blob/630ec79/THIRD_PARTY_LIC... / https://archive.is/0NUAa https://archive.is/0NUAa > AWS and they do this with other projects Remember similar concerns raised viz Firecracker (which is a hard fork of Google crosvm: https://news.ycombinator.com/item?id=22360232 https://news.ycombinator.com/item?id=22360232), but don't recall it being a pattern.
- wswope 3y ago> AWS and they do this with other projects Probably a nod to the ElasticSearch licensing debacle.
- gfodor 3y agoGross.
- drschwabe 3y agoAmazon's repo here is open source at least so perhaps this could be remedied with an edit to their README and a PR - go for it ! Agree with you there should be credit where credit is due - I have been using QuickJS for some time and its awesome. For the cost of about 1MB you can get entire modern JS in your C/C++ binary.
- darknavi 3y agoWe use QuickJS in Minecraft for our scripting engine. It's nice that he has got back into it these days and been adding ES2023 support. I also had to double check but we do indeed list it on our website, albeit in a sea of others. https://www.minecraft.net/en-us/attribution https://www.minecraft.net/en-us/attribution
- kentonv 3y agoThe benchmark shows Node.js taking 1.5s(!) to start vs. LLRT taking under 100ms. But the benchmark appears to be a very small program that imports the AWS SDK. Later on in the readme, it's mentioned that LLRT embeds the AWS SDK into the binary. So... How much of what the benchmark is showing here is just the fact that the embedded SDK loads much faster than application code? I mean, it's certainly a valid optimization for Lambdas that are primarily making AWS API calls. But it'd also be interesting to know how much faster LLRT is at loading arbitrary code. As well as interesting to know if the embedding mechanism could be made directly available to applications to make their own code load faster. (Disclosure: I work on Cloudflare Workers, a competitor. But, honestly interested in this.)
- ushakov 3y agoEven faster would be just having FFI, so you could skip Rust <> C <> JS conversion every time you call a function. This is what hermes is currently doing.
- kentonv 3y agoTrue but probably very tedious to achieve full compatibility with the existing JS API that way, which seems like it was a big goal here.
- ushakov 3y agoNot really. I have built a fetch-like API for hermes on top of libcurl using generated FFI-bindings (they have a bindgen tool in the repo). All I had to do is just write the JS-wrapper that would call the underlying C-lib and return a Promise. It feels wild, people have been calling UI-libs from JS as well: https://twitter.com/tmikov/status/1720103356738474060 https://twitter.com/tmikov/status/1720103356738474060 I can imagine someone making a libuv binding and then just recreating Node.JS with the FFI
- kentonv 3y agoI mean that the AWS SDK is a large API surface, they'd potentially have to re-implement all of it in Rust and then keep up with future changes, otherwise you have to tell people it's not 100% compatible. I'm told this library is multiple megabytes of JavaScript. (That's why Node takes so long to load it...)
- rafetefe 3y agoWish someone pay me to build stuff like this, such a cool project to work to
- ushakov 3y agoFastly, Cloudflare, Vercel, Fermyon, Wasmer, Azion, Deno, Netlify, SecondState, Cosmonic are paying to build stuff like this, just to name a few
- ushakov 3y agoForgot to mention Shopify as well https://shopify.engineering/javascript-in-webassembly-for-shopify-functions https://shopify.engineering/javascript-in-webassembly-for-sh... And Bun of course :)
- jerrysievert 3y agopljs (https://github.com/plv8/pljs https://github.com/plv8/pljs) is a postgres language plugin that also uses quickjs. given how much faster the startup is compared to v8, and that most postgres functions are very small and do very little compared to a node.js program, it is quite good. I can definitely see aws' lambda operations gaining quite a bit from this.
- simlevesque 3y agoDoes anyone know if this is what is used for Cloudfront Functions ? (Not lambda@edge)
- austin-cheney 3y agoLLRT does not use a JIT. Does that mean it’s just old style slow code from an interpreter? I can certainly see how that would substantially reduce cpu and memory cost but the code would execute 50-250x slower.
- incrudible 3y agoHere are some benchmarks: https://bellard.org/quickjs/bench.html https://bellard.org/quickjs/bench.html If startup time is relevant to you then you are probably not doing much computation in the first place.
- ushakov 3y agoMakes sense for a project like that. Lambdas are mostly used to glue AWS APIs anyways
- jFriedensreich 3y agoi wonder why no one is talking about workerd, cloudflares much more mature approach that is behind cloudflare workers. deno is great but has much broader scope than llrt, workerd or winterjs. these are really low latency edge runtimes not general purpose ones.
- grav 3y agoFor ref: - https://blog.cloudflare.com/workerd-open-source-workers-runtime/ https://blog.cloudflare.com/workerd-open-source-workers-runt... - https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd
- AntonCTO 3y agoIf I understand it correctly, it exchanges execution speed (by not using JIT) for faster starts. Instead, they could pick up the old FB project https://prepack.io/ https://prepack.io/ to optimize both start AND execution speed. It would be very interesting to see it together with node's snapshots: https://blog.logrocket.com/snapshot-flags-node-js-v18-8/ https://blog.logrocket.com/snapshot-flags-node-js-v18-8/, which reduces the start significantly.
- bvarbanov 3y agoI am working on similar runtime QuickJS/C/C++ using async stackful coroutines. Last year tests were about 14 times faster for networking related to node.js Here are tests results: https://voda.io/test.html https://voda.io/test.html I plan open source it later this year - have improve build and remove some client's code.
- WuxiFingerHold 3y agoI think Jarred Sumner (creator of bun) summarised it well: Only faster for code that runs under 5 ms. But a lot of code runs under 5 ms. (I hope the quote is right, I'm not on twitter.) I think it makes Lambda better, but not more attractable to use cases that are better served by scale-to-zero managed containers runtimes like Fargate.