4 ms·
I wrote this comment based on pre-release benchmarks, based on Physical benchmarks run against Git commits from Jan 23 - but it should still largely apply to th
by lstamour 6y ago
I wrote this comment based on pre-release benchmarks, based on Physical benchmarks run against Git commits from Jan 23 - but it should still largely apply to the final benchmarks:
It’s kind of frustrating to spend a few hours investigating preview results of the latest techempower benchmarks just to come to the conclusion that (a) truly high performance requires tuning your code to linux, to the network interface cards, and a deep understanding of what your code is doing and (b) right now at the top of the benchmarks, common high performance Rust code is mostly “unsafe” memory-wise, and the same is true of C++ for obvious reasons. After that:
* When looking at garbage collected languages with very minimal implementations (Jooby mostly), Java can come out on top but requires 4 Gigs of memory which is more than I would ever like to use.
* After Java comes C# but performance slows down the second you want to do anything less optimized, like use third-party libraries.
* Finally, we’ve Go, which in a surprising twist is basically neck and neck with some pure PHP server implementations, but once I include those, I might as well mention there’s a JS implementation that skips the Node.js async event loop and offers performance on par with C++ and Rust, but heavily uses C++ internally and is thus more “unsafe”.
What this tells me is kind of what I expected, implementation matters more than language… at this point, memory usage aside for Java, an efficient HTTP/1.1 server can be implemented in basically any language and when tuned or stripped down tends to run faster than the commonly used web servers, like Nginx, Tomcat, Express, etc. Often this means writing plain text RAW HTTP to a socket and managing sockets efficiently.
Which brings us back to Go, though. Of all the implementations I’ve looked at so far, even the PHP one, only the Go benchmark is written like “standard Go” such that most people writing a service in Go will write something high performance without trying to make it high performance. Effectively, you don’t need to ensure all your dependencies use special optimized code routines to get something relatively optimized working quickly in Go with low memory overhead unlike Java.
I’m a bit shocked by this conclusion as I was really hoping that Rust’s high performance use cases would win out, as it’s true that Rust can get 3x faster than Go, but on the Rust side, both actix and ntex are too immature as neither’s hit 1.0 yet, while tokio has hit 1.0 but its server, warp, is slower than Go.
Irritating is that each of the techempower benchmarks use different implementations. For example, there’s a benchmark that’s supposed to measure 20 calls to Postgres, but one benchmark gets to the top by making only one long SQL statement that changes 20 rows. Another implementation uses pgx’s Batch functionality to send multiple queries in batches (a big timesaver, but not technically standard libpq), but then the standard Go variant doesn’t use Batching even when it could (which means we can’t compare custom implementations to generic Go ones fairly): https://github.com/TechEmpower/FrameworkBenchmarks/search?l=Go&q=batch https://github.com/TechEmpower/FrameworkBenchmarks/search?l=...
- newlisper 6y agoWhere did you see memory consumption results?
- billywhizz 6y agothere's a nice tool here for visualising the results in more detail... https://ajdust.github.io/tfbvis/?testrun=Citrine_started2020-12-29_f78282e3-ed37-4e7c-9862-3163de23b23b&testtype=plaintext&round=false https://ajdust.github.io/tfbvis/?testrun=Citrine_started2020...
- dmm 6y ago> right now at the top of the benchmarks, common high performance Rust code is mostly “unsafe” memory-wise I only see a single "unsafe" in the Rust implementation[0]? And that's only in the "raw" Actix instance, not the pg. Or are you talking about Actix framework being mostly unsafe? > only the Go benchmark is written like “standard Go” such that most people writing a service in Go will write Maybe it's just me but the Actix apis seem pretty ordinary. [0] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Rust/actix/src/main_raw.rs#L119 https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
- lstamour 6y ago> about Actix framework being mostly unsafe Yep, that's what I meant. To me, until Rust unsafe implementations hit 1.0 as web frameworks, not just Tokio, I can't consider them as stable or bug-free as more mature web servers. It's arbitrary, yes, but it's also an important distinction. 1.0 generally means "you can rely on this to not change, and to have a useful set of functionality" and bug fixes equivalent to an LTS release. This is especially true if functionality is common enough that it makes its way into a standard library or conventional best practices. Generally, it has to hit "1.0" before it does so, though, in any stable manner...
- covidthrow 6y agoThe Actix unsafe code has long since been resolved. There are very few instances of it, and each are small, specific, easy to reason about, and wholly internal. I'd guess the risk is about on par with—or safer than—VM runtime languages written in C/++. Far as 1.0 in Rust, the predominant culture is to treat "0.X.x" releases roughly equivalent to a "1.x" semantic. I have run into very few issues of API breaking changes in "0.X.x" libraries, and even those are basically handed to you thanks to Rust's robust compile-time checks. Of course your mileage may vary, but I've had a way worse time with even well-established breaks in Ruby, Python, and Go libraries. I understand your risk aversion; I am similar. But I've found Rust to be an absolute delight for stability. The shaky ground of the first few years has largely given way to impressive robustness over the last few. I strongly urge you to consider Rust if it matches your needs. You may be quite pleased!