3 ms·

This has been my experience with Rust as well. The safety actually does cost some performance. Just like how C++'s "zero-cost" abstractions cost something. I

This has been my experience with Rust as well. The safety actually does cost some performance. Just like how C++'s "zero-cost" abstractions cost something. It seems like a straightforward trade between speed and safety to me, but nobody in the Rust community wants to admit it. As such, there aren't a lot of public benchmarks of applications out there that can help the rest of us figure out when we actually should use rust - when the performance is worth the safety (no, Rust evangelists, that kind of safety is not always worth it, and no, we don't always pay the same costs for it).

junon 4y ago
I understand exactly what you mean at the conceptual level, but I've consistently found speedups over c++ in complex applications.
- pclmulqdq 4y ago
  There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. The examples I have seen where a Rust rewrite is faster than original C++ code usually have glaring performance issues in the C++: Often, it is code that does a ton of unnecessary copies. The Rust code often doesn't copy as much because the Rust language makes that kind of copying inconvenient or because the person doing the rewrite is a Rust expert and not a C++ expert. The other kind of speedup I have seen is when people swap a C++ std::map for a Rust btree_map, and don't realize that absl::btree_map (a C++ btree map in Google's library) is what Rust's btree_map is based on. I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite.
  - herbstein 4y ago
    > There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. Cliff Biffle's great series of blog posts called "Learn Rust the Dangerous Way" takes the fastest C program from a benchmark game entry, rewrites it naïvely in Rust using copious amounts of `unsafe`, and then transforms it into program without `unsafe` while keeping it idiomatic. The last version is faster than the C implementation. It's a toy example but it's real-world example. http://cliffle.com/p/dangerust/ http://cliffle.com/p/dangerust/
    - pclmulqdq 4y ago
      I'll read it and take a look. I am aware that Rust does very well on the benchmarks game. However, the benchmarks game (and microbenchmark-based comparisons in general) is hard to take seriously if you are thinking about application performance. Some languages microbenchmark very well, but don't translate that to system performance (C is the poster child of this effect), and some microbenchmark poorly but work very well in practical systems (Go is the most popular language with a big gap here, but some functional language like OCaml or Haskell probably has the biggest gap). The reasons for these gaps can include things like it being harder to use the optimal data structure for your application (eg C code using red-black trees instead of btrees in 2023) and large code size causing terrible caching behavior (heavily templated C++). I also remember seeing something here about some non-optimal calling convention in the Rust compiler, which would be another thing that shows up in a system that doesn't in a microbenchmark. Microbenchmarks are not a good replacement for system-level comparisons.
      - igouy 4y ago
        > … hard to take seriously… Even when we're shown "Benchmarks are a crock" ? https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#benchmarks-are-a-crock https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > Microbenchmarks are not a good replacement for system-level comparisons. For example — https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html#reimplement https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
        pclmulqdq 4y ago
        I think we're in agreement here. Personally, I was really looking forward to seeing "we rewrote our database in Rust from C++ and QPS improved 5% because fast code was easier to write." This kind of benchmark is not BS, and actually would save you real money. Instead, the main arguments for the claim that Rust is the same speed as C++ are microbenchmarks, which actually are pretty useless when you are comparing very different implementations. The time they are useful is when you are refining an implementation. If you want to scrutinize the benchmarks game even further, they use gcc for their c compiler, where clang would probably be a better choice since the benchmarks are arithmetic-heavy.
  - marcosdumay 4y ago
    > or because the person doing the rewrite is a Rust expert and not a C++ expert On this subject... it's much easier to become a Rust expert than a C++ expert. I do expect the world to have more Rust experts than C++ ones today, even with much lower usage of the language and much less time for people to learn it.
  - dralley 4y ago
    >Often, it is code that does a ton of unnecessary copies. Often, code written in C / C++ does unnecessary copies precisely because the risk of avoiding those copies is hours of debugging, crashes in production, or security flaws. So this is a legitimate benefit of Rust. Being able to code more aggressively up front, without fear of something blowing up in your face later, or much later when the intern touches the wrong line. >I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite. Here's one from Bryan Cantrill http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/ http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performa... The point he makes is that while Rust is not strictly faster than C in most cases, it gives you the comfort of being able to use more highly optimized libraries and data structures without needing to have the extreme level of trust in the author that you do in C / C++. This is basically the same argument.
    - pclmulqdq 4y ago
      This is a comparison with C that I am familiar with. It illustrates the reasons why C++ won for many applications (compared to C). The author also is on record as someone who hates C++, and is very careful about not making any claims in that blog post about C++ because that's not his area of expertise. Compared to C, where a data structure library is at best a heap of cobbled-together macros that make a lot of assumptions about the underlying types, C++ has well-developed libraries for all of the data structures mentioned. And you need to trust the author plenty to make a btree that works - that is a very complicated data structure. It's not just about memory safety, there can be a lot of functional bugs, too. The author of the C++ one is Google/Facebook, who I trust to write a solid library, while the author of the Rust one is someone who was inspired by the C++ version. At this point, both are well-tested, but there are a lot of Rust libs that are in the same position, but relatively untested.
      - bcantrill 4y ago
        True: not my area of expertise as I have only written ~100,000 lines of C++ -- and that that is insufficient for C++ to be an area of expertise is itself very revealing...
- TylerE 4y ago
  I bet you'd see speedups if you rewrote in C++, too.
  - toss1 4y ago
    Good point, but what is the comparative effort for both of those rewrites making similar performance improvements, and what skill level in the coder is needed to realize those similar improvements? E.g., will the C++ rewrite for X improvement take longer, or require a more highly skilled programmer, or would the Rust programmer require some high level of expertise to yield the results but a junior C++ guy/gal could knock it out quickly? Has anyone done real comparisons of this sort, such as taking software package X and assigning a performance rewrite to a junior Rust dev, a senor Rust dev, a junior C++ dev and a senior C++ dev, giving each a couple weeks then checking the yielded performance? Also, how does this interact on teams? It's been a long time since I've written any C++, and I've not tried Rust, but it seems some of the Rust safety features make it harder for devs to step on each other's toes.
  - junon 4y ago
    Not sure your point. Going from C++ to Rust, is what I meant.
    - nequo 4y ago
      Parent may have meant that the second implementation might perform better, even if it’s in the same language, because you learn from the first implementation.
      - TylerE 4y ago
        Precisely. The old “build one to throw away”
jjtheblunt 4y ago
> The safety actually does cost some performance. I thought the safety was compile time optimization and semantic analysis, to enable code generated to feature performance optimization that reflects in improved performance at runtime. So, where are you seeing performance penalties?
- pclmulqdq 4y ago
  I am not enough of a Rust compiler expert to tell you exactly where the slowdowns show up, but the common theme has been that a benchmark of a small section of the critical path shows equal performance (sometimes a little faster, sometimes a little slower), and then when you expand to a larger scope, benchmarks start to slow down in comparison to their C++ alternatives. My intuitive guesses would be that the culprit is actually either larger code size, worse code layout, being unable to optimize certain code sections to the same level (undefined behavior in C++ is actually helpful for speed, and not all of it is actually unsafe), something about how calling conventions or struct layouts differ, or another "second-order" effect.
  - jjtheblunt 4y ago
    that's interesting ... thank you
pkolaczk 4y ago
This series of posts disagrees: http://cliffle.com/p/dangerust/1/ http://cliffle.com/p/dangerust/1/ Spoiler: the end result uses idiomatic Rust, doesn't use unsafe and is faster than the hand-optimized version in C (and hand optimized one in Rust).
deleted 4y ago
[deleted]

junon 4y ago

I understand exactly what you mean at the conceptual level, but I've consistently found speedups over c++ in complex applications.

pclmulqdq 4y ago
There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. The examples I have seen where a Rust rewrite is faster than original C++ code usually have glaring performance issues in the C++: Often, it is code that does a ton of unnecessary copies. The Rust code often doesn't copy as much because the Rust language makes that kind of copying inconvenient or because the person doing the rewrite is a Rust expert and not a C++ expert. The other kind of speedup I have seen is when people swap a C++ std::map for a Rust btree_map, and don't realize that absl::btree_map (a C++ btree map in Google's library) is what Rust's btree_map is based on. I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite.
- herbstein 4y ago
  > There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. Cliff Biffle's great series of blog posts called "Learn Rust the Dangerous Way" takes the fastest C program from a benchmark game entry, rewrites it naïvely in Rust using copious amounts of `unsafe`, and then transforms it into program without `unsafe` while keeping it idiomatic. The last version is faster than the C implementation. It's a toy example but it's real-world example. http://cliffle.com/p/dangerust/ http://cliffle.com/p/dangerust/
  - pclmulqdq 4y ago
    I'll read it and take a look. I am aware that Rust does very well on the benchmarks game. However, the benchmarks game (and microbenchmark-based comparisons in general) is hard to take seriously if you are thinking about application performance. Some languages microbenchmark very well, but don't translate that to system performance (C is the poster child of this effect), and some microbenchmark poorly but work very well in practical systems (Go is the most popular language with a big gap here, but some functional language like OCaml or Haskell probably has the biggest gap). The reasons for these gaps can include things like it being harder to use the optimal data structure for your application (eg C code using red-black trees instead of btrees in 2023) and large code size causing terrible caching behavior (heavily templated C++). I also remember seeing something here about some non-optimal calling convention in the Rust compiler, which would be another thing that shows up in a system that doesn't in a microbenchmark. Microbenchmarks are not a good replacement for system-level comparisons.
    - igouy 4y ago
      > … hard to take seriously… Even when we're shown "Benchmarks are a crock" ? https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#benchmarks-are-a-crock https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > Microbenchmarks are not a good replacement for system-level comparisons. For example — https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html#reimplement https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
      - pclmulqdq 4y ago
        I think we're in agreement here. Personally, I was really looking forward to seeing "we rewrote our database in Rust from C++ and QPS improved 5% because fast code was easier to write." This kind of benchmark is not BS, and actually would save you real money. Instead, the main arguments for the claim that Rust is the same speed as C++ are microbenchmarks, which actually are pretty useless when you are comparing very different implementations. The time they are useful is when you are refining an implementation. If you want to scrutinize the benchmarks game even further, they use gcc for their c compiler, where clang would probably be a better choice since the benchmarks are arithmetic-heavy.
- marcosdumay 4y ago
  > or because the person doing the rewrite is a Rust expert and not a C++ expert On this subject... it's much easier to become a Rust expert than a C++ expert. I do expect the world to have more Rust experts than C++ ones today, even with much lower usage of the language and much less time for people to learn it.
- dralley 4y ago
  >Often, it is code that does a ton of unnecessary copies. Often, code written in C / C++ does unnecessary copies precisely because the risk of avoiding those copies is hours of debugging, crashes in production, or security flaws. So this is a legitimate benefit of Rust. Being able to code more aggressively up front, without fear of something blowing up in your face later, or much later when the intern touches the wrong line. >I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite. Here's one from Bryan Cantrill http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/ http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performa... The point he makes is that while Rust is not strictly faster than C in most cases, it gives you the comfort of being able to use more highly optimized libraries and data structures without needing to have the extreme level of trust in the author that you do in C / C++. This is basically the same argument.
  - pclmulqdq 4y ago
    This is a comparison with C that I am familiar with. It illustrates the reasons why C++ won for many applications (compared to C). The author also is on record as someone who hates C++, and is very careful about not making any claims in that blog post about C++ because that's not his area of expertise. Compared to C, where a data structure library is at best a heap of cobbled-together macros that make a lot of assumptions about the underlying types, C++ has well-developed libraries for all of the data structures mentioned. And you need to trust the author plenty to make a btree that works - that is a very complicated data structure. It's not just about memory safety, there can be a lot of functional bugs, too. The author of the C++ one is Google/Facebook, who I trust to write a solid library, while the author of the Rust one is someone who was inspired by the C++ version. At this point, both are well-tested, but there are a lot of Rust libs that are in the same position, but relatively untested.
    - bcantrill 4y ago
      True: not my area of expertise as I have only written ~100,000 lines of C++ -- and that that is insufficient for C++ to be an area of expertise is itself very revealing...
TylerE 4y ago
I bet you'd see speedups if you rewrote in C++, too.
- toss1 4y ago
  Good point, but what is the comparative effort for both of those rewrites making similar performance improvements, and what skill level in the coder is needed to realize those similar improvements? E.g., will the C++ rewrite for X improvement take longer, or require a more highly skilled programmer, or would the Rust programmer require some high level of expertise to yield the results but a junior C++ guy/gal could knock it out quickly? Has anyone done real comparisons of this sort, such as taking software package X and assigning a performance rewrite to a junior Rust dev, a senor Rust dev, a junior C++ dev and a senior C++ dev, giving each a couple weeks then checking the yielded performance? Also, how does this interact on teams? It's been a long time since I've written any C++, and I've not tried Rust, but it seems some of the Rust safety features make it harder for devs to step on each other's toes.
- junon 4y ago
  Not sure your point. Going from C++ to Rust, is what I meant.
  - nequo 4y ago
    Parent may have meant that the second implementation might perform better, even if it’s in the same language, because you learn from the first implementation.
    - TylerE 4y ago
      Precisely. The old “build one to throw away”

jjtheblunt 4y ago

> The safety actually does cost some performance. I thought the safety was compile time optimization and semantic analysis, to enable code generated to feature performance optimization that reflects in improved performance at runtime. So, where are you seeing performance penalties?

pclmulqdq 4y ago
I am not enough of a Rust compiler expert to tell you exactly where the slowdowns show up, but the common theme has been that a benchmark of a small section of the critical path shows equal performance (sometimes a little faster, sometimes a little slower), and then when you expand to a larger scope, benchmarks start to slow down in comparison to their C++ alternatives. My intuitive guesses would be that the culprit is actually either larger code size, worse code layout, being unable to optimize certain code sections to the same level (undefined behavior in C++ is actually helpful for speed, and not all of it is actually unsafe), something about how calling conventions or struct layouts differ, or another "second-order" effect.
- jjtheblunt 4y ago
  that's interesting ... thank you

pkolaczk 4y ago

This series of posts disagrees: http://cliffle.com/p/dangerust/1/ http://cliffle.com/p/dangerust/1/ Spoiler: the end result uses idiomatic Rust, doesn't use unsafe and is faster than the hand-optimized version in C (and hand optimized one in Rust).

deleted 4y ago

[deleted]