6 ms·

I understand exactly what you mean at the conceptual level, but I've consistently found speedups over c++ in complex applications.

by junon 4y ago

pclmulqdq 4y ago
There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. The examples I have seen where a Rust rewrite is faster than original C++ code usually have glaring performance issues in the C++: Often, it is code that does a ton of unnecessary copies. The Rust code often doesn't copy as much because the Rust language makes that kind of copying inconvenient or because the person doing the rewrite is a Rust expert and not a C++ expert. The other kind of speedup I have seen is when people swap a C++ std::map for a Rust btree_map, and don't realize that absl::btree_map (a C++ btree map in Google's library) is what Rust's btree_map is based on. I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite.
- herbstein 4y ago
  > There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. Cliff Biffle's great series of blog posts called "Learn Rust the Dangerous Way" takes the fastest C program from a benchmark game entry, rewrites it naïvely in Rust using copious amounts of `unsafe`, and then transforms it into program without `unsafe` while keeping it idiomatic. The last version is faster than the C implementation. It's a toy example but it's real-world example. http://cliffle.com/p/dangerust/ http://cliffle.com/p/dangerust/
  - pclmulqdq 4y ago
    I'll read it and take a look. I am aware that Rust does very well on the benchmarks game. However, the benchmarks game (and microbenchmark-based comparisons in general) is hard to take seriously if you are thinking about application performance. Some languages microbenchmark very well, but don't translate that to system performance (C is the poster child of this effect), and some microbenchmark poorly but work very well in practical systems (Go is the most popular language with a big gap here, but some functional language like OCaml or Haskell probably has the biggest gap). The reasons for these gaps can include things like it being harder to use the optimal data structure for your application (eg C code using red-black trees instead of btrees in 2023) and large code size causing terrible caching behavior (heavily templated C++). I also remember seeing something here about some non-optimal calling convention in the Rust compiler, which would be another thing that shows up in a system that doesn't in a microbenchmark. Microbenchmarks are not a good replacement for system-level comparisons.
    - igouy 4y ago
      > … hard to take seriously… Even when we're shown "Benchmarks are a crock" ? https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#benchmarks-are-a-crock https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > Microbenchmarks are not a good replacement for system-level comparisons. For example — https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html#reimplement https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
      - pclmulqdq 4y ago
        I think we're in agreement here. Personally, I was really looking forward to seeing "we rewrote our database in Rust from C++ and QPS improved 5% because fast code was easier to write." This kind of benchmark is not BS, and actually would save you real money. Instead, the main arguments for the claim that Rust is the same speed as C++ are microbenchmarks, which actually are pretty useless when you are comparing very different implementations. The time they are useful is when you are refining an implementation. If you want to scrutinize the benchmarks game even further, they use gcc for their c compiler, where clang would probably be a better choice since the benchmarks are arithmetic-heavy.
        nequo 4y ago
        When you get sufficiently close to the metal, the performance gains seem to be coming from better algorithms rather than from the programming language. Like you are saying: > The reasons for these gaps can include things like it being harder to use the optimal data structure for your application (eg C code using red-black trees instead of btrees in 2023) Bryan Cantrill described this experience[1] and maybe that is exactly what you are referring to. But if you're just looking for a language to get work done with, does it matter if Rust is faster because of better off-the-shelf data structures and algorithms or because of some inherent magic in the programming language? [1] See point 9 here: http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/ http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-...
  - igouy 4y ago
    Also gcc #9 n-body program was converted to Rust #9 — https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.html#intrinsics https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
    - dralley 4y ago
      You've linked to an implementation full of hand-written SIMD, I'm not sure that's a great comparison one way or the other. Which is my biggest gripe with benchmark games. There really ought to be strict categories for "straightforward, idiomatic code someone with 3 YoE could write", "optimized-but-still maintainable code an experienced senior engineer would write", and "unrestricted wizardry".
      - igouy 4y ago
        Please provide a magic trick that will assign programs to "strict categories" in a way that no one will dispute :-) > … straightforward, idiomatic code… Even with tiny programs, once you ask for "idiomatic code" things stop being straightforward: different languages do things differently — https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/simple.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > … hand-written SIMD… One approach is to filter those programs into their own section — hand-written vector instructions | "unsafe". Another is to use source code size as a proxy — https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody-gz.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
      - nequo 4y ago
        It seems inherently difficult to define strict categories around the soft constraints that you are listing. But it would be interesting to try to approximate something like those categories. The project repo[1] contains the following call for idiomatic code: Please, people ask to see more "idiomatic" programs — - we already have enough exhaustively optimized Rust and C programs. - we already have enough hand-written vector SIMD and "unsafe" programs. Thank you. [1] https://salsa.debian.org/benchmarksgame-team/benchmarksgame https://salsa.debian.org/benchmarksgame-team/benchmarksgame
- marcosdumay 4y ago
  > or because the person doing the rewrite is a Rust expert and not a C++ expert On this subject... it's much easier to become a Rust expert than a C++ expert. I do expect the world to have more Rust experts than C++ ones today, even with much lower usage of the language and much less time for people to learn it.
- dralley 4y ago
  >Often, it is code that does a ton of unnecessary copies. Often, code written in C / C++ does unnecessary copies precisely because the risk of avoiding those copies is hours of debugging, crashes in production, or security flaws. So this is a legitimate benefit of Rust. Being able to code more aggressively up front, without fear of something blowing up in your face later, or much later when the intern touches the wrong line. >I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite. Here's one from Bryan Cantrill http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/ http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performa... The point he makes is that while Rust is not strictly faster than C in most cases, it gives you the comfort of being able to use more highly optimized libraries and data structures without needing to have the extreme level of trust in the author that you do in C / C++. This is basically the same argument.
  - pclmulqdq 4y ago
    This is a comparison with C that I am familiar with. It illustrates the reasons why C++ won for many applications (compared to C). The author also is on record as someone who hates C++, and is very careful about not making any claims in that blog post about C++ because that's not his area of expertise. Compared to C, where a data structure library is at best a heap of cobbled-together macros that make a lot of assumptions about the underlying types, C++ has well-developed libraries for all of the data structures mentioned. And you need to trust the author plenty to make a btree that works - that is a very complicated data structure. It's not just about memory safety, there can be a lot of functional bugs, too. The author of the C++ one is Google/Facebook, who I trust to write a solid library, while the author of the Rust one is someone who was inspired by the C++ version. At this point, both are well-tested, but there are a lot of Rust libs that are in the same position, but relatively untested.
    - bcantrill 4y ago
      True: not my area of expertise as I have only written ~100,000 lines of C++ -- and that that is insufficient for C++ to be an area of expertise is itself very revealing...
TylerE 4y ago
I bet you'd see speedups if you rewrote in C++, too.
- toss1 4y ago
  Good point, but what is the comparative effort for both of those rewrites making similar performance improvements, and what skill level in the coder is needed to realize those similar improvements? E.g., will the C++ rewrite for X improvement take longer, or require a more highly skilled programmer, or would the Rust programmer require some high level of expertise to yield the results but a junior C++ guy/gal could knock it out quickly? Has anyone done real comparisons of this sort, such as taking software package X and assigning a performance rewrite to a junior Rust dev, a senor Rust dev, a junior C++ dev and a senior C++ dev, giving each a couple weeks then checking the yielded performance? Also, how does this interact on teams? It's been a long time since I've written any C++, and I've not tried Rust, but it seems some of the Rust safety features make it harder for devs to step on each other's toes.
- junon 4y ago
  Not sure your point. Going from C++ to Rust, is what I meant.
  - nequo 4y ago
    Parent may have meant that the second implementation might perform better, even if it’s in the same language, because you learn from the first implementation.
    - TylerE 4y ago
      Precisely. The old “build one to throw away”

I understand exactly what you mean at the conceptual level, but I've consistently found speedups over c++ in complex applications.

by junon 4y ago

I understand exactly what you mean at the conceptual level, but I've consistently found speedups over c++ in complex applications.

pclmulqdq 4y ago

There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. The examples I have seen where a Rust rewrite is faster than original C++ code usually have glaring performance issues in the C++: Often, it is code that does a ton of unnecessary copies. The Rust code often doesn't copy as much because the Rust language makes that kind of copying inconvenient or because the person doing the rewrite is a Rust expert and not a C++ expert. The other kind of speedup I have seen is when people swap a C++ std::map for a Rust btree_map, and don't realize that absl::btree_map (a C++ btree map in Google's library) is what Rust's btree_map is based on. I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite.

herbstein 4y ago
> There is a lot of C++ code out there. A lot of it was not written by people who understand performance or who care. It is very easy to take that kind of code and make it faster, even without a switch to Rust. Cliff Biffle's great series of blog posts called "Learn Rust the Dangerous Way" takes the fastest C program from a benchmark game entry, rewrites it naïvely in Rust using copious amounts of `unsafe`, and then transforms it into program without `unsafe` while keeping it idiomatic. The last version is faster than the C implementation. It's a toy example but it's real-world example. http://cliffle.com/p/dangerust/ http://cliffle.com/p/dangerust/
- pclmulqdq 4y ago
  I'll read it and take a look. I am aware that Rust does very well on the benchmarks game. However, the benchmarks game (and microbenchmark-based comparisons in general) is hard to take seriously if you are thinking about application performance. Some languages microbenchmark very well, but don't translate that to system performance (C is the poster child of this effect), and some microbenchmark poorly but work very well in practical systems (Go is the most popular language with a big gap here, but some functional language like OCaml or Haskell probably has the biggest gap). The reasons for these gaps can include things like it being harder to use the optimal data structure for your application (eg C code using red-black trees instead of btrees in 2023) and large code size causing terrible caching behavior (heavily templated C++). I also remember seeing something here about some non-optimal calling convention in the Rust compiler, which would be another thing that shows up in a system that doesn't in a microbenchmark. Microbenchmarks are not a good replacement for system-level comparisons.
  - igouy 4y ago
    > … hard to take seriously… Even when we're shown "Benchmarks are a crock" ? https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#benchmarks-are-a-crock https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > Microbenchmarks are not a good replacement for system-level comparisons. For example — https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html#reimplement https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
    - pclmulqdq 4y ago
      I think we're in agreement here. Personally, I was really looking forward to seeing "we rewrote our database in Rust from C++ and QPS improved 5% because fast code was easier to write." This kind of benchmark is not BS, and actually would save you real money. Instead, the main arguments for the claim that Rust is the same speed as C++ are microbenchmarks, which actually are pretty useless when you are comparing very different implementations. The time they are useful is when you are refining an implementation. If you want to scrutinize the benchmarks game even further, they use gcc for their c compiler, where clang would probably be a better choice since the benchmarks are arithmetic-heavy.
      - nequo 4y ago
        When you get sufficiently close to the metal, the performance gains seem to be coming from better algorithms rather than from the programming language. Like you are saying: > The reasons for these gaps can include things like it being harder to use the optimal data structure for your application (eg C code using red-black trees instead of btrees in 2023) Bryan Cantrill described this experience[1] and maybe that is exactly what you are referring to. But if you're just looking for a language to get work done with, does it matter if Rust is faster because of better off-the-shelf data structures and algorithms or because of some inherent magic in the programming language? [1] See point 9 here: http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/ http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-...
- igouy 4y ago
  Also gcc #9 n-body program was converted to Rust #9 — https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.html#intrinsics https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
  - dralley 4y ago
    You've linked to an implementation full of hand-written SIMD, I'm not sure that's a great comparison one way or the other. Which is my biggest gripe with benchmark games. There really ought to be strict categories for "straightforward, idiomatic code someone with 3 YoE could write", "optimized-but-still maintainable code an experienced senior engineer would write", and "unrestricted wizardry".
    - igouy 4y ago
      Please provide a magic trick that will assign programs to "strict categories" in a way that no one will dispute :-) > … straightforward, idiomatic code… Even with tiny programs, once you ask for "idiomatic code" things stop being straightforward: different languages do things differently — https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/simple.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > … hand-written SIMD… One approach is to filter those programs into their own section — hand-written vector instructions | "unsafe". Another is to use source code size as a proxy — https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody-gz.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
    - nequo 4y ago
      It seems inherently difficult to define strict categories around the soft constraints that you are listing. But it would be interesting to try to approximate something like those categories. The project repo[1] contains the following call for idiomatic code: Please, people ask to see more "idiomatic" programs — - we already have enough exhaustively optimized Rust and C programs. - we already have enough hand-written vector SIMD and "unsafe" programs. Thank you. [1] https://salsa.debian.org/benchmarksgame-team/benchmarksgame https://salsa.debian.org/benchmarksgame-team/benchmarksgame
marcosdumay 4y ago
> or because the person doing the rewrite is a Rust expert and not a C++ expert On this subject... it's much easier to become a Rust expert than a C++ expert. I do expect the world to have more Rust experts than C++ ones today, even with much lower usage of the language and much less time for people to learn it.
dralley 4y ago
>Often, it is code that does a ton of unnecessary copies. Often, code written in C / C++ does unnecessary copies precisely because the risk of avoiding those copies is hours of debugging, crashes in production, or security flaws. So this is a legitimate benefit of Rust. Being able to code more aggressively up front, without fear of something blowing up in your face later, or much later when the intern touches the wrong line. >I haven't seen any examples where C or C++ code written with performance in mind gets a rewrite to Rust and goes faster, and I have seen the opposite. Here's one from Bryan Cantrill http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/ http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performa... The point he makes is that while Rust is not strictly faster than C in most cases, it gives you the comfort of being able to use more highly optimized libraries and data structures without needing to have the extreme level of trust in the author that you do in C / C++. This is basically the same argument.
- pclmulqdq 4y ago
  This is a comparison with C that I am familiar with. It illustrates the reasons why C++ won for many applications (compared to C). The author also is on record as someone who hates C++, and is very careful about not making any claims in that blog post about C++ because that's not his area of expertise. Compared to C, where a data structure library is at best a heap of cobbled-together macros that make a lot of assumptions about the underlying types, C++ has well-developed libraries for all of the data structures mentioned. And you need to trust the author plenty to make a btree that works - that is a very complicated data structure. It's not just about memory safety, there can be a lot of functional bugs, too. The author of the C++ one is Google/Facebook, who I trust to write a solid library, while the author of the Rust one is someone who was inspired by the C++ version. At this point, both are well-tested, but there are a lot of Rust libs that are in the same position, but relatively untested.
  - bcantrill 4y ago
    True: not my area of expertise as I have only written ~100,000 lines of C++ -- and that that is insufficient for C++ to be an area of expertise is itself very revealing...

TylerE 4y ago

I bet you'd see speedups if you rewrote in C++, too.

toss1 4y ago
Good point, but what is the comparative effort for both of those rewrites making similar performance improvements, and what skill level in the coder is needed to realize those similar improvements? E.g., will the C++ rewrite for X improvement take longer, or require a more highly skilled programmer, or would the Rust programmer require some high level of expertise to yield the results but a junior C++ guy/gal could knock it out quickly? Has anyone done real comparisons of this sort, such as taking software package X and assigning a performance rewrite to a junior Rust dev, a senor Rust dev, a junior C++ dev and a senior C++ dev, giving each a couple weeks then checking the yielded performance? Also, how does this interact on teams? It's been a long time since I've written any C++, and I've not tried Rust, but it seems some of the Rust safety features make it harder for devs to step on each other's toes.
junon 4y ago
Not sure your point. Going from C++ to Rust, is what I meant.
- nequo 4y ago
  Parent may have meant that the second implementation might perform better, even if it’s in the same language, because you learn from the first implementation.
  - TylerE 4y ago
    Precisely. The old “build one to throw away”