4 ms·

Great writeup. This part stuck out to me: > The justification for "unsafe" C and C++ libraries is often that some performance is left on the table with safe Ru

Great writeup. This part stuck out to me: > The justification for "unsafe" C and C++ libraries is often that some performance is left on the table with safe Rust, primarily due to bounds checking. On modern hardware this justification has always seemed a bit suspicious to me, and I think this is another piece of (circumstantial) evidence that it often just doesn't matter in practice That's too strong a statement even though the sentiment is correct. Most software will not see a bottleneck because of bounds checking because it's not in the fast path. All we can say in this case is that the work that libyaml was doing is not impacted by bounds checking. It's not because of hardware but because of software inefficiencies / problem being solved isn't impacted by bounds checking - most likely libyaml is not running at full speed on the underlying hardware and thus any bounds checking is just noise because there's so much slack already. However, I have definitely encountered situations in very specific circumstances where bounds checking is a bottleneck when you're trying to actually run at hardware speeds and unsafe is required to recover performance - of course I benchmarked very very carefully to the specific unsafe that's the problem. I'll also give a shoutout to the assume crate which I've found a much nicer mechanism to cause the compiler to elide bounds checking in many cases when building optimized while keeping it for debug builds & validating your assertion is correct. The annoying bit is that you can't ask the Rust compiler to elide all bounds check for a benchmark to validate the impact of the bounds checking. Remember - on modern hardware your CPU is capable of executing ~4-6 billion operations / s, process main memory at ~100GB/s, and access disk at millions of times / s at throughputs of ~1-2 GB/s. This is all consumer grade hardware. Typical software applications are not typically targeting to try to run at full utilization but instead try to offer reasonable performance with a more straightforward solution that can be supported as cheaply as possible maintenance wise because the bottleneck cost is typically the programmer's time rather than how fast the software can be made to run.

TylerE 3y ago
Throughputs on modern hardware are insane. Latency/time-til-first-byte can often still be pretty sucky.
- vlovich123 3y ago
  Care to give an example? A disk read is comparatively slow to compute/memory but it’s still in the tens or hundreds of microseconds if I recall correctly and memory is hundreds of nanoseconds. I am convinced latency issues are a remnant of how we build abstractions into software stacks and not a fault of the hardware.
  - TylerE 3y ago
    Now have 10 or 15 apps hammering away at the same disk at the same time and see how it behaves. Even SSDs slow down as read sizes go down.
  - TillE 3y ago
    Cache misses will destroy the performance of any CPU-intensive program. You really really want to do mostly sequential reads of RAM if you're processing a lot of data.
epage 3y ago
Unless this was updated after your post, I think it was covered in the following line: > It's certainly possible to build things where bounds checking makes a difference. This just isn't that. I will add that idiomatic Rust causes a good number of bounds checks to be optimized away. Its impressive the difference this can make. I maintain a parser combinator library that started as a fork. With some simple changes, I've seen massive performance improvements and my best guess is that those changes allowed bounds checks to be removed.
- norman784 3y ago
  Can you share what kind of changes, in your case, improved the performance?
  - wredue 3y ago
    Avoid directly indexing if possible. Use the collection functions and slices instead.
    - epage 3y ago
      I'd add careful use of inlining so the compiler can see both the bounds check and the index to know the bounds check can be removed.
    - deleted 3y ago
      [deleted]
    - deleted 3y ago
      [deleted]
- vlovich123 3y ago
  Possible I missed it when I read it. Would need the author to indicate whether or not it was edited. Yes, idiomatic Rust can cause a number of bounds checks to be elided when using iterators. Typically that's when you have all the data that you need to process up front. When you don't have that ability (e.g. data is being fed in piecemeal and you're copying into a temporary buffer), the bounds checking elision is something you need to do yourself.
stouset 3y ago
I think the general takeaway is that for most use cases, Rust’s additional safety comes at barely any cost. In some cases the safety gains performance (in not checking thing that the compiler can assert statically), in some cases there’s a hit, but it usually ends up somewhere in the middle. The takeaway is that Rust’s safety and maintainability generally don’t impose corresponding penalties. And in the cases they do and it’s critical, you can always punt to unsafe.
- ssokolow 3y ago
  *nod* Give https://blog.readyset.io/bounds-checks/ https://blog.readyset.io/bounds-checks/ a read. They tried doing a comparison between ReadySet compiled normally and ReadySet with bounds checking removed so thoroughly that they needed to use a patched toolchain to achieve it and found the difference to be within the noise threshold. Their conclusion was: > At the end of the day, it seems like at least for this kind of large-scale, complex application, the cost of pervasive runtime bounds checking is negligible. It’s tough to say precisely why this is, but my intuition is that CPU branch prediction is simply good enough in practice that the cost of the extra couple of instructions and a branch effectively ends up being zero - and compilers like LLVM are good enough at local optimizations to optimize most bounds checks away entirely. Not to mention, it’s likely that quite a few (if not the majority) of the bounds checks we removed are actually necessary, in that they’re validating some kind of user input or other edge conditions where we want to panic on an out of bounds access.
  - vlovich123 3y ago
    As I noted originally, that’s not the reason. It’s because no one is trying to write code that can do hundreds of millions of high level operations per second. There’s so much “inefficiency” in the software stack (or a problem domain where more work is being done in the hot path) that bounds checking is in the noise typically. Readyset’s experiment is flawed because any existence of bounds check on some critical hot path would have already been noticed and fixed as low hanging fruit. So the remaining bounds check is off the hot path where it doesn’t matter. But in the hot path where you should be bounded by HW limits, it can be a significant slowdown. Prediction can help but it’s not free. So for most people, they only need to care about bounds checking when they’re doing something in the hot path and even then only when their hot path is running into HW limits. If their hot path is some complicated CPU computation, bounds checking should be but a blip unless you do something stupid like check the bounds too frequently. So the general advice to not worry too much about bounds checks when writing Rust is directionally correct for the vast majority of people, but recognize it’s incorrect in places and it’s hard to notice because it’s such a small thing hidden in code gen without an easy flag to test the impact.
norir 3y ago
The paradox of bounds checking is that it both rarely matters and always matters.
hedora 3y ago
I suspect that information about memory aliasing from the borrow checker (that things are not aliased) allows the compiler to add in many optimizations that C/C++ can't perform, and the benefit from that is similar to the cost of bounds checking. The memcpy vs. memmove thing comes to mind. So does stuff like iterating over the indices of a pointer to one std::vector while modifying another pointer to a std::vector. If you know you have exclusive access to the vector you are reading, then you know the size isn't changing, so you can just iterate from 0 to size and elide any bounds checking. With C++, if the pointers to the vectors could alias, then it will have to bounds check at runtime. Templates and Link time optimization helps with this sort of thing, but I'm guessing Rust still has an advantage.

TylerE 3y ago

Throughputs on modern hardware are insane. Latency/time-til-first-byte can often still be pretty sucky.

vlovich123 3y ago
Care to give an example? A disk read is comparatively slow to compute/memory but it’s still in the tens or hundreds of microseconds if I recall correctly and memory is hundreds of nanoseconds. I am convinced latency issues are a remnant of how we build abstractions into software stacks and not a fault of the hardware.
- TylerE 3y ago
  Now have 10 or 15 apps hammering away at the same disk at the same time and see how it behaves. Even SSDs slow down as read sizes go down.
- TillE 3y ago
  Cache misses will destroy the performance of any CPU-intensive program. You really really want to do mostly sequential reads of RAM if you're processing a lot of data.

epage 3y ago

Unless this was updated after your post, I think it was covered in the following line: > It's certainly possible to build things where bounds checking makes a difference. This just isn't that. I will add that idiomatic Rust causes a good number of bounds checks to be optimized away. Its impressive the difference this can make. I maintain a parser combinator library that started as a fork. With some simple changes, I've seen massive performance improvements and my best guess is that those changes allowed bounds checks to be removed.

norman784 3y ago
Can you share what kind of changes, in your case, improved the performance?
- wredue 3y ago
  Avoid directly indexing if possible. Use the collection functions and slices instead.
  - epage 3y ago
    I'd add careful use of inlining so the compiler can see both the bounds check and the index to know the bounds check can be removed.
  - deleted 3y ago
    [deleted]
  - deleted 3y ago
    [deleted]
vlovich123 3y ago
Possible I missed it when I read it. Would need the author to indicate whether or not it was edited. Yes, idiomatic Rust can cause a number of bounds checks to be elided when using iterators. Typically that's when you have all the data that you need to process up front. When you don't have that ability (e.g. data is being fed in piecemeal and you're copying into a temporary buffer), the bounds checking elision is something you need to do yourself.

stouset 3y ago

I think the general takeaway is that for most use cases, Rust’s additional safety comes at barely any cost. In some cases the safety gains performance (in not checking thing that the compiler can assert statically), in some cases there’s a hit, but it usually ends up somewhere in the middle. The takeaway is that Rust’s safety and maintainability generally don’t impose corresponding penalties. And in the cases they do and it’s critical, you can always punt to unsafe.

ssokolow 3y ago
*nod* Give https://blog.readyset.io/bounds-checks/ https://blog.readyset.io/bounds-checks/ a read. They tried doing a comparison between ReadySet compiled normally and ReadySet with bounds checking removed so thoroughly that they needed to use a patched toolchain to achieve it and found the difference to be within the noise threshold. Their conclusion was: > At the end of the day, it seems like at least for this kind of large-scale, complex application, the cost of pervasive runtime bounds checking is negligible. It’s tough to say precisely why this is, but my intuition is that CPU branch prediction is simply good enough in practice that the cost of the extra couple of instructions and a branch effectively ends up being zero - and compilers like LLVM are good enough at local optimizations to optimize most bounds checks away entirely. Not to mention, it’s likely that quite a few (if not the majority) of the bounds checks we removed are actually necessary, in that they’re validating some kind of user input or other edge conditions where we want to panic on an out of bounds access.
- vlovich123 3y ago
  As I noted originally, that’s not the reason. It’s because no one is trying to write code that can do hundreds of millions of high level operations per second. There’s so much “inefficiency” in the software stack (or a problem domain where more work is being done in the hot path) that bounds checking is in the noise typically. Readyset’s experiment is flawed because any existence of bounds check on some critical hot path would have already been noticed and fixed as low hanging fruit. So the remaining bounds check is off the hot path where it doesn’t matter. But in the hot path where you should be bounded by HW limits, it can be a significant slowdown. Prediction can help but it’s not free. So for most people, they only need to care about bounds checking when they’re doing something in the hot path and even then only when their hot path is running into HW limits. If their hot path is some complicated CPU computation, bounds checking should be but a blip unless you do something stupid like check the bounds too frequently. So the general advice to not worry too much about bounds checks when writing Rust is directionally correct for the vast majority of people, but recognize it’s incorrect in places and it’s hard to notice because it’s such a small thing hidden in code gen without an easy flag to test the impact.

norir 3y ago

The paradox of bounds checking is that it both rarely matters and always matters.

hedora 3y ago

I suspect that information about memory aliasing from the borrow checker (that things are not aliased) allows the compiler to add in many optimizations that C/C++ can't perform, and the benefit from that is similar to the cost of bounds checking. The memcpy vs. memmove thing comes to mind. So does stuff like iterating over the indices of a pointer to one std::vector while modifying another pointer to a std::vector. If you know you have exclusive access to the vector you are reading, then you know the size isn't changing, so you can just iterate from 0 to size and elide any bounds checking. With C++, if the pointers to the vectors could alias, then it will have to bounds check at runtime. Templates and Link time optimization helps with this sort of thing, but I'm guessing Rust still has an advantage.