5 ms·

This is .. amazing? Using what essentially should amount to a strong typedef .. causes code to be millions of times slower. This is worth bookmarking.

by hsn915 5y ago

readams 5y ago
I think it's more accurate to say that there's an optimization that applies only to a very specific situation that's millions of times faster than without the optimization. It's a bit strange to say that before the u8 specialization this was not a problem and it's suddenly a problem now.
- ixtli 5y ago
  Very well said. I actually came here to write a comment criticizing the post because the fact that this was “negative” has to do with the order in which the experiment took place. Flip it and you have a different sentiment. A similar point can be made but an explanation of “it’s very slow” presumes an assumption that every statement in rust will be maximally optimized via specialization which I consider to be an unreasonable expectation.
  - throwaway17_17 5y ago
    I’m not arguing against you, but why do you feel like expecting maximal optimization is unreasonable? Is it just when limited to specialization or do you think it applies more broadly in terms of general optimization?
- hsn915 5y ago
  I expect the special case handling for u8 to also apply to types that are just a strong typedef around u8.
  - bilkow 5y ago
    Rust has "typedefs" and those are called type aliases. The intention of the newtype pattern is to specifically create a new type without the behaviour of the wrapped type, so if there's a special case just for u8, I would expect it not to be on the newtype.
    - jcranberry 5y ago
      A "strong typedef" is not a typedef.
Dylan16807 5y ago
It's not really a matter of code going faster or slower, though. The real difference in speed is infinity/undefined. One version initializes the memory and the other one doesn't. There's a lot of subtle ways to cause that in different languages.
- tialaramex 5y ago
  Technically it postpones initializing the memory, and then it exits before that is needed. If you actually use the memory, both cases work out the same. There are too many languages where you can easily make the mistake of reading uninitialized memory, and that can't happen here, if you ask to sum() the vector you'll get a zero answer in both cases... and both programs will be similarly slow.
  - tialaramex 5y ago
    (Note that you can't actually "sum" the code sample's wrapper type because it lacks the appropriate operations. I spent a while trying to actually make a working example where I trusted I was measuring something "real" and not artefacts and I eventually gave up. Definitely begin with an actual problem, so that you know what you're trying to do and can measure whether you're doing it or else you may just be wanking)
tialaramex 5y ago
This is mostly in fact an example of a big problem with benchmarks. What's measured here is the difference between never using an object (and so the actual work to prepare it can be elided, albeit by the OS not the programming language) and doing all the work up front in the expectation you'll use it, but then not using it. u8 turns into "Hey, Linux kernel, zero these pages if I ever read them" (and then never reading them) whereas the opaque type turns into memset() zeroing all the pages. Not doing any work is in fact a million times faster but your real program would have needed to do work, otherwise why bother having the vector? Whereupon the benefit disappears. Where possible design your benchmarks to really do the thing you think you're measuring. If what you're measuring is nothing then be sceptical about supposed "performance" measured for that, since it's nothing, you're probably exploring the same space as the people who wanted to find out how much the human soul weighs (trick question, there is no such thing, but they put a lot of effort into trying to measure it anyway).
- qw3rty01 5y ago
  This definitely brings it closer, but on my machine, touching the entire array still ends up being 2 seconds slower (5.4s for the calloc side, 7.8s for the clone side)
  - irishsultan 5y ago
    It's not using clone but memset. Of course memsetting memory to 0 when the operating system already set it to 0 is still silly and something that could be further optimized, but it's not cloning things.
    - qw3rty01 5y ago
      It does clone: https://stdrs.dev/nightly/x86_64-pc-windows-gnu/src/alloc/vec/mod.rs.html#2205-2212 https://stdrs.dev/nightly/x86_64-pc-windows-gnu/src/alloc/ve... it might be optimized to a memset, but it's the clone codepath, as opposed to the u8 specialization codepath as discussed in the article But the original commenter is talking about how the benchmark isn't useful because it doesn't touch memory, but you get similar results even if you do touch memory.
jokethrowaway 5y ago
That first example is misleading. It's doing different things: In the first case it's not doing anything and it will request memory dynamically. In the second case it's zeroing 16gb of RAM. The explanation linked at the end explains it better.
- davrosthedalek 5y ago
  Yeah, but taking >5s to zero 16GB is still atrociously slow.
PragmaticPulp 5y ago
It’s misleading. One version is reserving the memory and will zero it upon use. The other version allocates the memory and zeros every byte of the memory up front by actually writing zeroes to it. It’s basically competing abstractions. Letting the kernel reserve pages and only zeroing them when used will hide the cost and amortize it across access. It’s worth knowing about but it’s also an artificial problem revealed by benchmarks.

This is .. amazing? Using what essentially should amount to a strong typedef .. causes code to be millions of times slower. This is worth bookmarking.

by hsn915 5y ago

This is .. amazing? Using what essentially should amount to a strong typedef .. causes code to be millions of times slower. This is worth bookmarking.

readams 5y ago

I think it's more accurate to say that there's an optimization that applies only to a very specific situation that's millions of times faster than without the optimization. It's a bit strange to say that before the u8 specialization this was not a problem and it's suddenly a problem now.

ixtli 5y ago
Very well said. I actually came here to write a comment criticizing the post because the fact that this was “negative” has to do with the order in which the experiment took place. Flip it and you have a different sentiment. A similar point can be made but an explanation of “it’s very slow” presumes an assumption that every statement in rust will be maximally optimized via specialization which I consider to be an unreasonable expectation.
- throwaway17_17 5y ago
  I’m not arguing against you, but why do you feel like expecting maximal optimization is unreasonable? Is it just when limited to specialization or do you think it applies more broadly in terms of general optimization?
hsn915 5y ago
I expect the special case handling for u8 to also apply to types that are just a strong typedef around u8.
- bilkow 5y ago
  Rust has "typedefs" and those are called type aliases. The intention of the newtype pattern is to specifically create a new type without the behaviour of the wrapped type, so if there's a special case just for u8, I would expect it not to be on the newtype.
  - jcranberry 5y ago
    A "strong typedef" is not a typedef.

Dylan16807 5y ago

It's not really a matter of code going faster or slower, though. The real difference in speed is infinity/undefined. One version initializes the memory and the other one doesn't. There's a lot of subtle ways to cause that in different languages.

tialaramex 5y ago
Technically it postpones initializing the memory, and then it exits before that is needed. If you actually use the memory, both cases work out the same. There are too many languages where you can easily make the mistake of reading uninitialized memory, and that can't happen here, if you ask to sum() the vector you'll get a zero answer in both cases... and both programs will be similarly slow.
- tialaramex 5y ago
  (Note that you can't actually "sum" the code sample's wrapper type because it lacks the appropriate operations. I spent a while trying to actually make a working example where I trusted I was measuring something "real" and not artefacts and I eventually gave up. Definitely begin with an actual problem, so that you know what you're trying to do and can measure whether you're doing it or else you may just be wanking)

tialaramex 5y ago

This is mostly in fact an example of a big problem with benchmarks. What's measured here is the difference between never using an object (and so the actual work to prepare it can be elided, albeit by the OS not the programming language) and doing all the work up front in the expectation you'll use it, but then not using it. u8 turns into "Hey, Linux kernel, zero these pages if I ever read them" (and then never reading them) whereas the opaque type turns into memset() zeroing all the pages. Not doing any work is in fact a million times faster but your real program would have needed to do work, otherwise why bother having the vector? Whereupon the benefit disappears. Where possible design your benchmarks to really do the thing you think you're measuring. If what you're measuring is nothing then be sceptical about supposed "performance" measured for that, since it's nothing, you're probably exploring the same space as the people who wanted to find out how much the human soul weighs (trick question, there is no such thing, but they put a lot of effort into trying to measure it anyway).

qw3rty01 5y ago
This definitely brings it closer, but on my machine, touching the entire array still ends up being 2 seconds slower (5.4s for the calloc side, 7.8s for the clone side)
- irishsultan 5y ago
  It's not using clone but memset. Of course memsetting memory to 0 when the operating system already set it to 0 is still silly and something that could be further optimized, but it's not cloning things.
  - qw3rty01 5y ago
    It does clone: https://stdrs.dev/nightly/x86_64-pc-windows-gnu/src/alloc/vec/mod.rs.html#2205-2212 https://stdrs.dev/nightly/x86_64-pc-windows-gnu/src/alloc/ve... it might be optimized to a memset, but it's the clone codepath, as opposed to the u8 specialization codepath as discussed in the article But the original commenter is talking about how the benchmark isn't useful because it doesn't touch memory, but you get similar results even if you do touch memory.

jokethrowaway 5y ago

That first example is misleading. It's doing different things: In the first case it's not doing anything and it will request memory dynamically. In the second case it's zeroing 16gb of RAM. The explanation linked at the end explains it better.

davrosthedalek 5y ago
Yeah, but taking >5s to zero 16GB is still atrociously slow.

PragmaticPulp 5y ago

It’s misleading. One version is reserving the memory and will zero it upon use. The other version allocates the memory and zeros every byte of the memory up front by actually writing zeroes to it. It’s basically competing abstractions. Letting the kernel reserve pages and only zeroing them when used will hide the cost and amortize it across access. It’s worth knowing about but it’s also an artificial problem revealed by benchmarks.