4 ms·

Here’s Intel versus AMD relative error of RCPPS instruction: http://const.me/tmp/vrcpps-errors-chart.png http://const.me/tmp/vrcpps-errors-chart.png AMD is Ryze

Here’s Intel versus AMD relative error of RCPPS instruction: http://const.me/tmp/vrcpps-errors-chart.png http://const.me/tmp/vrcpps-errors-chart.png AMD is Ryzen 5 3600, Intel is Core i3 6157U. Over the complete range of floats, AMD is more precise on average, 0.000078 versus 0.000095 relative error. However, Intel has 0.000300 maximum relative error, AMD 0.000315. Both are well within the spec. The documentation says “maximum relative error for this approximation is less than 1.5*2^-12”, in human language that would be 3.6621E-4. Source code that compares them by creating 16GB binary files with the complete range of floats: https://gist.github.com/Const-me/a6d36f70a3a77de00c61cf4f6c17c7ac https://gist.github.com/Const-me/a6d36f70a3a77de00c61cf4f6c1...

dvt 6y ago
This is great (I was actually just toying with the _mm_rcp_ss intrinsic)! It would also indicate that RCPSS is not the culprit.
- Const-me 6y ago
  > It would also indicate that RCPSS is not the culprit. I’m not sure about that. I think it might be. Despite both are within specs, the results are numerically different. Modern DirectXMath doesn‘t use these approximated instruction for inverting matrices, it’s open source, there: https://github.com/microsoft/DirectXMath/blob/83634c742a85d1027765af53fbe79506fd72e0c3/Inc/DirectXMathMatrix.inl#L983 https://github.com/microsoft/DirectXMath/blob/83634c742a85d1...
  - dvt 6y ago
    > Despite both are within specs, the results are numerically different. True, but it's quite common that floating point ops throwaway bits (i.e. beyond the epsilon) will be numerically different between vendors. > Modern DirectXMath doesn‘t use these approximated instruction for inverting matrices, it’s open source, there: https://github.com/microsoft/DirectXMath/blob/83634c742a85d1.. https://github.com/microsoft/DirectXMath/blob/83634c742a85d1.... Good find, neither does Wine, actually: https://doxygen.reactos.org/dc/dd8/d3dx9math_8h.html#a3870c69a57318daf46180e01c9c20594 https://doxygen.reactos.org/dc/dd8/d3dx9math_8h.html#a3870c6... Would be interesting to see if this bug also happens on Linux.
    - Const-me 6y ago
      BTW, there’s much less reason to use these approximations on modern hardware. Divps instruction (32-bit float divide, the precise version) was relatively slow back then, e.g. on AMD K8 it had 18-30 cycles both latency and throughput, 21 cycles on AMD Jaguar, Core 2 Duo did in 6-18 cycles. Fortunately, they fixed the CPUs. Skylake has 11 cycles latency of that instruction, Ryzen 10 cycles. It’s similar for square root. Modern CPUs compute non-approximated 32-bit sqrt(x) in 9-12 cycles (Ryzen being the fastest at 9-10 but Skylake ain’t bad either at 12 cycles), old CPUs were spending like 20-40 cycles on that.
    - creato 6y ago
      > True, but it's quite common that floating point ops throwaway bits (i.e. beyond the epsilon) will be numerically different between vendors. This is a common sentiment, and it is perhaps a helpful way to look at things if you don't want to dig into the details (even if it's not quite right). But it's worth understanding the magnitude of errors. rcpps will be 3-4 orders of magnitude "more wrong" compared to the typical operation (if you view the "epsilon" of most operations to be the error after rounding). Or said another way: it would take the cumulative error from many thousands of adds and multiplies to produce the same error as one rcpps operation.
    - Const-me 6y ago
      > it's quite common that floating point ops throwaway bits (i.e. beyond the epsilon) will be numerically different between vendors. I think these implementation-specific shenanigans only applied to legacy x87 instructions which used 80-bit registers. SSE and AVX instructions operate on 32 or 64-bit floats. The math there, including rounding behavior, is well specified in these IEEE standards. I’ve made a small test app: https://github.com/Const-me/SimdPrecisionTest/blob/master/main.cpp https://github.com/Const-me/SimdPrecisionTest/blob/master/ma... It abuses AES instructions to generate long pseudo-random sequence of bits, then re-interprets these bits as floats, does some FP math on these numbers, and saves the output. I’ve tested addition, multiplication, FMA, and float to integer conversions. All 4 output files, 1 GB / each, are bitwise equal between AMD desktop and Intel laptop.
      - 6y ago
  - deleted 6y ago
    [deleted]
creato 6y ago
Funny, I think this graph is from a thread where I got roasted for arguing that differences in CPU implementations meant that Intel (or anyone else really) would need to be careful about shipping SIMD software on processors they don't test.
- Dylan16807 6y ago
  Rightfully so. Because they were replacing the SIMD code with non-SIMD code that was similarly untested, with the side effect of crippling performance on their competitor's chips. That's a really bad kind of "being careful".
  - creato 6y ago
    I'm not aware of instructions that have this implementation defined behavior risk in non-SIMD code, and you can be sure that Intel has tested this. If Microsoft had shipped a non-SIMD version of D3DXMatrixInverse, this bug likely could not have existed (or it would have been caught on the developer's machines). In this case (games), they probably made the right tradeoff despite the bug. But in general? I don't want to rehash the argument, I'm just really, really glad I'm not the one flipping that switch (or subject to the hordes of HNers who think I'm a monster for not flipping it).
- Const-me 6y ago
  Correct, I've scavenged the comment, and the data+code, from an old thread here on HN.
acqq 6y ago
Related, LLVM developers evaluated the use of these instructions too: https://bugs.llvm.org/show_bug.cgi?id=21385 https://bugs.llvm.org/show_bug.cgi?id=21385 On your graph, what’s red and what’s blue and what is on x axis?
- Const-me 6y ago
  AMD is red. X axis is source value, I've extracted a small subset of the data to graph in Excel, there: https://gist.github.com/Const-me/a6d36f70a3a77de00c61cf4f6c17c7ac#file-vrcpps-errors-cpp-L105 https://gist.github.com/Const-me/a6d36f70a3a77de00c61cf4f6c1... Also, I've just recalled Y is absolute error not relative; that's why it follows shape of 1/x.
  - acqq 6y ago
    > I've extracted a small subset of the data to graph in Excel Thanks. If you still have the data (or if you can regenerate it) it would actually be possible to make a few small graphs that would cover the whole set: Basically, the idea is to have x points as much as there are different exponents, which in 32-bit floats is at most 256. Then on the y axis one draws the number of bits of maximum distance between the really correct value of the mantissa and the calculated mantissa in the whole interval of one exponent. Those would allow comparing the implementations of Intel and AMD separately, and these graphs are what I'd be very interested to see. So the idea is to find the maximums in an interval, and there are then a limited number of the intervals. Only if such graphs match between AMD and Intel it would be interesting to compare inside of the intervals, but differences on that level would be these that I'd expect would be the obvious ones making the most problems, where the results like in the article (full black instead of shadow) wouldn't be surprising. For that don't have to juggle with huge files, it's only 256 values per each CPU in that pass that are to be compared.
    - Const-me 6y ago
      Can re-generate the data files, it’s couple pages of code to copy-paste from my gist and compile, but I’m not sure how to implement what you wrote. The results are not guaranteed to have same exponent, e.g. 1 / 0.499999 can be either 1.999999 or 2.0000001, both are correct within the precision, but have different exponents.

dvt 6y ago

This is great (I was actually just toying with the _mm_rcp_ss intrinsic)! It would also indicate that RCPSS is not the culprit.

Const-me 6y ago
> It would also indicate that RCPSS is not the culprit. I’m not sure about that. I think it might be. Despite both are within specs, the results are numerically different. Modern DirectXMath doesn‘t use these approximated instruction for inverting matrices, it’s open source, there: https://github.com/microsoft/DirectXMath/blob/83634c742a85d1027765af53fbe79506fd72e0c3/Inc/DirectXMathMatrix.inl#L983 https://github.com/microsoft/DirectXMath/blob/83634c742a85d1...
- dvt 6y ago
  > Despite both are within specs, the results are numerically different. True, but it's quite common that floating point ops throwaway bits (i.e. beyond the epsilon) will be numerically different between vendors. > Modern DirectXMath doesn‘t use these approximated instruction for inverting matrices, it’s open source, there: https://github.com/microsoft/DirectXMath/blob/83634c742a85d1.. https://github.com/microsoft/DirectXMath/blob/83634c742a85d1.... Good find, neither does Wine, actually: https://doxygen.reactos.org/dc/dd8/d3dx9math_8h.html#a3870c69a57318daf46180e01c9c20594 https://doxygen.reactos.org/dc/dd8/d3dx9math_8h.html#a3870c6... Would be interesting to see if this bug also happens on Linux.
  - Const-me 6y ago
    BTW, there’s much less reason to use these approximations on modern hardware. Divps instruction (32-bit float divide, the precise version) was relatively slow back then, e.g. on AMD K8 it had 18-30 cycles both latency and throughput, 21 cycles on AMD Jaguar, Core 2 Duo did in 6-18 cycles. Fortunately, they fixed the CPUs. Skylake has 11 cycles latency of that instruction, Ryzen 10 cycles. It’s similar for square root. Modern CPUs compute non-approximated 32-bit sqrt(x) in 9-12 cycles (Ryzen being the fastest at 9-10 but Skylake ain’t bad either at 12 cycles), old CPUs were spending like 20-40 cycles on that.
  - creato 6y ago
    > True, but it's quite common that floating point ops throwaway bits (i.e. beyond the epsilon) will be numerically different between vendors. This is a common sentiment, and it is perhaps a helpful way to look at things if you don't want to dig into the details (even if it's not quite right). But it's worth understanding the magnitude of errors. rcpps will be 3-4 orders of magnitude "more wrong" compared to the typical operation (if you view the "epsilon" of most operations to be the error after rounding). Or said another way: it would take the cumulative error from many thousands of adds and multiplies to produce the same error as one rcpps operation.
  - Const-me 6y ago
    > it's quite common that floating point ops throwaway bits (i.e. beyond the epsilon) will be numerically different between vendors. I think these implementation-specific shenanigans only applied to legacy x87 instructions which used 80-bit registers. SSE and AVX instructions operate on 32 or 64-bit floats. The math there, including rounding behavior, is well specified in these IEEE standards. I’ve made a small test app: https://github.com/Const-me/SimdPrecisionTest/blob/master/main.cpp https://github.com/Const-me/SimdPrecisionTest/blob/master/ma... It abuses AES instructions to generate long pseudo-random sequence of bits, then re-interprets these bits as floats, does some FP math on these numbers, and saves the output. I’ve tested addition, multiplication, FMA, and float to integer conversions. All 4 output files, 1 GB / each, are bitwise equal between AMD desktop and Intel laptop.
    - 6y ago
- deleted 6y ago
  [deleted]

creato 6y ago

Funny, I think this graph is from a thread where I got roasted for arguing that differences in CPU implementations meant that Intel (or anyone else really) would need to be careful about shipping SIMD software on processors they don't test.

Dylan16807 6y ago
Rightfully so. Because they were replacing the SIMD code with non-SIMD code that was similarly untested, with the side effect of crippling performance on their competitor's chips. That's a really bad kind of "being careful".
- creato 6y ago
  I'm not aware of instructions that have this implementation defined behavior risk in non-SIMD code, and you can be sure that Intel has tested this. If Microsoft had shipped a non-SIMD version of D3DXMatrixInverse, this bug likely could not have existed (or it would have been caught on the developer's machines). In this case (games), they probably made the right tradeoff despite the bug. But in general? I don't want to rehash the argument, I'm just really, really glad I'm not the one flipping that switch (or subject to the hordes of HNers who think I'm a monster for not flipping it).
Const-me 6y ago
Correct, I've scavenged the comment, and the data+code, from an old thread here on HN.

acqq 6y ago

Related, LLVM developers evaluated the use of these instructions too: https://bugs.llvm.org/show_bug.cgi?id=21385 https://bugs.llvm.org/show_bug.cgi?id=21385 On your graph, what’s red and what’s blue and what is on x axis?

Const-me 6y ago
AMD is red. X axis is source value, I've extracted a small subset of the data to graph in Excel, there: https://gist.github.com/Const-me/a6d36f70a3a77de00c61cf4f6c17c7ac#file-vrcpps-errors-cpp-L105 https://gist.github.com/Const-me/a6d36f70a3a77de00c61cf4f6c1... Also, I've just recalled Y is absolute error not relative; that's why it follows shape of 1/x.
- acqq 6y ago
  > I've extracted a small subset of the data to graph in Excel Thanks. If you still have the data (or if you can regenerate it) it would actually be possible to make a few small graphs that would cover the whole set: Basically, the idea is to have x points as much as there are different exponents, which in 32-bit floats is at most 256. Then on the y axis one draws the number of bits of maximum distance between the really correct value of the mantissa and the calculated mantissa in the whole interval of one exponent. Those would allow comparing the implementations of Intel and AMD separately, and these graphs are what I'd be very interested to see. So the idea is to find the maximums in an interval, and there are then a limited number of the intervals. Only if such graphs match between AMD and Intel it would be interesting to compare inside of the intervals, but differences on that level would be these that I'd expect would be the obvious ones making the most problems, where the results like in the article (full black instead of shadow) wouldn't be surprising. For that don't have to juggle with huge files, it's only 256 values per each CPU in that pass that are to be compared.
  - Const-me 6y ago
    Can re-generate the data files, it’s couple pages of code to copy-paste from my gist and compile, but I’m not sure how to implement what you wrote. The results are not guaranteed to have same exponent, e.g. 1 / 0.499999 can be either 1.999999 or 2.0000001, both are correct within the precision, but have different exponents.