3 ms·

It's hard to understand how such an error could occur. Due to the nature of the floating point format, it seems very difficult for a bug to transform 1.0 into

It's hard to understand how such an error could occur. Due to the nature of the floating point format, it seems very difficult for a bug to transform 1.0 into 0.999999. Floating point 1.0 has a sign bit of 0, an exponent of 0 (after bias), and a significand of 1.0. It's hard to imagine a sequence of transformations that could somehow result in getting 0.999999999 after conversion. I made a little "floating point binary explorer": http://ideone.com/61L4V6 http://ideone.com/61L4V6 (Apologies for the awful code quality; it's just a quick hack.) It prints each bit of a 32-bit float, which for 1.0 is "00111111 10000000 00000000 00000000". The first bit is the sign bit, the next 8 bits are the exponent, and the remaining 23 bits are the significand. So, sign=0, exponent=127, significand=0. The significand is actually 24-bit, not 23, and the leading "hidden bit" is always 1 (nitpick: unless the floating point value is 0.0 or -0.0). That means the significand in this case isn't actually zero, but "1 followed by 23 zeroes," so it's 2^24. The final value for a 32-bit float is: (significand / 2^24) * 2^(exponent - 127) That comes out to (2^24 / 2^24) * 2^(127 - 127), which is 1.0 * 2^0 which equals 1.0. For representing 2.0, everything is exactly the same, except the encoded exponent is 128 instead of 127. That comes out to (2^24 / 2^24) * 2^(128 - 127), or 1.0 * 2^1 which equals 2.0. Shader programs often use half precision floats (16-bit total, with 1 bit for sign, 5 bits for exponent, and 10 for significand) but the concepts should be exactly the same. 1.0 and 2.0 should always be 1.0 * 2^0 and 1.0 * 2^1 respectively, regardless of whether we're using half/single/double precision. I'm having trouble figuring out how your program wound up with 0.999999, even if there was a bug in the conversion. 32-bit floating point 0.999999 is "00111111 01111111 11111111 11101111" which is very different from the binary 1.0 float value: http://ideone.com/Qlqu4W http://ideone.com/Qlqu4W It has an exponent of 126 and almost the highest possible significand, which comes out to 1.9999 * 2^(126 - 127) = 1.9999 * 2^-1 = 0.9999. So in order for 1.0 to be erroneously transformed into 0.9999, the bug would need to invert all the bits of the significand, but subtract 1 from the exponent. Is there anything in particular that I should watch out for on the A7 GPU? I was just hoping to hear about any details you may have left out for simplicity, or any other factors that could've triggered the bug. > Even on desktop machines, I had similar problems with large sets of tests that would produce different results depending on which (modern desktop) processor they ran on. > Lesson: floating point is certainly dangerous for consistent representations that "matter". Regardless of a compiler decision or what the spec says. Are there floating point bugs on desktop processors? The only one I've heard of was the infamous Pentium FDIV bug http://en.wikipedia.org/wiki/Pentium_FDIV_bug http://en.wikipedia.org/wiki/Pentium_FDIV_bug and the only reason that wasn't hotfixed to match the spec was because the logic was hardwired into the chip rather than implemented in microcode, making a patch impossible. It seems like any deviation from the floating point standard in a modern desktop CPU would be noticed almost immediately, because there is a lot of code which relies on the exact IEEE floating point standard in order to function properly. There was recently a hash algorithm featured on HN which used floating point, and the only reason such an algorithm can be trusted is if every desktop processor properly adheres to the spec.

acqq 12y ago
I also suspect the 0.9999.. was an actual number in the commenter's inputs to the fragment shader. Unless he checked the binary representation of the numbers in question (in the input) he could have missed what's actually going on. Formatting and printing the number in base 10 hides the only real thing, which is the actual bits that represent the number.
- jacquesm 12y ago
  It's not rare for GPUs to be pushed so close to the boundaries of what the underlying hardware can do that you'll see little bits of noise like this. For 'consumer' stuff this usually does not matter, but I think it is one of the reasons (besides revenues) that Nvidia has a 'compute' line and a 'display' line. Even if they're based on the exact same tech it would not surprise me one bit if the safety margins were eroded considerably on the consumer stuff and if it wasn't tested as good before shipping (on the chip level).
  - sillysaurus3 12y ago
    ATI would probably be a better example. Nvidia has historically been more precise and has implemented specs pretty faithfully. ATI has of course become AMD, and the reputation of modern AMD graphics cards among developers seems to be pretty great. It would surprise me if sending the value 1.0 from a vertex program to fragment program ever resulted in anything except 1.0 on a desktop GPU. It's not unusual to truncate a float in a fragment program and the use the resulting integer as an index. I think Nvidia has a fairly rigorous battery of tests which they use to ensure compliance with specs. The tests check to make sure that values are being interpolated correctly, to check for off-by-one errors at the edges of triangles, and other such corner cases. The tests are automated, because once you implement a graphics spec correctly, you can generate an output image which should be the same every time it's generated. That way, if an error creeps into the driver, it should be caught pretty quickly, since it will cause a deviation in the rendered output examined by the test suite. I'm not sure how often a problem occurs in an Nvidia consumer card but not their 'compute' line. Nvidia tries to use the same driver codebase across as many of their products as possible. Here's an interesting thread where someone points out that certain OpenCL code was giving the correct result on x86 and Nvidia cards, but not AMD cards: http://devgurus.amd.com/thread/145582 http://devgurus.amd.com/thread/145582 In that thread, it's suggested that the reason Nvidia gave the correct result was because their CUDA compute specs had better error margins compared to OpenCL specs. So in that case, it would appear that Nvidia's compute line actually enhanced their accuracy in other contexts such as OpenCL. Rather than implement a different driver for OpenCL, CUDA, and consumer graphics, it seems that Nvidia uses the same driver to power all three, which seems to enhance accuracy rather than hinder it. While writing up this comment and researching this, I stumbled across this interesting paper from 2011 that details exactly what programmers can expect when programming to an Nvidia GPU (in this case using CUDA): https://developer.nvidia.com/sites/default/files/akamai/cuda/files/NVIDIA-CUDA-Floating-Point.pdf https://developer.nvidia.com/sites/default/files/akamai/cuda... ... It's a pretty fun read which helps to solidify the impression that Nvidia cares deeply about providing accurate floating point results. I also came across this article which was too awesome not to mention: http://randomascii.wordpress.com/2013/07/16/floating-point-determinism/ http://randomascii.wordpress.com/2013/07/16/floating-point-d... ... It's not really related to GPU floating point, but it's an excellent exploration of floating point determinism and the effect of various floating point settings. For mobile graphics stacks like those in cellphones / iPads etc, I share your skepticism. I'm not sure how rigorously tested mobile GPUs are, or how faithfully those stacks adhere to specs. I was hoping to get more info from the original commenter about how to reproduce the bug in the A7 to get a better idea of what to watch out for on mobile.

acqq 12y ago

I also suspect the 0.9999.. was an actual number in the commenter's inputs to the fragment shader. Unless he checked the binary representation of the numbers in question (in the input) he could have missed what's actually going on. Formatting and printing the number in base 10 hides the only real thing, which is the actual bits that represent the number.

jacquesm 12y ago
It's not rare for GPUs to be pushed so close to the boundaries of what the underlying hardware can do that you'll see little bits of noise like this. For 'consumer' stuff this usually does not matter, but I think it is one of the reasons (besides revenues) that Nvidia has a 'compute' line and a 'display' line. Even if they're based on the exact same tech it would not surprise me one bit if the safety margins were eroded considerably on the consumer stuff and if it wasn't tested as good before shipping (on the chip level).
- sillysaurus3 12y ago
  ATI would probably be a better example. Nvidia has historically been more precise and has implemented specs pretty faithfully. ATI has of course become AMD, and the reputation of modern AMD graphics cards among developers seems to be pretty great. It would surprise me if sending the value 1.0 from a vertex program to fragment program ever resulted in anything except 1.0 on a desktop GPU. It's not unusual to truncate a float in a fragment program and the use the resulting integer as an index. I think Nvidia has a fairly rigorous battery of tests which they use to ensure compliance with specs. The tests check to make sure that values are being interpolated correctly, to check for off-by-one errors at the edges of triangles, and other such corner cases. The tests are automated, because once you implement a graphics spec correctly, you can generate an output image which should be the same every time it's generated. That way, if an error creeps into the driver, it should be caught pretty quickly, since it will cause a deviation in the rendered output examined by the test suite. I'm not sure how often a problem occurs in an Nvidia consumer card but not their 'compute' line. Nvidia tries to use the same driver codebase across as many of their products as possible. Here's an interesting thread where someone points out that certain OpenCL code was giving the correct result on x86 and Nvidia cards, but not AMD cards: http://devgurus.amd.com/thread/145582 http://devgurus.amd.com/thread/145582 In that thread, it's suggested that the reason Nvidia gave the correct result was because their CUDA compute specs had better error margins compared to OpenCL specs. So in that case, it would appear that Nvidia's compute line actually enhanced their accuracy in other contexts such as OpenCL. Rather than implement a different driver for OpenCL, CUDA, and consumer graphics, it seems that Nvidia uses the same driver to power all three, which seems to enhance accuracy rather than hinder it. While writing up this comment and researching this, I stumbled across this interesting paper from 2011 that details exactly what programmers can expect when programming to an Nvidia GPU (in this case using CUDA): https://developer.nvidia.com/sites/default/files/akamai/cuda/files/NVIDIA-CUDA-Floating-Point.pdf https://developer.nvidia.com/sites/default/files/akamai/cuda... ... It's a pretty fun read which helps to solidify the impression that Nvidia cares deeply about providing accurate floating point results. I also came across this article which was too awesome not to mention: http://randomascii.wordpress.com/2013/07/16/floating-point-determinism/ http://randomascii.wordpress.com/2013/07/16/floating-point-d... ... It's not really related to GPU floating point, but it's an excellent exploration of floating point determinism and the effect of various floating point settings. For mobile graphics stacks like those in cellphones / iPads etc, I share your skepticism. I'm not sure how rigorously tested mobile GPUs are, or how faithfully those stacks adhere to specs. I was hoping to get more info from the original commenter about how to reproduce the bug in the A7 to get a better idea of what to watch out for on mobile.