3 ms·

Why use floats when you can basically have doubles for the same cost? Floats tend to cause nasty surprises when you deal with real world units.

LPDWORD 7y ago
> Why use floats when you can basically have doubles for the same cost? Floats costing the same as doubles is a myth stemming from x87 arithmetic, which is obsolete. On an x64 CPU running 64-bit code, your compiler can often pack four floats into one SSE register. Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87.
- vardump 7y ago
  The difference is meaningless in scalar code. Not everything is or can be vectorized. Pretty much no difference between one double and one float in a SSE XMM register. > Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. > Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87. So use floats when you have a lot of data.
  - LPDWORD 7y ago
    If you already knew there was a cost difference, why didn't you just say so? > The difference is meaningless in scalar code. Not everything is or can be vectorized. Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. > Pretty much no difference between one double and one float in a SSE XMM register. Even then, some arithmetic operations have higher throughput with floats than with doubles. > I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. > So use floats when you have a lot of data. That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference.
    - vardump 7y ago
      > Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. They can, but can you honestly call it a common case for scalar code? > Even then, some arithmetic operations have higher throughput with floats than with doubles. By far the most (90-99%) of FP computation is additions and multiplications (or fused multiply adds). For scalar case (1 double or float in SSE register), they take precisely as long on modern x86 hardware. Sure, float div executes in 11 instead of 13-14 clocks for doubles and I'm sure transcendentals are even worse, but they're rarely needed. Even then, if the dependency chain allows, the cost is often OoO scheduled away in integer dominated code. > Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. Except that SSE instructions pretty much are micro-ops as-is. Despite similarly sounding term, microcode has nothing to do with micro-ops. > That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference. Well, I've written a lot of SSE, AVX etc. SIMD code. There sure is a big difference when you're processing large amounts of data. But... I've seen floats introducing silly precision related bugs [0] and a ton of useless float -> double -> float conversion chains. Most of the time most programmers should default to double. [0]: Example: https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ https://randomascii.wordpress.com/2012/02/13/dont-store-that...
GnarfGnarf 7y ago
Good point. The class definitions can easily be changed from float to double in just a few lines. I used to use a lot of doubles. The I wrote some DirectX and GPUs all use float. I figured if floats are good enough for 3D, they're good enough for me :o)
- jcranmer 7y ago
  With floating-point code, you will slowly lose precision in your numbers. (How quickly is a matter of what calculations you're doing, if you're doing them in an error-tolerant manner, and how bad the initial error is in the first place). Single-precision floating-point starts you off with about 6-7 decimal digits of precision, and the intermediate rounding will mean that you're constantly introducing error in the 7th digit. Computer graphics are pretty tolerant of error--you only need about a 1-2 digits of precision generally--and achieving higher FLOPS is more useful than higher precision. Engineering tolerances are usually going to be tighter, perhaps 4-5 digits for particularly tight tolerances. Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences.
  - vardump 7y ago
    Yup. Although I'd like to add a typical issue when there's a small change in a large quantity. Say time. If you measure time in microseconds, and use floats, your precision will be halved to 2 us in just 33.5 seconds. In 24 hours, the precision has detoriated to over 2 milliseconds. One famous example is the deadly Patriot missile bug: https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf > Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences. Some other ones are iteration and simply long computation chains introducing systematic error.

LPDWORD 7y ago

> Why use floats when you can basically have doubles for the same cost? Floats costing the same as doubles is a myth stemming from x87 arithmetic, which is obsolete. On an x64 CPU running 64-bit code, your compiler can often pack four floats into one SSE register. Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87.

vardump 7y ago
The difference is meaningless in scalar code. Not everything is or can be vectorized. Pretty much no difference between one double and one float in a SSE XMM register. > Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. > Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87. So use floats when you have a lot of data.
- LPDWORD 7y ago
  If you already knew there was a cost difference, why didn't you just say so? > The difference is meaningless in scalar code. Not everything is or can be vectorized. Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. > Pretty much no difference between one double and one float in a SSE XMM register. Even then, some arithmetic operations have higher throughput with floats than with doubles. > I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. > So use floats when you have a lot of data. That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference.
  - vardump 7y ago
    > Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. They can, but can you honestly call it a common case for scalar code? > Even then, some arithmetic operations have higher throughput with floats than with doubles. By far the most (90-99%) of FP computation is additions and multiplications (or fused multiply adds). For scalar case (1 double or float in SSE register), they take precisely as long on modern x86 hardware. Sure, float div executes in 11 instead of 13-14 clocks for doubles and I'm sure transcendentals are even worse, but they're rarely needed. Even then, if the dependency chain allows, the cost is often OoO scheduled away in integer dominated code. > Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. Except that SSE instructions pretty much are micro-ops as-is. Despite similarly sounding term, microcode has nothing to do with micro-ops. > That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference. Well, I've written a lot of SSE, AVX etc. SIMD code. There sure is a big difference when you're processing large amounts of data. But... I've seen floats introducing silly precision related bugs [0] and a ton of useless float -> double -> float conversion chains. Most of the time most programmers should default to double. [0]: Example: https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ https://randomascii.wordpress.com/2012/02/13/dont-store-that...

GnarfGnarf 7y ago

Good point. The class definitions can easily be changed from float to double in just a few lines. I used to use a lot of doubles. The I wrote some DirectX and GPUs all use float. I figured if floats are good enough for 3D, they're good enough for me :o)

jcranmer 7y ago
With floating-point code, you will slowly lose precision in your numbers. (How quickly is a matter of what calculations you're doing, if you're doing them in an error-tolerant manner, and how bad the initial error is in the first place). Single-precision floating-point starts you off with about 6-7 decimal digits of precision, and the intermediate rounding will mean that you're constantly introducing error in the 7th digit. Computer graphics are pretty tolerant of error--you only need about a 1-2 digits of precision generally--and achieving higher FLOPS is more useful than higher precision. Engineering tolerances are usually going to be tighter, perhaps 4-5 digits for particularly tight tolerances. Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences.
- vardump 7y ago
  Yup. Although I'd like to add a typical issue when there's a small change in a large quantity. Say time. If you measure time in microseconds, and use floats, your precision will be halved to 2 us in just 33.5 seconds. In 24 hours, the precision has detoriated to over 2 milliseconds. One famous example is the deadly Patriot missile bug: https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf > Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences. Some other ones are iteration and simply long computation chains introducing systematic error.