4 ms·

I've developed a set of "strongly typed" classes in C++ that have saved me a ton of grief. Here are some examples. All classes are fundamentally floats. Errors

I've developed a set of "strongly typed" classes in C++ that have saved me a ton of grief. Here are some examples. All classes are fundamentally floats. Errors are caught at compilation. cInches i1, i2, i3; cCm cm1, cm2; i1 = 1.f; // error i1 = cInches(1.f); // OK i3 = i1 + i2; // OK cm1 = i1; // error i1 = i2 * 2.f; // OK float f1 = i1 / i2; // OK i1 = i2 * i3; // error! square inches are not inches int add(cInches in1, cInches in2); add(i1, i2); // OK add(cm1, cm2); // error 'Typedef's just don't cut it.

vardump 7y ago
Why use floats when you can basically have doubles for the same cost? Floats tend to cause nasty surprises when you deal with real world units.
- LPDWORD 7y ago
  > Why use floats when you can basically have doubles for the same cost? Floats costing the same as doubles is a myth stemming from x87 arithmetic, which is obsolete. On an x64 CPU running 64-bit code, your compiler can often pack four floats into one SSE register. Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87.
  - vardump 7y ago
    The difference is meaningless in scalar code. Not everything is or can be vectorized. Pretty much no difference between one double and one float in a SSE XMM register. > Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. > Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87. So use floats when you have a lot of data.
    - LPDWORD 7y ago
      If you already knew there was a cost difference, why didn't you just say so? > The difference is meaningless in scalar code. Not everything is or can be vectorized. Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. > Pretty much no difference between one double and one float in a SSE XMM register. Even then, some arithmetic operations have higher throughput with floats than with doubles. > I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. > So use floats when you have a lot of data. That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference.
      - vardump 7y ago
        > Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. They can, but can you honestly call it a common case for scalar code? > Even then, some arithmetic operations have higher throughput with floats than with doubles. By far the most (90-99%) of FP computation is additions and multiplications (or fused multiply adds). For scalar case (1 double or float in SSE register), they take precisely as long on modern x86 hardware. Sure, float div executes in 11 instead of 13-14 clocks for doubles and I'm sure transcendentals are even worse, but they're rarely needed. Even then, if the dependency chain allows, the cost is often OoO scheduled away in integer dominated code. > Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. Except that SSE instructions pretty much are micro-ops as-is. Despite similarly sounding term, microcode has nothing to do with micro-ops. > That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference. Well, I've written a lot of SSE, AVX etc. SIMD code. There sure is a big difference when you're processing large amounts of data. But... I've seen floats introducing silly precision related bugs [0] and a ton of useless float -> double -> float conversion chains. Most of the time most programmers should default to double. [0]: Example: https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ https://randomascii.wordpress.com/2012/02/13/dont-store-that...
- GnarfGnarf 7y ago
  Good point. The class definitions can easily be changed from float to double in just a few lines. I used to use a lot of doubles. The I wrote some DirectX and GPUs all use float. I figured if floats are good enough for 3D, they're good enough for me :o)
  - jcranmer 7y ago
    With floating-point code, you will slowly lose precision in your numbers. (How quickly is a matter of what calculations you're doing, if you're doing them in an error-tolerant manner, and how bad the initial error is in the first place). Single-precision floating-point starts you off with about 6-7 decimal digits of precision, and the intermediate rounding will mean that you're constantly introducing error in the 7th digit. Computer graphics are pretty tolerant of error--you only need about a 1-2 digits of precision generally--and achieving higher FLOPS is more useful than higher precision. Engineering tolerances are usually going to be tighter, perhaps 4-5 digits for particularly tight tolerances. Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences.
    - vardump 7y ago
      Yup. Although I'd like to add a typical issue when there's a small change in a large quantity. Say time. If you measure time in microseconds, and use floats, your precision will be halved to 2 us in just 33.5 seconds. In 24 hours, the precision has detoriated to over 2 milliseconds. One famous example is the deadly Patriot missile bug: https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf > Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences. Some other ones are iteration and simply long computation chains introducing systematic error.
blub 7y ago
There's a few libraries which implement this for all units in e.g. SI, like boost units or nholthaus/units on github. There's also the safe numerics lib in boost which catches UB and other issues around integer operations preferably at compile time and, if not possible, at run-time. From what I read about Ada, its types offer better guarantees than what's typically available in mainstream programming languages: https://learn.adacore.com/courses/intro-to-ada/chapters/strongly_typed_language.html#integers https://learn.adacore.com/courses/intro-to-ada/chapters/stro...
vardump 7y ago
I guess no one here bothered to actually benchmark scalar (like unit type/conversion code) floats vs doubles on modern x86 hardware. It will perform identically. Heck, don't believe me, check Agner's x86 instruction tables [0]. Please default to doubles and optimize only when it's actually an issue. 99% of cases it's not, unless you're doing throughput computing. In that case floats provide up to 2x improvement. Keep in mind modern x86 CPUs can perform up to 16 double precision FLOPs (FMA) per cycle per core — do you really think it's going to be the bottleneck in your code? 32-bit floats expose you to nasty issues with even "simple" stuff like errors in naively computed sums. Issues that mostly just wouldn't matter if 64-bit doubles were used. See a proper floating point summation algorithm [1]. That said, even 64-bit doubles (or any arbitrarily long imprecise format) won't shield you from all of floating point weirdness. But it sure does help in common cases. With floats you'll get issues like this: https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ https://randomascii.wordpress.com/2012/02/13/dont-store-that... [0]: https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf [1]: https://en.wikipedia.org/wiki/Kahan_summation_algorithm https://en.wikipedia.org/wiki/Kahan_summation_algorithm

vardump 7y ago

Why use floats when you can basically have doubles for the same cost? Floats tend to cause nasty surprises when you deal with real world units.

LPDWORD 7y ago
> Why use floats when you can basically have doubles for the same cost? Floats costing the same as doubles is a myth stemming from x87 arithmetic, which is obsolete. On an x64 CPU running 64-bit code, your compiler can often pack four floats into one SSE register. Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87.
- vardump 7y ago
  The difference is meaningless in scalar code. Not everything is or can be vectorized. Pretty much no difference between one double and one float in a SSE XMM register. > Even when that doesn't happen, CPU microcode can likely do more with floats than with doubles. I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. > Lastly, memory bandwidth usage and cache occupancy doubles, which is true even with x87. So use floats when you have a lot of data.
  - LPDWORD 7y ago
    If you already knew there was a cost difference, why didn't you just say so? > The difference is meaningless in scalar code. Not everything is or can be vectorized. Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. > Pretty much no difference between one double and one float in a SSE XMM register. Even then, some arithmetic operations have higher throughput with floats than with doubles. > I have no idea what that means. As far as I know, there's no CPU microcode dealing with floating point numbers. Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. > So use floats when you have a lot of data. That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference.
    - vardump 7y ago
      > Multiple scalars can be packed into one SSE register even in scalar code. This is not the same as vectorization. They can, but can you honestly call it a common case for scalar code? > Even then, some arithmetic operations have higher throughput with floats than with doubles. By far the most (90-99%) of FP computation is additions and multiplications (or fused multiply adds). For scalar case (1 double or float in SSE register), they take precisely as long on modern x86 hardware. Sure, float div executes in 11 instead of 13-14 clocks for doubles and I'm sure transcendentals are even worse, but they're rarely needed. Even then, if the dependency chain allows, the cost is often OoO scheduled away in integer dominated code. > Your CPU doesn't execute the ISA directly in hardware, it first converts it into architecture specific micro-ops. That includes FP operations. Except that SSE instructions pretty much are micro-ops as-is. Despite similarly sounding term, microcode has nothing to do with micro-ops. > That's not the point. A lot of the time, the cost difference will indeed be irrelevant. The point is that there is a cost difference. Well, I've written a lot of SSE, AVX etc. SIMD code. There sure is a big difference when you're processing large amounts of data. But... I've seen floats introducing silly precision related bugs [0] and a ton of useless float -> double -> float conversion chains. Most of the time most programmers should default to double. [0]: Example: https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ https://randomascii.wordpress.com/2012/02/13/dont-store-that...
GnarfGnarf 7y ago
Good point. The class definitions can easily be changed from float to double in just a few lines. I used to use a lot of doubles. The I wrote some DirectX and GPUs all use float. I figured if floats are good enough for 3D, they're good enough for me :o)
- jcranmer 7y ago
  With floating-point code, you will slowly lose precision in your numbers. (How quickly is a matter of what calculations you're doing, if you're doing them in an error-tolerant manner, and how bad the initial error is in the first place). Single-precision floating-point starts you off with about 6-7 decimal digits of precision, and the intermediate rounding will mean that you're constantly introducing error in the 7th digit. Computer graphics are pretty tolerant of error--you only need about a 1-2 digits of precision generally--and achieving higher FLOPS is more useful than higher precision. Engineering tolerances are usually going to be tighter, perhaps 4-5 digits for particularly tight tolerances. Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences.
  - vardump 7y ago
    Yup. Although I'd like to add a typical issue when there's a small change in a large quantity. Say time. If you measure time in microseconds, and use floats, your precision will be halved to 2 us in just 33.5 seconds. In 24 hours, the precision has detoriated to over 2 milliseconds. One famous example is the deadly Patriot missile bug: https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf https://sdqweb.ipd.kit.edu/publications/pdfs/saglam2016a.pdf > Furthermore, you're more likely to use nastier equations (logarithms and exponents are particularly bad for precision), so single-precision could well introduce error with real-world consequences. Some other ones are iteration and simply long computation chains introducing systematic error.

blub 7y ago

There's a few libraries which implement this for all units in e.g. SI, like boost units or nholthaus/units on github. There's also the safe numerics lib in boost which catches UB and other issues around integer operations preferably at compile time and, if not possible, at run-time. From what I read about Ada, its types offer better guarantees than what's typically available in mainstream programming languages: https://learn.adacore.com/courses/intro-to-ada/chapters/strongly_typed_language.html#integers https://learn.adacore.com/courses/intro-to-ada/chapters/stro...

vardump 7y ago

I guess no one here bothered to actually benchmark scalar (like unit type/conversion code) floats vs doubles on modern x86 hardware. It will perform identically. Heck, don't believe me, check Agner's x86 instruction tables [0]. Please default to doubles and optimize only when it's actually an issue. 99% of cases it's not, unless you're doing throughput computing. In that case floats provide up to 2x improvement. Keep in mind modern x86 CPUs can perform up to 16 double precision FLOPs (FMA) per cycle per core — do you really think it's going to be the bottleneck in your code? 32-bit floats expose you to nasty issues with even "simple" stuff like errors in naively computed sums. Issues that mostly just wouldn't matter if 64-bit doubles were used. See a proper floating point summation algorithm [1]. That said, even 64-bit doubles (or any arbitrarily long imprecise format) won't shield you from all of floating point weirdness. But it sure does help in common cases. With floats you'll get issues like this: https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ https://randomascii.wordpress.com/2012/02/13/dont-store-that... [0]: https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf [1]: https://en.wikipedia.org/wiki/Kahan_summation_algorithm https://en.wikipedia.org/wiki/Kahan_summation_algorithm