4 ms·

There's always fixed point integers[0]. They're less flexible than floats, but at least the amount of precision available is much more clear and uniform, and al

by tbirdz 10y ago

There's always fixed point integers[0]. They're less flexible than floats, but at least the amount of precision available is much more clear and uniform, and all the rounding is pretty explicit. https://en.wikipedia.org/wiki/Fixed-point_arithmetic https://en.wikipedia.org/wiki/Fixed-point_arithmetic https://en.wikipedia.org/wiki/Q_%28number_format%29 https://en.wikipedia.org/wiki/Q_%28number_format%29

rwallace 10y ago
Back in the late eighties, early nineties, I was of the opinion that fixed point is really the way to go, the only thing holding it back was lack of adequate compiler support. Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. The headaches of floating point, annoying though they can be at times, are much less onerous.
- MaulingMonkey 10y ago
  > Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. Floating point doesn't actually fix this issue. A fun bug I dealt with was a black screen caused by a few pixels having NaN values - which were then picked up by a blur pass which would poison basically the entire back buffer. The original NaNs were caused by divides by near-zero values causing out-of-range results. You now add precision issues: Multiple coordinate systems are a common hack-around to maintain local precision when using single-precision based physics engines. Stuttering effects when time is represented as "floating point seconds since uptime" when integer wraparound would avoid any problem in the first place. And this is before even adding e.g. half precision or worse floats into the mix. There's a reason floating point isn't used for a lot of serious banking calculations.
  - rwallace 10y ago
    To clarify, any time I in any way praise floating point, I'm always talking about double precision. I think single precision floating point should never have been invented, and given that it unfortunately was, it's best to behave as though it doesn't exist. Don't get me started about half precision. Financial calculations are another matter; they usually require decimal arithmetic, not because it doesn't give rounding errors, but because it gives the same rounding errors an accountant gets when he checks the result with his desk calculator, and that's the requirement.
    - MaulingMonkey 10y ago
      > To clarify, any time I in any way praise floating point, I'm always talking about double precision. 64-bit fixed point solves a lot of problems too :) > I think single precision floating point should never have been invented, and given that it unfortunately was, it's best to behave as though it doesn't exist. Don't get me started about half precision. A 4x memory bandwidth & storage penalty for HDR buffers would be absolute murder (bloating a single 4k RGBA surface to 265MB if I've done my math right), so I'm glad we have the option. And in the historical context lower precision options were invented in, RAM wasn't nearly as cheap as it is today. Tradeoffs.

by tbirdz 10y ago

rwallace 10y ago

Back in the late eighties, early nineties, I was of the opinion that fixed point is really the way to go, the only thing holding it back was lack of adequate compiler support. Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. The headaches of floating point, annoying though they can be at times, are much less onerous.

MaulingMonkey 10y ago
> Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. Floating point doesn't actually fix this issue. A fun bug I dealt with was a black screen caused by a few pixels having NaN values - which were then picked up by a blur pass which would poison basically the entire back buffer. The original NaNs were caused by divides by near-zero values causing out-of-range results. You now add precision issues: Multiple coordinate systems are a common hack-around to maintain local precision when using single-precision based physics engines. Stuttering effects when time is represented as "floating point seconds since uptime" when integer wraparound would avoid any problem in the first place. And this is before even adding e.g. half precision or worse floats into the mix. There's a reason floating point isn't used for a lot of serious banking calculations.
- rwallace 10y ago
  To clarify, any time I in any way praise floating point, I'm always talking about double precision. I think single precision floating point should never have been invented, and given that it unfortunately was, it's best to behave as though it doesn't exist. Don't get me started about half precision. Financial calculations are another matter; they usually require decimal arithmetic, not because it doesn't give rounding errors, but because it gives the same rounding errors an accountant gets when he checks the result with his desk calculator, and that's the requirement.
  - MaulingMonkey 10y ago
    > To clarify, any time I in any way praise floating point, I'm always talking about double precision. 64-bit fixed point solves a lot of problems too :) > I think single precision floating point should never have been invented, and given that it unfortunately was, it's best to behave as though it doesn't exist. Don't get me started about half precision. A 4x memory bandwidth & storage penalty for HDR buffers would be absolute murder (bloating a single 4k RGBA surface to 265MB if I've done my math right), so I'm glad we have the option. And in the historical context lower precision options were invented in, RAM wasn't nearly as cheap as it is today. Tradeoffs.