IEEE754

4 ms·

1. 1.▲
   by IEEE754 3y ago

   Correct.
2. 2.▲
   by IEEE754 5y ago

   > does not interpret the 0.1 as the real number The focus should be on _rational_ numbers. This particular example is all about representation error - precision is implicated, but not the cause. Ignore precision for a second: The inputs
3. 3.▲
   by IEEE754 7y ago

   > Over the past 50 years, my computer has adapted to how humans normally operate in nearly every other way. Why do they continue to use this system which produces results different from what any normal person expects? If you look at any
4. 4.▲
   by IEEE754 7y ago

   FWIW, their "basic answer" page is the simplest i've seen that neither lies or omits critical factors in the 0.1 + 0.2 problem. It's probably a good starting point that induces some lingering questions tempting you to fi
5. 5.▲
   by IEEE754 7y ago

   > 1. Replace the default format for numbers with a decimal point in suitably high level languages with a infinite precision format. Unlimited precision in any radix point based format does not solve representation error. If you don'
6. 6.▲
   by IEEE754 7y ago

   > Most think of it as base-10 [...] I'm surprised nobody has made a decent fixed point lib that is widely used already. Note that fixed radix point does not solve the common issues with representing rational base 10 fractions. A ba