6 ms·

In my opinion, the confusion that arises when programmers get results from floating-point computations that are not what they expect stems from this: > Floats

In my opinion, the confusion that arises when programmers get results from floating-point computations that are not what they expect stems from this: > Floats represent continuous values. But as you probably know, this isn't possible. The concept of infinite precision is interesting in theory, but disappears when any actual calculation needs to be made, whether on a digital electric computer or not. I wonder if this is not a flaw in the crude mechanical representation of numbers, but a flaw in the decision to base floating-point computation on the concept of continuous numbers. I believe that a better model for floating-point computational representation and manipulation would be to reflect the rules of scientific measurements - that each number includes an explicit amount of precision that is preserved during mathematical operations. This would not only keep JavaScript newbies from freaking out when they add 0.1 and 0.2, but prevent problems of thinking calculation results are correct, when they are not. If you aren't getting what I am saying, let me give an example. Let's say, for some reason, you want to measure the diameter of a ball. You have a measuring tape so you wrap it around the widest part and record that it is 23.5cm. To calculate the radius, you should divide by π. If you do this in double-precision floating point, you will get 7.480282325319081, but this is nonsense. You can't create a result that is magically more precise than your initial measurement though division or multiplication. The correct answer is 7.48cm. This preserves the amount of precision in the least precise operand, and is arguably the most correct result.

roywiggins 8y ago
Sounds a bit like unums: https://en.wikipedia.org/wiki/Unum_(number_format) https://en.wikipedia.org/wiki/Unum_(number_format)
zamadatix 8y ago
I'm not convinced that has something to do with why floats are so confusing. The same could be said about integers but people are fine with 1000/16 as it matches how they naturally think about integers even though in the computer it's done in binary. To me it seems much more likely the simple explanation that 2^E+1.M for some number of bits is unnatural to people used to n*10^E with an unlimited number of digits. To drive this home I don't think I ever heard anyone be confused why 2 fixed point numbers resulted in a certain value after being told what a fixed point number was.
im3w1l 8y ago
I've seen this idea of storing the precision mentioned many times on HN, but I must say I don't believe in it outside of some few niches. First reason being that it's much more complex and it's unclear what the complexity buys us. Second reason is that it doesn't model how variables co-vary. As a toy example imagine that I have a number x: 5+-1. Then I let y = x - 1: 4+-1 Finally I let z = 1 / (x - y). Now, by construction z will be very close to 1. But a system naively tracking uncertainties will be very concerned x - y. If it does a worst case analysis it gets 1+-2. If it does an average case analysis assuming independent gaussian errors it gets 1+-sqrt(2). When we perform the division our uncertainty goes infinite.
- protonfish 8y ago
  I don't see any reason to claim that explicit-precision floating-point is more complex, just different. Yes, change is uncomfortable and takes effort, but it does necessarily mean the new way is inherently more complicated. I worry about objections based on "that's not what we are taught in school." I think that what we teach can (and should) be improved if need be, and not used as a motivation to deny criticism of established dogma. I am not sure if I fully understand your example, but I don't see any problem with it. Using basic significant figure rules, this is (with an additional step for clarity): x: 5e0 y = x - 1: 4e0 z1 = x - y: 1e0 z = 1/1e0 z2 = 1e0 The answer seems to simply be 1+-1. "Significant Figures" are a simplification of precision where the precision is an integer that represents the total digits of the least precise measurement. A more accurate way is to represent precision as standard deviation, then calculate the precision of the result with basic statistical techniques.
- jcranmer 8y ago
  I recently saw a presentation that modeled floating point error using Monte Carlo simulation. Any time you introduce uncertainty (such as the sub-ULP rounding error in every operation), you can insert random variation and find which bits end up being stable to find out how good your answer is. While the point was modelling floating point error, it's not hard to extend the idea to modelling the input measurement error in the first place.
comicjk 8y ago
I like the idea of storing the precision by default. Some algorithms already do this in order to keep track of what they're losing in lossy arithmetic operations, such as adding small numbers to big ones. It would be nontrivial to implement this efficiently - the simplest implementation would require a second float for each original float that has an uncertainty, doubling the time and memory requirements. A tensorflow-like system which can see the whole flow of numbers might be able to provide precision estimates efficiently only where needed.
- protonfish 8y ago
  I don't know if an entire floating-point number is necessary to keep track of precision. If we are willing to accept precision to the closest decimal digit, and my calculations of the max mantissa of a double-precision floating-point is 9.0071993e+15, we could store the precision in 4 bits.

roywiggins 8y ago

Sounds a bit like unums: https://en.wikipedia.org/wiki/Unum_(number_format) https://en.wikipedia.org/wiki/Unum_(number_format)

zamadatix 8y ago

I'm not convinced that has something to do with why floats are so confusing. The same could be said about integers but people are fine with 1000/16 as it matches how they naturally think about integers even though in the computer it's done in binary. To me it seems much more likely the simple explanation that 2^E+1.M for some number of bits is unnatural to people used to n*10^E with an unlimited number of digits. To drive this home I don't think I ever heard anyone be confused why 2 fixed point numbers resulted in a certain value after being told what a fixed point number was.

im3w1l 8y ago

I've seen this idea of storing the precision mentioned many times on HN, but I must say I don't believe in it outside of some few niches. First reason being that it's much more complex and it's unclear what the complexity buys us. Second reason is that it doesn't model how variables co-vary. As a toy example imagine that I have a number x: 5+-1. Then I let y = x - 1: 4+-1 Finally I let z = 1 / (x - y). Now, by construction z will be very close to 1. But a system naively tracking uncertainties will be very concerned x - y. If it does a worst case analysis it gets 1+-2. If it does an average case analysis assuming independent gaussian errors it gets 1+-sqrt(2). When we perform the division our uncertainty goes infinite.

protonfish 8y ago
I don't see any reason to claim that explicit-precision floating-point is more complex, just different. Yes, change is uncomfortable and takes effort, but it does necessarily mean the new way is inherently more complicated. I worry about objections based on "that's not what we are taught in school." I think that what we teach can (and should) be improved if need be, and not used as a motivation to deny criticism of established dogma. I am not sure if I fully understand your example, but I don't see any problem with it. Using basic significant figure rules, this is (with an additional step for clarity): x: 5e0 y = x - 1: 4e0 z1 = x - y: 1e0 z = 1/1e0 z2 = 1e0 The answer seems to simply be 1+-1. "Significant Figures" are a simplification of precision where the precision is an integer that represents the total digits of the least precise measurement. A more accurate way is to represent precision as standard deviation, then calculate the precision of the result with basic statistical techniques.
jcranmer 8y ago
I recently saw a presentation that modeled floating point error using Monte Carlo simulation. Any time you introduce uncertainty (such as the sub-ULP rounding error in every operation), you can insert random variation and find which bits end up being stable to find out how good your answer is. While the point was modelling floating point error, it's not hard to extend the idea to modelling the input measurement error in the first place.

comicjk 8y ago

I like the idea of storing the precision by default. Some algorithms already do this in order to keep track of what they're losing in lossy arithmetic operations, such as adding small numbers to big ones. It would be nontrivial to implement this efficiently - the simplest implementation would require a second float for each original float that has an uncertainty, doubling the time and memory requirements. A tensorflow-like system which can see the whole flow of numbers might be able to provide precision estimates efficiently only where needed.

protonfish 8y ago
I don't know if an entire floating-point number is necessary to keep track of precision. If we are willing to accept precision to the closest decimal digit, and my calculations of the max mantissa of a double-precision floating-point is 9.0071993e+15, we could store the precision in 4 bits.