3 ms·

This result is strictly a result of floating point math. You never get this kind of result with fixed point or binary coded decimal math. The computer Languag

This result is strictly a result of floating point math. You never get this kind of result with fixed point or binary coded decimal math. The computer Language M never has this kind of problem.

compiler-guy 4y ago
The article isn’t about the imprecision in general, but rather about the details of this particular equations imprecision, and how that leads to this particular answer.
pwdisswordfishc 4y ago
How do you represent 1/10 exactly in binary fixed-point representation?
- tsukikage 4y ago
  ...or, indeed, 1/3 as binary coded decimal.
- amelius 4y ago
  > How do you represent 1/10 exactly in binary fixed-point representation? The base of your number system does not need to be the same as the base associated with the fixed point position. Easiest to explain: if you have a uint64 that represents a monetary value, you can let it express the number of dollarcents instead of dollars. Then you can express 1/10 dollar as 10 dollarcents.
  - jstanley 4y ago
    The claim was "you never get this kind of result with fixed point math", not "you can contrive to avoid this kind of result with fixed point math if you use a different base for the fractional part".
- whitten 4y ago
  1/10 is exactly 0.1
lifthrasiir 4y ago
So 1/3 + 1/3 + 1/3 = 1 in M, right?
- dragonwriter 4y ago
  Well, it is in racket? Welcome to Racket v8.7 [cs]. > (/ 1 3) 1/3 > (+ (/ 1 3) (/ 1 3) (/ 1 3)) 1 > (integer? (+ (/ 1 3) (/ 1 3) (/ 1 3))) #t
PaulHoule 4y ago
It is not "floating point math" but "floating point math where the exponent is expressed as a power of 2" that is the problem. That is, if the exponent is a power of 2 you can write 1/2, 1/4, 1/8 exactly but you can't write 1/3, 1/5, 1/10, etc. If the exponent is base 10 then you can write 1/5, 1/10, 1/1000 and such exactly. Note the mantissa and the exponent are both integers and so far as this problem is concerned it does not matter if these are written in binary or BCD or some other representation. There is some controversy about what is better, if you use a binary mantissa the math is a little faster and more accurate, if you use a decimal mantissa conversions to and from ASCII are quicker and ASCII conversions are a major part of real life math workloads. I've long thought that this problem is one of a list of problems that many people encounter on the path to bending computers to their will and that some people decide that computer programming isn't for them because of this kind of problem. I think the kind of person who learns Python to put their outside-of-computing skills on wheels is particularly affected. It is a "disruptive technology" problem because the person who is using IEEE floats heavily has accommodated to this problem and would not give up the slightest amount of performance. Decimal FP can be implemented in software but is slow in software. IBM has had hardware Decimal FP in their mainframes for a very long time and there is even an IEEE standard. (A company that has been using mainframes for a long time cut a check for the wrong amount because the abused the number system back in 1963 and thus learned their lesson a long time ago.) The best hope I have is that the "social justice" people can be led to believe that unintuitive numerics keep underrepresented people out of the field and that they threaten Intel that they'll tear down their headquarters unless they catch up to where mainframes were 50 years ago. It could be a huge win for the industry and for the DEI office because employers would have to buy everyone a new computer and even white guys might think the DEI office was doing good work if it meant they got to replace their 5 year old corporate craptop. I mean, how is it that a few people with two fingers get to oppress all the rest of us with ten?
edflsafoiewq 4y ago
Fixed point behaves well under addition and multiplication by integers. Otherwise it isn't very good. There are far more numbers in the range (1,infinity) than (0,1) for example, so 1/x loses catastrophic amounts of information.
dragonwriter 4y ago
> This result is strictly a result of floating point math. This specific result is a result of binary floating point math with a particular precision. More precision, or decimal floating point, will fix that, but have similar kinds of errors for the same operation on different numbers. fixed-size BCD/fixed-point math has other limitations, its not a general solution. The general solution is to have a numeric tower where representations and operations meet the following rules: 1. The default representation of any exact literal (without a modifier representing a particular inexact representation, e.g., as an optimization or a necessity of interfacing with an external library) is exact, 2. Any operation on any operations between exact representations that can be done exactly is unless specified otherwise (as it might be for the same reasons discussed above), and stored in a representation that can represent the result exactly. 3. Essentially inexact operations or operations on inexact numbers are conducted in a way and produce output representations that minimize additional imprecision introduced, except when explicitly specified otherwise. Computer algebra systems where the “top level representation is symbolic are potentially the ultimate expression of this, but the Scheme numeric tower is pretty good (but at least Racket, and I think schemes in general, represent exact decimal fractions as floats still, so don’t entirely avoid the problem, but at least division of integers produces exact rationals.) Lots of languages default to putting numbers expressed as literals into either fixed-sized integers (not bad, especially when those are often 64-bit now which rarely has much practical distinction from arbitrary precision in most applications) or fixed-sized binary floats (which are more problematic, especially given the mismatch between clean binary and clean decimal representations.) This is very good for efficiency, because computers can process fixed sized integers and binary floats very quickly. But, especially for floats, it can be bad for correctness when doing arithmetic where the input is all clean decimal literals.
- aag 4y ago
  Yes, Scheme has the concept of exactness[0]. In most Schemes, (exact? 2.3) ==> #f. But at least the exact? procedure exists. It can tell you whether your result might have been affected by these problems. If you want perfect answers, you can always use the rationals that are built into Scheme implementations with the full numeric tower. [0] https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-8.html#TAG:__tex2page_sec_6.2.2 https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-8.html#...
dragonwriter 4y ago
Tangentially (and separate from my other response because it's a whole different issue): > The computer Language M Which one? MUMPS (also known as M), or the Power Query Formula Language (also known as M)? OR something else?
- whitten 4y ago
  M alternately named MUMPS M guarantees over 15 digits of precision, which is why it is heavily used in banking and financial applications

This result is strictly a result of floating point math. You never get this kind of result with fixed point or binary coded decimal math. The computer Language M never has this kind of problem.

compiler-guy 4y ago

The article isn’t about the imprecision in general, but rather about the details of this particular equations imprecision, and how that leads to this particular answer.

pwdisswordfishc 4y ago

How do you represent 1/10 exactly in binary fixed-point representation?

tsukikage 4y ago
...or, indeed, 1/3 as binary coded decimal.
amelius 4y ago
> How do you represent 1/10 exactly in binary fixed-point representation? The base of your number system does not need to be the same as the base associated with the fixed point position. Easiest to explain: if you have a uint64 that represents a monetary value, you can let it express the number of dollarcents instead of dollars. Then you can express 1/10 dollar as 10 dollarcents.
- jstanley 4y ago
  The claim was "you never get this kind of result with fixed point math", not "you can contrive to avoid this kind of result with fixed point math if you use a different base for the fractional part".
whitten 4y ago
1/10 is exactly 0.1

lifthrasiir 4y ago

So 1/3 + 1/3 + 1/3 = 1 in M, right?

dragonwriter 4y ago
Well, it is in racket? Welcome to Racket v8.7 [cs]. > (/ 1 3) 1/3 > (+ (/ 1 3) (/ 1 3) (/ 1 3)) 1 > (integer? (+ (/ 1 3) (/ 1 3) (/ 1 3))) #t

PaulHoule 4y ago

It is not "floating point math" but "floating point math where the exponent is expressed as a power of 2" that is the problem. That is, if the exponent is a power of 2 you can write 1/2, 1/4, 1/8 exactly but you can't write 1/3, 1/5, 1/10, etc. If the exponent is base 10 then you can write 1/5, 1/10, 1/1000 and such exactly. Note the mantissa and the exponent are both integers and so far as this problem is concerned it does not matter if these are written in binary or BCD or some other representation. There is some controversy about what is better, if you use a binary mantissa the math is a little faster and more accurate, if you use a decimal mantissa conversions to and from ASCII are quicker and ASCII conversions are a major part of real life math workloads. I've long thought that this problem is one of a list of problems that many people encounter on the path to bending computers to their will and that some people decide that computer programming isn't for them because of this kind of problem. I think the kind of person who learns Python to put their outside-of-computing skills on wheels is particularly affected. It is a "disruptive technology" problem because the person who is using IEEE floats heavily has accommodated to this problem and would not give up the slightest amount of performance. Decimal FP can be implemented in software but is slow in software. IBM has had hardware Decimal FP in their mainframes for a very long time and there is even an IEEE standard. (A company that has been using mainframes for a long time cut a check for the wrong amount because the abused the number system back in 1963 and thus learned their lesson a long time ago.) The best hope I have is that the "social justice" people can be led to believe that unintuitive numerics keep underrepresented people out of the field and that they threaten Intel that they'll tear down their headquarters unless they catch up to where mainframes were 50 years ago. It could be a huge win for the industry and for the DEI office because employers would have to buy everyone a new computer and even white guys might think the DEI office was doing good work if it meant they got to replace their 5 year old corporate craptop. I mean, how is it that a few people with two fingers get to oppress all the rest of us with ten?

edflsafoiewq 4y ago

Fixed point behaves well under addition and multiplication by integers. Otherwise it isn't very good. There are far more numbers in the range (1,infinity) than (0,1) for example, so 1/x loses catastrophic amounts of information.

dragonwriter 4y ago

> This result is strictly a result of floating point math. This specific result is a result of binary floating point math with a particular precision. More precision, or decimal floating point, will fix that, but have similar kinds of errors for the same operation on different numbers. fixed-size BCD/fixed-point math has other limitations, its not a general solution. The general solution is to have a numeric tower where representations and operations meet the following rules: 1. The default representation of any exact literal (without a modifier representing a particular inexact representation, e.g., as an optimization or a necessity of interfacing with an external library) is exact, 2. Any operation on any operations between exact representations that can be done exactly is unless specified otherwise (as it might be for the same reasons discussed above), and stored in a representation that can represent the result exactly. 3. Essentially inexact operations or operations on inexact numbers are conducted in a way and produce output representations that minimize additional imprecision introduced, except when explicitly specified otherwise. Computer algebra systems where the “top level representation is symbolic are potentially the ultimate expression of this, but the Scheme numeric tower is pretty good (but at least Racket, and I think schemes in general, represent exact decimal fractions as floats still, so don’t entirely avoid the problem, but at least division of integers produces exact rationals.) Lots of languages default to putting numbers expressed as literals into either fixed-sized integers (not bad, especially when those are often 64-bit now which rarely has much practical distinction from arbitrary precision in most applications) or fixed-sized binary floats (which are more problematic, especially given the mismatch between clean binary and clean decimal representations.) This is very good for efficiency, because computers can process fixed sized integers and binary floats very quickly. But, especially for floats, it can be bad for correctness when doing arithmetic where the input is all clean decimal literals.

aag 4y ago
Yes, Scheme has the concept of exactness[0]. In most Schemes, (exact? 2.3) ==> #f. But at least the exact? procedure exists. It can tell you whether your result might have been affected by these problems. If you want perfect answers, you can always use the rationals that are built into Scheme implementations with the full numeric tower. [0] https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-8.html#TAG:__tex2page_sec_6.2.2 https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-8.html#...

dragonwriter 4y ago

Tangentially (and separate from my other response because it's a whole different issue): > The computer Language M Which one? MUMPS (also known as M), or the Power Query Formula Language (also known as M)? OR something else?

whitten 4y ago
M alternately named MUMPS M guarantees over 15 digits of precision, which is why it is heavily used in banking and financial applications