6 ms·

Articles like this make me wonder how everything even works if basic math functions can be so wrong and how a simple concept like floor can have different defin

Articles like this make me wonder how everything even works if basic math functions can be so wrong and how a simple concept like floor can have different definitions/ implementations

marketingPro 6y ago
Someone correct me if I'm wrong because it's been years since my EE class. Basic math ends at division. Heck if you learn how math works on computers, it feels like a number trick. Those are reliable. The issue comes when you need to track never ending decimal points. Even if every transistor was dedicated to your problem, you'd still need to round. So even if you have 32 bits of decimals, there will be rounding sometimes. The human has to decide what to do. Instead of converting the binary to decimal and calling it done, it's more important to be consistent and predictable. The issues described in the article arent 5/2= 2 billion. The article talks about rounding giving you always even numbers, which you can imagine leads to issues if you are using both float and using logic that involves even and odd numbers. Edit- can someone correct me or add details, I'm very interested but I'm not taking ENG 240 again... :p
- esperent 6y ago
  Exactly this. The floor function works on binary numbers but has to round them to a decimal place. It's not simple, it just appears to be.
  - bonoboTP 6y ago
    Floor has nothing to do with decimal.
    - erichocean 6y ago
      The period in "3.14" is called a "decimal." The digits after the decimal in "3.14" are enumerated as "decimal places." What each decimal place "means" depends on the numeric base. So, "decimal places" or "rounding to a decimal place" has nothing to do with base 10 (which is what I think you are implying): decimal and "decimal places" exist in all bases, including base 2, and the meaning of each "decimal place" depends on the numeric base (base 10, base 2, etc.).
      - nitrogen 6y ago
        Sometimes people use "binary point" to distinguish from decimal point and avoid the confusion of the dec- prefix.
        erichocean 6y ago
        "Fraction point" and "fraction (or fractional) places" might be a decent alternative that clearly works in any base, while also conveying its purpose.
        SAI_Peregrinus 6y ago
        "Radix point" or "radix place" are the generic terms I've seen most often.
- CydeWeys 6y ago
  This is why the big rational data type is so useful on occasion. When you just want to get one calculation exactly right, and performance doesn't matter so much because you aren't doing huge numbers of repeated operations (as in ML), use a big rational with its arbitrary precision numerator and denominator and you don't need to worry about even epsilon error.
  - bonoboTP 6y ago
    Except if you run exp, log, sqrt, sin, cos etc. Then you immediately leave the rationals.
    - CydeWeys 6y ago
      Yes, it definitely depends on what you're doing with the numbers, but personally I don't recall ever having needed trigonometric functions in my decade plus working life doing standard line of business type stuff.
      - marketingPro 6y ago
        I can't tell if you are dismissing this, but as an FYI trig is used in Engineering.
        CydeWeys 6y ago
        I said "it definitely depends on what you're doing with the numbers". That's not a dismissal; I'm simply pointing out that neither I nor many if not most other software engineers use trig on a frequent basis. I'm well aware that it's used in actual engineering, but that's just not what most of us here do.
  - visarga 6y ago
    > because you aren't doing huge numbers of repeated operations (as in ML) In ML precision doesn't count so much. It's often possible that we train on 32bits and deploy the model on 16bits. In fact stochasticity is useful and we add it in by dropout or other things. You can drop any connection (or millions of them) in a neural net and get almost the same result. There have been papers that showed that reducing the number of bits used to represent weights might even improve the network because it has a regularising effect. The real brain is also stochastic.
Frost1x 6y ago
It often doesn't to high degrees of precision and reliability. It works within some tolerance range and those tolerance ranges are managed or acceptable levels of error (often unnoticed). I recently helped move/port one fairly complex numeric model from a language and environment to another language and environment and there was an assumption made by upper management that this was simple and exact deterministic results would be easily had. I warned that this could be an involved process because of simple variations, lack of equivalent mappings between environments and languages, some of which pointed out in this article. Management were quite surprised with the level of effort needed to dig down layers of abstraction and identify all these sort of variations in numeric libraries from consistent high level abstractions they were familiar with, and why those high level abstractions simply didn't exist in the other language and environment. They were additionally surprised inthe level of effort needed to get satisfactory error tolerance ranges between the two models when having to recreate high level abstractions identically from the source language and environment to the target language and environment that simply didn't exist prior to the work. They didn't realize the variation until we performed a detailed analysis on results and found very small but statistically distinguishable results from the two versions. We ended up identifying a few dozen small assumptions they made about their high level abstractions from the source environment that were false and caused the variation.
pierrebai 6y ago
The answer is that the bugs the article claims are corners cases, most of which will not matter in most practical uses of floating point maths... and some of which will is due to not emulating a weird IEEE floating point rule about rounding numbers of the form N.5 to the even number instead of up like everyone is taught in school. So "wrong" as used in the article simply means "not the same as the IEEE behavior", not "wrong" as in "wildly inaccurate". It matters if you want 100% exact emulation to get identical results. It doesn't if you don't and just want speed.
- CydeWeys 6y ago
  > some of which will is due to not emulating a weird IEEE floating point rule about rounding numbers of the form N.5 to the even number instead of up like everyone is taught in school. Round-to-even (also known as banker's rounding, because it's used in finance too) isn't weird at all. It minimizes rounding error, especially on repeated calculations. The rounding rule that everyone learns in elementary school is bad and results in systematically biased computations. We can and should do better than that flawed approach. This post explains it in more depth: https://mathematica.stackexchange.com/a/2120 https://mathematica.stackexchange.com/a/2120
- phkahler 6y ago
  >> The answer is that the bugs the article claims are corners cases, most of which will not matter in most practical uses of floating point maths... Most bugs are corner cases of one sort or another. At the application level you can decide how important the corners are. When you're creating a math library that many others are going to use, and there is a standard, I think the expectations should be higher.
- brucedawson 6y ago
  I recently saw a talk (internal only unfortunately) about how some "minor" changes in sin/cos/whatever functions in Chrome broke multiple web sites. It was things like mapping sites and the "minor" changes in the output of sin led to huge errors in the maps. Which is to say, that minor errors often don't matter, but if you are offering up a general purpose solution then those minor errors probably matter to _somebody_.
jlebar 6y ago
Six Stages of Debugging 1. That can’t happen. 2. That doesn’t happen on my machine. 3. That shouldn’t happen. 4. Why does that happen? 5. Oh, I see. 6. How did that ever work? http://plasmasturm.org/log/6debug/ http://plasmasturm.org/log/6debug/
- dredmorbius 6y ago
  The DNS haiku; It's not DNS. There's no way it's DNS. It was DNS. https://www.cyberciti.biz/humour/a-haiku-about-dns/ https://www.cyberciti.biz/humour/a-haiku-about-dns/
SoSoRoCoCo 6y ago
You should see how confusing modulo becomes of the method isn't explicitly specified: https://en.wikipedia.org/wiki/Modulo_operation https://en.wikipedia.org/wiki/Modulo_operation

Articles like this make me wonder how everything even works if basic math functions can be so wrong and how a simple concept like floor can have different definitions/ implementations

marketingPro 6y ago

Someone correct me if I'm wrong because it's been years since my EE class. Basic math ends at division. Heck if you learn how math works on computers, it feels like a number trick. Those are reliable. The issue comes when you need to track never ending decimal points. Even if every transistor was dedicated to your problem, you'd still need to round. So even if you have 32 bits of decimals, there will be rounding sometimes. The human has to decide what to do. Instead of converting the binary to decimal and calling it done, it's more important to be consistent and predictable. The issues described in the article arent 5/2= 2 billion. The article talks about rounding giving you always even numbers, which you can imagine leads to issues if you are using both float and using logic that involves even and odd numbers. Edit- can someone correct me or add details, I'm very interested but I'm not taking ENG 240 again... :p

esperent 6y ago
Exactly this. The floor function works on binary numbers but has to round them to a decimal place. It's not simple, it just appears to be.
- bonoboTP 6y ago
  Floor has nothing to do with decimal.
  - erichocean 6y ago
    The period in "3.14" is called a "decimal." The digits after the decimal in "3.14" are enumerated as "decimal places." What each decimal place "means" depends on the numeric base. So, "decimal places" or "rounding to a decimal place" has nothing to do with base 10 (which is what I think you are implying): decimal and "decimal places" exist in all bases, including base 2, and the meaning of each "decimal place" depends on the numeric base (base 10, base 2, etc.).
    - nitrogen 6y ago
      Sometimes people use "binary point" to distinguish from decimal point and avoid the confusion of the dec- prefix.
      - erichocean 6y ago
        "Fraction point" and "fraction (or fractional) places" might be a decent alternative that clearly works in any base, while also conveying its purpose.
      - SAI_Peregrinus 6y ago
        "Radix point" or "radix place" are the generic terms I've seen most often.
CydeWeys 6y ago
This is why the big rational data type is so useful on occasion. When you just want to get one calculation exactly right, and performance doesn't matter so much because you aren't doing huge numbers of repeated operations (as in ML), use a big rational with its arbitrary precision numerator and denominator and you don't need to worry about even epsilon error.
- bonoboTP 6y ago
  Except if you run exp, log, sqrt, sin, cos etc. Then you immediately leave the rationals.
  - CydeWeys 6y ago
    Yes, it definitely depends on what you're doing with the numbers, but personally I don't recall ever having needed trigonometric functions in my decade plus working life doing standard line of business type stuff.
    - marketingPro 6y ago
      I can't tell if you are dismissing this, but as an FYI trig is used in Engineering.
      - CydeWeys 6y ago
        I said "it definitely depends on what you're doing with the numbers". That's not a dismissal; I'm simply pointing out that neither I nor many if not most other software engineers use trig on a frequent basis. I'm well aware that it's used in actual engineering, but that's just not what most of us here do.
- visarga 6y ago
  > because you aren't doing huge numbers of repeated operations (as in ML) In ML precision doesn't count so much. It's often possible that we train on 32bits and deploy the model on 16bits. In fact stochasticity is useful and we add it in by dropout or other things. You can drop any connection (or millions of them) in a neural net and get almost the same result. There have been papers that showed that reducing the number of bits used to represent weights might even improve the network because it has a regularising effect. The real brain is also stochastic.

Frost1x 6y ago

It often doesn't to high degrees of precision and reliability. It works within some tolerance range and those tolerance ranges are managed or acceptable levels of error (often unnoticed). I recently helped move/port one fairly complex numeric model from a language and environment to another language and environment and there was an assumption made by upper management that this was simple and exact deterministic results would be easily had. I warned that this could be an involved process because of simple variations, lack of equivalent mappings between environments and languages, some of which pointed out in this article. Management were quite surprised with the level of effort needed to dig down layers of abstraction and identify all these sort of variations in numeric libraries from consistent high level abstractions they were familiar with, and why those high level abstractions simply didn't exist in the other language and environment. They were additionally surprised inthe level of effort needed to get satisfactory error tolerance ranges between the two models when having to recreate high level abstractions identically from the source language and environment to the target language and environment that simply didn't exist prior to the work. They didn't realize the variation until we performed a detailed analysis on results and found very small but statistically distinguishable results from the two versions. We ended up identifying a few dozen small assumptions they made about their high level abstractions from the source environment that were false and caused the variation.

pierrebai 6y ago

The answer is that the bugs the article claims are corners cases, most of which will not matter in most practical uses of floating point maths... and some of which will is due to not emulating a weird IEEE floating point rule about rounding numbers of the form N.5 to the even number instead of up like everyone is taught in school. So "wrong" as used in the article simply means "not the same as the IEEE behavior", not "wrong" as in "wildly inaccurate". It matters if you want 100% exact emulation to get identical results. It doesn't if you don't and just want speed.

CydeWeys 6y ago
> some of which will is due to not emulating a weird IEEE floating point rule about rounding numbers of the form N.5 to the even number instead of up like everyone is taught in school. Round-to-even (also known as banker's rounding, because it's used in finance too) isn't weird at all. It minimizes rounding error, especially on repeated calculations. The rounding rule that everyone learns in elementary school is bad and results in systematically biased computations. We can and should do better than that flawed approach. This post explains it in more depth: https://mathematica.stackexchange.com/a/2120 https://mathematica.stackexchange.com/a/2120
phkahler 6y ago
>> The answer is that the bugs the article claims are corners cases, most of which will not matter in most practical uses of floating point maths... Most bugs are corner cases of one sort or another. At the application level you can decide how important the corners are. When you're creating a math library that many others are going to use, and there is a standard, I think the expectations should be higher.
brucedawson 6y ago
I recently saw a talk (internal only unfortunately) about how some "minor" changes in sin/cos/whatever functions in Chrome broke multiple web sites. It was things like mapping sites and the "minor" changes in the output of sin led to huge errors in the maps. Which is to say, that minor errors often don't matter, but if you are offering up a general purpose solution then those minor errors probably matter to _somebody_.

jlebar 6y ago

Six Stages of Debugging 1. That can’t happen. 2. That doesn’t happen on my machine. 3. That shouldn’t happen. 4. Why does that happen? 5. Oh, I see. 6. How did that ever work? http://plasmasturm.org/log/6debug/ http://plasmasturm.org/log/6debug/

dredmorbius 6y ago
The DNS haiku; It's not DNS. There's no way it's DNS. It was DNS. https://www.cyberciti.biz/humour/a-haiku-about-dns/ https://www.cyberciti.biz/humour/a-haiku-about-dns/

SoSoRoCoCo 6y ago

You should see how confusing modulo becomes of the method isn't explicitly specified: https://en.wikipedia.org/wiki/Modulo_operation https://en.wikipedia.org/wiki/Modulo_operation