15 ms·

Or perhaps it makes us mistrust more than we should? How often are we working on problems where the difference between 0.1 and 0.100000000000000006 is of any pr

Or perhaps it makes us mistrust more than we should? How often are we working on problems where the difference between 0.1 and 0.100000000000000006 is of any practical importance? When I format a float out to 5 decimal places, I'm sort of making a statement that anything beyond that doesn't matter to me.

tomxor 7y ago
> When I format a float out to 5 decimal places, I'm sort of making a statement that anything beyond that doesn't matter to me. Yes, it's true the vast majority of the time it doesn't actually matter. However decimal formatting does such a good job at giving us the impression that these errors are merely edge-cases, and calculators automatically formatting to 10sf etc further that illusion. If people are not aware it's only an illusion (or just how far the rabbit hole goes) it can be dangerous when they go on to create or use things where that fact matters.
- jancsika 7y ago
  > I think that illusion can be a bit dangerous when we create things or use things based on that incorrect assumption. I'd be curious to hear some of the problems programmers have run into from this conceptual discrepancy. We've got probably billions of running instances of web frameworks build atop double precision IEEE 754 to choose from. Are there any obvious examples you know?
  - tomxor 7y ago
    I can't show you, but my job involves writing a lot of numerical grading code (as in code that grades calculated student answers in a number of different ways). I've had the pleasure of seeing many other systems pretty horrible attempts at this, both from the outside and in, in both cases numerical errors rooted in floating point math abound. To give an easy example, a common numerical property required for building formatters and graders of various aspects (scientific notation, significant figures etc), is the base 10 floored order of magnitude. The most common way of obtaining this is numerically using logarithms, but this has a number of unavoidable edge cases where it fails due to floating point error - resulting in myriad of grading errors and incorrect formatting by one sf. These are an easy target to find issues that _matter_ because users are essentially fuzzing the inputs, so they are bound to find an error if it exists, and they will also care when they do! When these oversights become a problem is very context sensitive. I suppose mine is quite biased.
  - karmakaze 7y ago
    Operations that you think of being associative are not. A simple example is adding small and large numbers together. If you add the small numbers together and then the large one (e.g. sum from smallest to largest) the small parts are better represented in the sum than sum from largest to smallest. Could happen if you have a series of small interest payments and are applying it to the starting principal.
    - Demiurge 7y ago
      I've worked with large datasets, aggregating millions of numbers, summing, dividing, averaging, etc... and I have tested the orders of operations, trying to force some accumulated error, and I've actually never been able to show any difference in the realm of 6-8 significant digits I looked at.
      - tomxor 7y ago
        > in the realm of 6-8 significant digits I looked at That is far inside the range of 64bit double precision. For error to propagate up to that range of significance depends on the math, but i doubt the aggregation you are describing would cause it... provided nothing silly happens to subtotals like intermediate rounding to precision (you'd be surprised). Something like compounding as the parent was describing are far more prone to significant error propagation.
      - karmakaze 7y ago
        I've seen rounding errors on something as simple as adding the taxes calculated per line item vs calculating a tax on a subtotal. This was flagged as incorrect downstream where taxes were calculated the other way. In a real-life transaction where pennies are not exchanged this could mean a difference of a nickel on a $20 purchase which isn't a meaningful difference but certainly not insignificant.
        Demiurge 7y ago
        How much was the difference? Was there any rounding involved at any step? When dealing with money, I see rounding and integer math all the time. As another comment has mentioned, within 53 bits of mantissa, the number range is so big, we are talking 16 digits. I''d be curious to see a real-world example where the float math is the source of error, as opposed to some other bug.
  - deleted 7y ago
    [deleted]
  - Const-me 7y ago
    Web frameworks usually don't need floats at all. Developers working on other things encounter these problems all the time. https://randomascii.wordpress.com/category/floating-point/ https://randomascii.wordpress.com/category/floating-point/ https://stackoverflow.com/questions/46028336/inconsistent-behaviour-in-identical-code/46028887 https://stackoverflow.com/questions/46028336/inconsistent-be...
  - recursive 7y ago
    I spent a day debugging a problem this created. Without going into irrelevant domain details, we had a set of objects, each of which has numeric properties A and B. The formal specification says objects are categorized in a certain way iff the sum of A and B is less than or equal to 0.3. The root problem, of course, was that 0.2 + 0.1 <= 0.3 isn't actually true in floating point arithmetic. It wasn't immediately obvious where the problem was since there were a lot of moving parts in play, and it did not immediately occur to us to doubt the seemingly simple arithmetic.
- bunderbunder 7y ago
  Fair enough. One thing along similar lines that I've always hated to see is when the floating-point type in a programming language is called "real".
  - tomxor 7y ago
    haha, yes it's a terrible name, such arrogance to suggest fitting infinite things into a very finite thing. In fact they couldn't even call it rational, because even after the precision limitation it's a subset of rationals (where representation error comes from due to base)... Inigo Quilez cam up with a really interesting way around this limitation where numbers are encoded with a numerator and denominator, he called "floating bar", this essentially does not have representation error, but it will likely hit precision errors sooner or at least in a different way (it's kinda difficult to compare directly).
    - bunderbunder 7y ago
      Yeah, that is more what I'm on about. I can accept that a computer's `int` type is not an infinite set, even if it does cause problems at the boundaries. It feels like more of a little white lie to me. Whereas, even between their minimum and maximum values, and even subject to their numeric precision, there are still so many rational numbers that an IEEE float can't represent. So it's not even a rational. Nor can it represent a single irrational number, thereby failing to capture one iota of what qualitatively distinguishes the real numbers. . . if Hacker News supported gifs, I'd be inserting one that features Mandy Patinkin right here.
      - tomxor 7y ago
        Yeah, I think this is the single aspect that everyone finds unintuitive, everyone can understand it not having infinite precision or magnitude. It's probably a very reasonable design choice if we could just know the reasoning behind it, I assume it will be mostly about practicality of performance and implementing the operators that have to work with the encoding.
    - gumby 7y ago
C1sc0cat 7y ago
It can in mathematical modelling
jandrese 7y ago
Small differences can matter. One that comes to mind for me is the old game Kohan: Immortal Sovereigns. There was a native Linux port for the game that could play online with the Windows machines. However, the pathfinding solution used floating point numbers to make the decisions. A very small difference in the way Linux and Windows rounded the numbers meant that an online game between a Linux and Windows host would de-synchronize after 15-30 minutes because a model that zigged on the Linux host would zag on the Windows host, and the game would detect the difference in state after a bit and kick one of the players out. There was no way to rejoin the game at this point either. This bug was never fixed, and the company that made the port (Loki) went out of business.
cbsmith 7y ago
You'd be surprised. Decades ago I had to convert mortgage software that was using 32-bit floating point to fixed point arithmetic because they discovered that the cumulative errors from computing payments over 30+ year mortgages could lead to non-trivial inconsistencies. Don't knock the impact of cumulative loss of precision.

tomxor 7y ago

> When I format a float out to 5 decimal places, I'm sort of making a statement that anything beyond that doesn't matter to me. Yes, it's true the vast majority of the time it doesn't actually matter. However decimal formatting does such a good job at giving us the impression that these errors are merely edge-cases, and calculators automatically formatting to 10sf etc further that illusion. If people are not aware it's only an illusion (or just how far the rabbit hole goes) it can be dangerous when they go on to create or use things where that fact matters.

jancsika 7y ago
> I think that illusion can be a bit dangerous when we create things or use things based on that incorrect assumption. I'd be curious to hear some of the problems programmers have run into from this conceptual discrepancy. We've got probably billions of running instances of web frameworks build atop double precision IEEE 754 to choose from. Are there any obvious examples you know?
- tomxor 7y ago
  I can't show you, but my job involves writing a lot of numerical grading code (as in code that grades calculated student answers in a number of different ways). I've had the pleasure of seeing many other systems pretty horrible attempts at this, both from the outside and in, in both cases numerical errors rooted in floating point math abound. To give an easy example, a common numerical property required for building formatters and graders of various aspects (scientific notation, significant figures etc), is the base 10 floored order of magnitude. The most common way of obtaining this is numerically using logarithms, but this has a number of unavoidable edge cases where it fails due to floating point error - resulting in myriad of grading errors and incorrect formatting by one sf. These are an easy target to find issues that _matter_ because users are essentially fuzzing the inputs, so they are bound to find an error if it exists, and they will also care when they do! When these oversights become a problem is very context sensitive. I suppose mine is quite biased.
- karmakaze 7y ago
  Operations that you think of being associative are not. A simple example is adding small and large numbers together. If you add the small numbers together and then the large one (e.g. sum from smallest to largest) the small parts are better represented in the sum than sum from largest to smallest. Could happen if you have a series of small interest payments and are applying it to the starting principal.
  - Demiurge 7y ago
    I've worked with large datasets, aggregating millions of numbers, summing, dividing, averaging, etc... and I have tested the orders of operations, trying to force some accumulated error, and I've actually never been able to show any difference in the realm of 6-8 significant digits I looked at.
    - tomxor 7y ago
      > in the realm of 6-8 significant digits I looked at That is far inside the range of 64bit double precision. For error to propagate up to that range of significance depends on the math, but i doubt the aggregation you are describing would cause it... provided nothing silly happens to subtotals like intermediate rounding to precision (you'd be surprised). Something like compounding as the parent was describing are far more prone to significant error propagation.
    - karmakaze 7y ago
      I've seen rounding errors on something as simple as adding the taxes calculated per line item vs calculating a tax on a subtotal. This was flagged as incorrect downstream where taxes were calculated the other way. In a real-life transaction where pennies are not exchanged this could mean a difference of a nickel on a $20 purchase which isn't a meaningful difference but certainly not insignificant.
      - Demiurge 7y ago
        How much was the difference? Was there any rounding involved at any step? When dealing with money, I see rounding and integer math all the time. As another comment has mentioned, within 53 bits of mantissa, the number range is so big, we are talking 16 digits. I''d be curious to see a real-world example where the float math is the source of error, as opposed to some other bug.
- deleted 7y ago
  [deleted]
- Const-me 7y ago
  Web frameworks usually don't need floats at all. Developers working on other things encounter these problems all the time. https://randomascii.wordpress.com/category/floating-point/ https://randomascii.wordpress.com/category/floating-point/ https://stackoverflow.com/questions/46028336/inconsistent-behaviour-in-identical-code/46028887 https://stackoverflow.com/questions/46028336/inconsistent-be...
- recursive 7y ago
  I spent a day debugging a problem this created. Without going into irrelevant domain details, we had a set of objects, each of which has numeric properties A and B. The formal specification says objects are categorized in a certain way iff the sum of A and B is less than or equal to 0.3. The root problem, of course, was that 0.2 + 0.1 <= 0.3 isn't actually true in floating point arithmetic. It wasn't immediately obvious where the problem was since there were a lot of moving parts in play, and it did not immediately occur to us to doubt the seemingly simple arithmetic.
bunderbunder 7y ago
Fair enough. One thing along similar lines that I've always hated to see is when the floating-point type in a programming language is called "real".
- tomxor 7y ago
  haha, yes it's a terrible name, such arrogance to suggest fitting infinite things into a very finite thing. In fact they couldn't even call it rational, because even after the precision limitation it's a subset of rationals (where representation error comes from due to base)... Inigo Quilez cam up with a really interesting way around this limitation where numbers are encoded with a numerator and denominator, he called "floating bar", this essentially does not have representation error, but it will likely hit precision errors sooner or at least in a different way (it's kinda difficult to compare directly).
  - bunderbunder 7y ago
    Yeah, that is more what I'm on about. I can accept that a computer's `int` type is not an infinite set, even if it does cause problems at the boundaries. It feels like more of a little white lie to me. Whereas, even between their minimum and maximum values, and even subject to their numeric precision, there are still so many rational numbers that an IEEE float can't represent. So it's not even a rational. Nor can it represent a single irrational number, thereby failing to capture one iota of what qualitatively distinguishes the real numbers. . . if Hacker News supported gifs, I'd be inserting one that features Mandy Patinkin right here.
    - tomxor 7y ago
      Yeah, I think this is the single aspect that everyone finds unintuitive, everyone can understand it not having infinite precision or magnitude. It's probably a very reasonable design choice if we could just know the reasoning behind it, I assume it will be mostly about practicality of performance and implementing the operators that have to work with the encoding.
  - gumby 7y ago

C1sc0cat 7y ago

It can in mathematical modelling

jandrese 7y ago

Small differences can matter. One that comes to mind for me is the old game Kohan: Immortal Sovereigns. There was a native Linux port for the game that could play online with the Windows machines. However, the pathfinding solution used floating point numbers to make the decisions. A very small difference in the way Linux and Windows rounded the numbers meant that an online game between a Linux and Windows host would de-synchronize after 15-30 minutes because a model that zigged on the Linux host would zag on the Windows host, and the game would detect the difference in state after a bit and kick one of the players out. There was no way to rejoin the game at this point either. This bug was never fixed, and the company that made the port (Loki) went out of business.

cbsmith 7y ago

You'd be surprised. Decades ago I had to convert mortgage software that was using 32-bit floating point to fixed point arithmetic because they discovered that the cumulative errors from computing payments over 30+ year mortgages could lead to non-trivial inconsistencies. Don't knock the impact of cumulative loss of precision.