4 ms·

> if you find yourself comparing floating point numbers, that's probably not what you want Can you motivate this with an example for me? For instance, I think

by nawgz 3y ago

> if you find yourself comparing floating point numbers, that's probably not what you want Can you motivate this with an example for me? For instance, I think of games or location data encoded as FP; then, clearly, comparing them is a critical task to know questions like "what is closer" and so on. What am I missing?

stefncb 3y ago
They meant using the equal operator, because floating point is inexact and can produce different representations for the same number depending on how it was obtained. Greater and less than are fine. Equal is usually implemented by seeing if a number fits inside a tight range.
- recursive 3y ago
  Every finite floating point value precisely represents an exact quantity. Arithmetic isn't lossless though. This isn't a special property of floats. Decimals behave the same way.
  - nawgz 3y ago
    I was with you until the end First point: FPs are unique. Clearly true Second point: Combing FPs isn't lossless, across most/all arithmetic. Clearly true, the root of our discussion Third point: decimal arithmetic is lossy too. Strongly disagree. The system of representation is what is lossy - floating point. Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers.
    - Dylan16807 3y ago
      Decimal is only lossless if you track an indefinitely-growing suffix to your number with a big bar over it to indicate "repeating". It's basically working with rational numbers but more painful. Decimal numbers with any limit on digits, even a hundred, are lossy. Decimal numbers as used by humans outside of computers have limits on digits, so they are lossy.
      - tadfisher 3y ago
        Likewise, bit-limited integers are lossy as they are incomplete representations of ℤ, and are not reciprocal with division. The point being that, just like floating-point representations of ℝ, division will always be lossy in the general case unless you choose a symbolic algebra instead of numeric.
      - nawgz 3y ago
        Maybe my point is uninteresting, but I feel you haven't really understood it. Combining two FP numbers with arithmetic leads to possible errors that you can't represent with FP on every operation. Combining two decimal numbers with non-division arithmetic never leads to that case. With division, sure, things are bad, but that's more of an exception than a rule to me. This is because decimal numbers don't really come with any inherent limit on digits, and it's a bit strange to add a clause to the other's claims before making a counterargument.
        recursive 3y ago
        > Combining two decimal numbers with non-division arithmetic never leads to that case. Most types are subject to overflow that has similar effects. Most of the FP error people encounter is actually division error. For example, the constant 0.1 contains an implicit division. The "tenths" place is defined by division by 10. I think that almost all perceptions of floating point lossiness come from this fact.
        Dylan16807 3y ago
        Well in part I didn't understand your point because you didn't mention division as an exception. But even then, multiplication causes an explosion in number length if you do it repeatedly. When you specifically talk about numbers not being in computers, I think it's fair to talk about digit limits. Most real-world use of decimals is done with less precision than the 16 digits we default to in computers. Let alone growing to 50, 100, 200, etc as you keep multiplying numbers together to perform some kind of analysis. Nobody uses decimal like that. Real-world decimal is lossy for a large swath of multiplication. I agree that if you're doing something like just adding numbers repeatedly in decimal, and those numbers have no repeating digits, then you have a nice purity of never losing precision. That's worth something. But on the other hand if you started with the same kind of numbers in floating point, let's say about 9 digits long, you could still add a million of them without losing precision. And nobody has said anything about irrational numbers as you dismissed in your other comment. So in summary: decimal division, usually lossy; decimal multiplication, usually lossy; decimal addition and subtraction, lossless but with the same kind of source numbers FP is usually lossless too
    - recursive 3y ago
      Write 1/3 in decimal. You're required to do it in finite memory.
    - kragen 3y ago
      you seem to be confusing the decimal system of representation with platonic abstract numbers, which is what gives rise to the apparent disagreement
      - nawgz 3y ago
        Do I? I think people are speaking shockingly imprecisely in this thread. here's my claim: > Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers Counterarguments are basically on the level of "you cant represent Pi as a finite decimal number", which I find uninteresting.
        kragen 3y ago
        that's slightly incoherent and reflective of the same confusion i identified; there is strictly speaking, no such thing as a decimal number, only a decimal numeral. normally the difference is subtle, but in this case it's the essence of the issue 'decimal' means 'base 10' numbers, the abstract entities we do arithmetic on, are not decimal or in any other base; it is correct that arithmetic on them is not in any sense lossy decimal is a system of representation that represents these abstract numbers as finite strings of digits; these are called 'numerals'. it can represent any integer, but only some fractions, those whose denominator has no prime factors other than 2 or 5. such fractions arise non-lossily as the result of the arithmetic operator of division when, for example, the dividend is 3. representing these in decimal requires rounding them, so decimal is lossy binary floating point is lossy in the same way, with the additional limitation to only being able to represent fractions whose denominator is a power of 2, whose numerator does not have too many significant bits (53 most often), and which are not too positive, too negative, or too small there are other systems of represention for numbers that are not lossy in this way. for example, improper fractions, mixed numbers, finite continued fractions, and decimal augmented with overbar to indicate repeated digits
        kragen 3y ago
        this should read 'when, for example, the divisor is 3 and the dividend is an integer not divisible by 3'
zokier 3y ago
Common way of comparing floats is to compare the difference to some epsilon value. See Python PEP-485 for example https://peps.python.org/pep-0485/ https://peps.python.org/pep-0485/
deleted 3y ago
[deleted]
orthoxerox 3y ago
This is specifically about equality. Comparing FPs for equality is very risky, as your numbers can differ by 0.00000000000001 without anyone noticing. Strict inequality (> and <) comparison are generally fine as long as you avoid NaNs.
- nawgz 3y ago
  Ok, sure, so it was just for a specific case of comparison. Well, followup, if some error Epsilon can be introduced during any manipulations, when you check for strict inequalities do you check x < y + Epsilon? Does the language implicitly do it for you?
  - klodolph 3y ago
    The language just gives you the direct comparison. If you know numerics, then you can come up with the correct value for epsilon. But that is hard work. There are various things that the language could do for you, like use a dynamic amount of precision, or interval arithmetic where the error bounds are saved—but! Most of the time people just want the answer faster and with less memory used, which is what you get with bare floats. The people who make it their job to care about numerical accuracy can do it better than the language runtime would anyway. Most of the problems are more easily solvable at a higher level anyway. Like, imagine Clippy saying to you, “It looks like you’re inverting a matrix. Are you sure that’s a good idea?”
HexDecOctBin 3y ago
Ideally, you'll want to do distance calculations in fixed point numbers (of desired resolution). Floating point works well as a first approximation, but unless you can make sure that you are not going to end up wandering in the weeds (and ensuring that required understanding of computational numerics), you should probably replace them with fixed point once you understand the problem and the solution and know the limits involved.
zdragnar 3y ago
Caveat: In the following, I use "comparison" to mean "check for equality". Floating point numbers lose precision because binary arithmetic doesn't represent decimal in all cases ( 1/3 is an easy example). It's not hard to get into a situation where you're asking if a number is 0.0 but due to a precision error the number you have is 0.00000000000001 or whatever and it should have been zero. If you're dealing with anything where precision is paramount (i.e. money or high precision machining) you should consider using something else that matches the real work precision you are trying to model. You can, of course, use floats and test differences rather than strict equality (i.e. distance is < 0.01) and that's fine too... if you remember to do it, and account for the potential for small precision differences to propagate throughout your calculations and accumulate into larger errors.
- tobr 3y ago
  > Caveat: In the following, I use "comparison" to mean "check for equality". You never use the word “comparison” again in the comment!
- Someone 3y ago
  > Floating point numbers lose precision because binary arithmetic doesn't represent decimal in all cases. That’s not the reason. A counterexample is decimal arithmetic. That represents decimal in all cases, yet loses precision when doing calculations. Once you start doing division that applies even to arbitrarily sized decimal numbers. The correct reason is because the subset of the reals that is representable as IEEE float isn’t closed under the operations people want to perform on them, including the basic ones of addition, multiplication and division.
jandrese 3y ago
IMHO That was poorly worded. What it should have said is "if you find yourself using equals to compare floating point numbers...". With Floating Point your comparisons should always be less than or greater than. Precision artifacts make the equals unreliable, and you should always be mindful of that when dealing with them.
BlueTemplar 3y ago
It's the hard task of trying to figure out the magnitude of the expected errors (which can accumulate), in the simplest case of a single operation you compare within epsilon distance : https://en.wikibooks.org/wiki/Floating_Point/Epsilon https://en.wikibooks.org/wiki/Floating_Point/Epsilon

by nawgz 3y ago

stefncb 3y ago

They meant using the equal operator, because floating point is inexact and can produce different representations for the same number depending on how it was obtained. Greater and less than are fine. Equal is usually implemented by seeing if a number fits inside a tight range.

recursive 3y ago
Every finite floating point value precisely represents an exact quantity. Arithmetic isn't lossless though. This isn't a special property of floats. Decimals behave the same way.
- nawgz 3y ago
  I was with you until the end First point: FPs are unique. Clearly true Second point: Combing FPs isn't lossless, across most/all arithmetic. Clearly true, the root of our discussion Third point: decimal arithmetic is lossy too. Strongly disagree. The system of representation is what is lossy - floating point. Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers.
  - Dylan16807 3y ago
    Decimal is only lossless if you track an indefinitely-growing suffix to your number with a big bar over it to indicate "repeating". It's basically working with rational numbers but more painful. Decimal numbers with any limit on digits, even a hundred, are lossy. Decimal numbers as used by humans outside of computers have limits on digits, so they are lossy.
    - tadfisher 3y ago
      Likewise, bit-limited integers are lossy as they are incomplete representations of ℤ, and are not reciprocal with division. The point being that, just like floating-point representations of ℝ, division will always be lossy in the general case unless you choose a symbolic algebra instead of numeric.
    - nawgz 3y ago
      Maybe my point is uninteresting, but I feel you haven't really understood it. Combining two FP numbers with arithmetic leads to possible errors that you can't represent with FP on every operation. Combining two decimal numbers with non-division arithmetic never leads to that case. With division, sure, things are bad, but that's more of an exception than a rule to me. This is because decimal numbers don't really come with any inherent limit on digits, and it's a bit strange to add a clause to the other's claims before making a counterargument.
      - recursive 3y ago
        > Combining two decimal numbers with non-division arithmetic never leads to that case. Most types are subject to overflow that has similar effects. Most of the FP error people encounter is actually division error. For example, the constant 0.1 contains an implicit division. The "tenths" place is defined by division by 10. I think that almost all perceptions of floating point lossiness come from this fact.
      - Dylan16807 3y ago
        Well in part I didn't understand your point because you didn't mention division as an exception. But even then, multiplication causes an explosion in number length if you do it repeatedly. When you specifically talk about numbers not being in computers, I think it's fair to talk about digit limits. Most real-world use of decimals is done with less precision than the 16 digits we default to in computers. Let alone growing to 50, 100, 200, etc as you keep multiplying numbers together to perform some kind of analysis. Nobody uses decimal like that. Real-world decimal is lossy for a large swath of multiplication. I agree that if you're doing something like just adding numbers repeatedly in decimal, and those numbers have no repeating digits, then you have a nice purity of never losing precision. That's worth something. But on the other hand if you started with the same kind of numbers in floating point, let's say about 9 digits long, you could still add a million of them without losing precision. And nobody has said anything about irrational numbers as you dismissed in your other comment. So in summary: decimal division, usually lossy; decimal multiplication, usually lossy; decimal addition and subtraction, lossless but with the same kind of source numbers FP is usually lossless too
  - recursive 3y ago
    Write 1/3 in decimal. You're required to do it in finite memory.
  - kragen 3y ago
    you seem to be confusing the decimal system of representation with platonic abstract numbers, which is what gives rise to the apparent disagreement
    - nawgz 3y ago
      Do I? I think people are speaking shockingly imprecisely in this thread. here's my claim: > Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers Counterarguments are basically on the level of "you cant represent Pi as a finite decimal number", which I find uninteresting.
      - kragen 3y ago
        that's slightly incoherent and reflective of the same confusion i identified; there is strictly speaking, no such thing as a decimal number, only a decimal numeral. normally the difference is subtle, but in this case it's the essence of the issue 'decimal' means 'base 10' numbers, the abstract entities we do arithmetic on, are not decimal or in any other base; it is correct that arithmetic on them is not in any sense lossy decimal is a system of representation that represents these abstract numbers as finite strings of digits; these are called 'numerals'. it can represent any integer, but only some fractions, those whose denominator has no prime factors other than 2 or 5. such fractions arise non-lossily as the result of the arithmetic operator of division when, for example, the dividend is 3. representing these in decimal requires rounding them, so decimal is lossy binary floating point is lossy in the same way, with the additional limitation to only being able to represent fractions whose denominator is a power of 2, whose numerator does not have too many significant bits (53 most often), and which are not too positive, too negative, or too small there are other systems of represention for numbers that are not lossy in this way. for example, improper fractions, mixed numbers, finite continued fractions, and decimal augmented with overbar to indicate repeated digits
        kragen 3y ago
        this should read 'when, for example, the divisor is 3 and the dividend is an integer not divisible by 3'

zokier 3y ago

Common way of comparing floats is to compare the difference to some epsilon value. See Python PEP-485 for example https://peps.python.org/pep-0485/ https://peps.python.org/pep-0485/

deleted 3y ago

[deleted]

orthoxerox 3y ago

This is specifically about equality. Comparing FPs for equality is very risky, as your numbers can differ by 0.00000000000001 without anyone noticing. Strict inequality (> and <) comparison are generally fine as long as you avoid NaNs.

nawgz 3y ago
Ok, sure, so it was just for a specific case of comparison. Well, followup, if some error Epsilon can be introduced during any manipulations, when you check for strict inequalities do you check x < y + Epsilon? Does the language implicitly do it for you?
- klodolph 3y ago
  The language just gives you the direct comparison. If you know numerics, then you can come up with the correct value for epsilon. But that is hard work. There are various things that the language could do for you, like use a dynamic amount of precision, or interval arithmetic where the error bounds are saved—but! Most of the time people just want the answer faster and with less memory used, which is what you get with bare floats. The people who make it their job to care about numerical accuracy can do it better than the language runtime would anyway. Most of the problems are more easily solvable at a higher level anyway. Like, imagine Clippy saying to you, “It looks like you’re inverting a matrix. Are you sure that’s a good idea?”

HexDecOctBin 3y ago

Ideally, you'll want to do distance calculations in fixed point numbers (of desired resolution). Floating point works well as a first approximation, but unless you can make sure that you are not going to end up wandering in the weeds (and ensuring that required understanding of computational numerics), you should probably replace them with fixed point once you understand the problem and the solution and know the limits involved.

zdragnar 3y ago

Caveat: In the following, I use "comparison" to mean "check for equality". Floating point numbers lose precision because binary arithmetic doesn't represent decimal in all cases ( 1/3 is an easy example). It's not hard to get into a situation where you're asking if a number is 0.0 but due to a precision error the number you have is 0.00000000000001 or whatever and it should have been zero. If you're dealing with anything where precision is paramount (i.e. money or high precision machining) you should consider using something else that matches the real work precision you are trying to model. You can, of course, use floats and test differences rather than strict equality (i.e. distance is < 0.01) and that's fine too... if you remember to do it, and account for the potential for small precision differences to propagate throughout your calculations and accumulate into larger errors.

tobr 3y ago
> Caveat: In the following, I use "comparison" to mean "check for equality". You never use the word “comparison” again in the comment!
Someone 3y ago
> Floating point numbers lose precision because binary arithmetic doesn't represent decimal in all cases. That’s not the reason. A counterexample is decimal arithmetic. That represents decimal in all cases, yet loses precision when doing calculations. Once you start doing division that applies even to arbitrarily sized decimal numbers. The correct reason is because the subset of the reals that is representable as IEEE float isn’t closed under the operations people want to perform on them, including the basic ones of addition, multiplication and division.

jandrese 3y ago

IMHO That was poorly worded. What it should have said is "if you find yourself using equals to compare floating point numbers...". With Floating Point your comparisons should always be less than or greater than. Precision artifacts make the equals unreliable, and you should always be mindful of that when dealing with them.

BlueTemplar 3y ago

It's the hard task of trying to figure out the magnitude of the expected errors (which can accumulate), in the simplest case of a single operation you compare within epsilon distance : https://en.wikibooks.org/wiki/Floating_Point/Epsilon https://en.wikibooks.org/wiki/Floating_Point/Epsilon