9 ms·

How to compare floats in Python

137 points by EntICOnc 5y ago

PennRobotics 5y ago
I never noticed math.isclose() and have often done the naive a - b < eps There is also (for rolling your own fn) import sys sys.float_info.epsilon although I can't immediately think of an advantage over the module.
- gotaquestion 5y ago
  Why do you consider that naive? Also, I wouldn't use epsilon, because that is a very small number and is related to representation of numbers. I would instead use a larger "tolerance" value, in `if |a-b| < tol ...` because that indicates the expected accuracy of your algorithm (unless you're really working with 1e-80 tolerance?). But TFA explains that. But back to point (a) is that not reasonable? (If not, I've got some code to change... :)
  - PennRobotics 5y ago
    Naive because it's just the easiest way. (Also, right, I forgot abs.) I imagine the downside with a chosen "epsilon" (which I used as an alias for any tolerance) is that it can be truncated on some machines, plus this equation on its own doesn't do relative accuracy. On an LEGO EV3, Python's float is 1e38 max with 7 significant digits. I've never run into problems, which is why I keep using it, too.
nabla9 5y ago
Not mentioned in the article but numpy.isclose() works for 0.0 case unlike math.isclose().
- da12 5y ago
  Could you explain what you mean the 0.0 case? math.isclose() has an abs_tol parameter for, among other things, handling comparisons to 0.
  - macintux 5y ago
    I assume “out of the box” is the distinction.
  - longemen3000 5y ago
    The Julia equivalent does a good job explaining the 0.0 case: https://docs.julialang.org/en/v1/base/math/#Base.isapprox https://docs.julialang.org/en/v1/base/math/#Base.isapprox
- PennRobotics 5y ago
  An explanation for the atol difference between numpy and math, by the author of math.isclose(): https://github.com/numpy/numpy/issues/10161#issuecomment-350021545 https://github.com/numpy/numpy/issues/10161#issuecomment-350... plus a ton of flogging/benchmarking common libraries (numpy, scipy, scikit-learn) and discussion about np.isclose() ... and also this gem: import numpy as np a = 1 b = 10 rtol = 1 atol = 0 np.isclose(a, b, rtol=rtol, atol=atol) # True np.isclose(b, a, rtol=rtol, atol=atol) # False (For the record:) import math math.isclose(a, b, rel_tol=rtol, abs_tol=atol) # True math.isclose(b, a, rel_tol=rtol, abs_tol=atol) # True
cuteboy19 5y ago
This is true for all languages, not just Python. Unfortunately C++ STL does not offer anything close to isclose (boost does however). Neither does Java. Float.compare is just == in disguise. But while going through the docs I stumbled upon this Gem > If f1 and f2 both represent Float.NaN, then the equals method returns true, even though Float.NaN==Float.NaN has the value false. https://docs.oracle.com/javase/7/docs/api/java/lang/Float.html https://docs.oracle.com/javase/7/docs/api/java/lang/Float.ht...
- saurik 5y ago
  > But while going through the docs I stumbled upon this Gem You make it sound like that's crazy or something but the goal of .compare is to be used as a "comparator" to provide an ordering of all possible values in the implementation of various data structures or sorting algorithms, while the goal of the various operators is to support programmatic logic.
  - cuteboy19 5y ago
    We need a little bit of background for this. In Java the == operator checks if two variables are pointing to the same object. The .equals() method checks for actual value equality. new String("Hi")==new String("Hi"); // False new String("Hi").equals(new String("Hi")); // True Except for primitives like ints and floats where == does value comparison. 1==1;// True For NaN the value comparison is false. Even then .equals returns true Float.NaN==Float.NaN; // False new Float(0f/0f).equals(new Float(0f/0f)); // True!!!
- mlochbaum 5y ago
  > This is true for all languages, not just Python. Not quite: APL does comparisons with a configurable relative tolerance, which is nonzero by default. My article [0] about comparing one number to many quickly in this system starts with a discussion of the reasoning and mathematics there. It brings a lot of implementation difficulty, particularly with hash tables. It's possible to build a hash on tolerant doubles by making two hashes per lookup, and even complex numbers with four per lookup, but no one's figured out how to deal with entries that contain multiple numbers yet. Nonetheless I would say it does make it easier to write working programs overall. But not by much: I've been working on an APL derivative and initially left out comparison tolerance as there were some specifics I was unsure of. A year or so later I noticed I hardly ever needed it even for numerical work and dropped any plans to add it to the language. [0] https://www.dyalog.com/blog/2018/11/tolerated-comparison-part-1/ https://www.dyalog.com/blog/2018/11/tolerated-comparison-par...
leetrout 5y ago
I abuse `Decimal` all the time in python. I usually make a little helper with a very short name like def d(in: Any) -> Decimal Then use that everywhere I expect a float (or just anywhere). Decimal's constructor is so forgiving allowing strings, ints, floats, other decimals it is so convenient to use. Of course there is the perf penalty but I always think the precision is worth the tradeoff.
- caturopath 5y ago
  Decimal has the same fundamental representation issues that float does, albeit with greater configuration.
  - bsder 5y ago
    It does, but those issues manifest themselves in ways that humans have been trained to operate. "Round to nearest even" for binary floating point is weird to anyone who doesn't have a numerics background. "Round 0.5 up" is normal to humans because that's what most have been taught. Future programming languages should probably default to Decimal Floating Point and allow people to opt-in to binary floating point on request.
    - caturopath 5y ago
      > It does, but those issues manifest themselves in ways that humans have been trained to operate. Not really. If you do computations like "convert Fahrenheit to Celsius" or "pay 6 days of interest at this APR" or a million other things, you run into the same basic faulty assumptions as ever
    - eru 5y ago
      Rounding to even makes sense for human-facing decimals as well. That's what they do in accounting, I think? At least Wikipedia tells me it's also called banker's rounding. Incidentally, it's also what Python's Decimal does by default. See https://docs.python.org/3/library/decimal.html#rounding-modes https://docs.python.org/3/library/decimal.html#rounding-mode...
- 5y ago
SloopJon 5y ago
See issue 1580 for a description of the change that displays floats using the shortestish decimal expansion that round trips: https://bugs.python.org/issue1580 https://bugs.python.org/issue1580 I thought there was a PEP for it, but evidently not. Although this is generally an improvement, it can contribute to confusion.
daenz 5y ago
Floating point numbers are a common source of issues in large open world games. Because of how a float is stored, there is more precision for values closer to the "origin" of the world (0, 0, 0). Which means there is less precision for very far away things. This can result in things like "z-fighting"[0], which I'm sure most gamers have seen, and also issues with physics. One solution to address some of the precision issues (not necessarily z-fighting, as it operates in a different coordinate system) is to dynamically re-adjust the player's world origin as they move throughout the world. This way, they are always getting the highest float precision for things near to them. 0. https://en.wikipedia.org/wiki/Z-fighting https://en.wikipedia.org/wiki/Z-fighting
- cyber_kinetist 5y ago
  I've heard that the Star Citizen engine devs had to change all of their math operations in CryEngine from single to double precision to add support for seamless large worlds (the player origin hack has limits...) Don't even want to imagine how awful of a nightmare that would have been.
  - spywaregorilla 5y ago
    I believe unreal engine is doing the same thing for their next major release to support large open worlds
- ryanianian 5y ago
  This is manifested as "far lands" in java editions of minecraft. Location away from origin gets out to precision limitations in floats resulting in jittering and other rendering oddities https://www.youtube.com/watch?v=crAa9-5tPEI https://www.youtube.com/watch?v=crAa9-5tPEI
- jbay808 5y ago
  Is there a reason that game developers don't use fixed-point math for Cartesian coordinates? A 64 bit integer and a 64 bit float both chop up your coordinate system into the same number of points, but with the integer, those points are equally spaced which is the behaviour you'd expect from a Cartesian coordinate system (based on the symmetry group of translational invariance). And even a 32-bit integer is still fine enough resolution to support four kilometres at one-micrometer resolution. With 64 bits per axis you can represent the entire solar system with 15 nm resolution, while maintaining equal resolution at any location, and exact distance calculations between any points no matter how close or how far.
Veedrac 5y ago
Either people are doing very strange things with floating point numbers or this is generally pretty poor advice. You should not generally try to ham-fistedly pretend that approximate floating point operations are producing exact results. You should not generally be looking for exact equality of floating point values except in the case where you have a justified reason to believe that a floating point computation would arrive at an exact result. You should not generally be adding approximate equality terms to your directional comparisons because all that does is switch the precise point at which you are dividing the input space, except now it is at a location that isn't consistent with the opposite test and isn't specified explicitly in the program text. You also shouldn't treat exact floating point values as if they are approximate—floating point is not approximate, some floating point operations are (conditionally) approximate. A value of exactly 1.0, say from the result of min(1.0, f), is not going to arbitrarily change to a different value. Decimal is also not a solution to floating point approximation. Decimal has all of the same rounding properties that binary floating point has, and actually it's a bit less well behaved. The differences are that Decimal round trips between textual decimal representations, and that Decimal supports an arbitrary choice of bit length. There are cases where one does need to check for convergence to arbitrary values, but these are probably not what you are doing, and if they are you should probably know more about the precise numerical guarantees that you have.
- cuteboy19 5y ago
  > Decimal has all of the same rounding properties that binary floating point has This is not true. > Unlike hardware based binary floating point, the decimal module has a user alterable precision (defaulting to 28 places) which can be as large as needed for a given problem > Both binary and decimal floating point are implemented in terms of published standards. While the built-in float type exposes only a modest portion of its capabilities, the decimal module exposes all required parts of the standard. When needed, the programmer has full control over rounding and signal handling. This includes an option to enforce exact arithmetic by using exceptions to block any inexact operations. https://docs.python.org/3/library/decimal.html https://docs.python.org/3/library/decimal.html
  - 5y ago
nealabq 5y ago
I have another problem with ==. I have some code that needs to know if two (nested) tuples of Any are equivalent. I'd like to test (tupleA == tupleB), but that doesn't work if NaN appears anywhere in tuples because (NaN != NaN). This is because == is an overloaded concept. Python has chosen to define == in the way that makes the most mathematical sense in the case of NaN. Maybe there's a metaprogramming way to override how == in interpreted for floats? with float_equality_test( math.isclose): assert (0.1 + 0.2) == 0.3 # This still allows (+0.0 == -0.0): def my_float_eq( a :float, b :float) ->bool: return (math.isnan( a) and math.isnan( b)) or (a == b) with float_equality_test( my_float_eq): assert (1, 2.0, math.nan) == (1, 2.0, math.nan)
- eru 5y ago
  If you use your own dataclasses (from dataclassy) instead of tuples, you can override this behaviour. Making NaNs compare unequal to themselves was a hack that dates back from the time modern floats were first introduced.
sacrosancty 5y ago
I'm not comfortable they recommend using an epsilon (isclose()). That's begging for mysterious bugs when the numbers are near the threshold. Better to just not compare floats for equality except when you're confident they can be exactly equal. Usually, that doesn't make sense, any more than comparing measurements of weight for equality.
deleted 5y ago
[deleted]
andrewmcwatters 5y ago
I remember having to write something similar for Lua, because there was no math.approximately() function and I was dealing with networking floats and comparing values over the wire to predicted movement in 2D space. https://github.com/Planimeter/grid-sdk/blob/master/engine/shared/mathlib.lua#L18 https://github.com/Planimeter/grid-sdk/blob/master/engine/sh... https://en.wikipedia.org/wiki/Machine_epsilon https://en.wikipedia.org/wiki/Machine_epsilon https://pubs.opengroup.org/onlinepubs/009696899/basedefs/float.h.html https://pubs.opengroup.org/onlinepubs/009696899/basedefs/flo...
petercooper 5y ago
I know the answer is "yes" because the majority tends to have a point.. but are there good reasons, beyond backwards compatibility, for languages to be using IEEE 754 floating point arithmetic nowadays rather than just storing decimals "precisely" (to a specific degree of resolution)? Or are any new languages eschewing IEEE 754 entirely? (I'm aware of BigDecimal, etc. but these still seem to be treated as a bonus feature rather than 'the way'.)
- rich_sasha 5y ago
  Floats have an enormous range, with fixed relative precision. Even a single-precision float can store numbers up to about 1.7e38 . Now of course you pay for that by losing absolute precision, but chances are that if you’re working with numbers like 1e20 you don’t much care about anything after the decimal point.
- bee_rider 5y ago
  They should have convenient hardware support (vector extensions for example). I'm sure you could design floating-point decimals that work nicely with vector extensions (which usually do cover integers), but it would be a significant project. If your language is going to plug into the numerical computing stack (BLAS+LAPACK then all the fun stuff built up on that) then it'll need to talk binary floats. All the annoying stuff that numericists understand but would like to not mess around with, like rounding directions and denormals, are handled nicely with 754 floats. These are all sort of backward compatibility/legacy issues in the sense that they are based on decisions made in the past, but the hardware and libraries aren't going anywhere I bet! Also, note that IEEE 754 does define a decimal interchange format. I bet they aren't handled as nicely in hardware, though.
  - longemen3000 5y ago
    In Julia at least, there are packages that provide alternatives to Bigfloats: Logarithmic numbers: more range, less precision https://github.com/cjdoris/LogarithmicNumbers.jl https://github.com/cjdoris/LogarithmicNumbers.jl Double arithmetic: stitch 2 floats together https://github.com/JuliaMath/DoubleFloats.jl https://github.com/JuliaMath/DoubleFloats.jl Of course, if you are calling BLAS/LAPACK, you are constrained to use floats, but the recommendation on DoubleFloats is clear: if you know you algorithms, use the increased precision only in the parts that matter
wheelerof4te 5y ago
Step one: You don't. At least, not exact numbers. You almost always have to round the numbers up, or just aproximate the value. So, while 1.2 + 2.3 != 3.5 this is always correct: 1.2 + 2.3 < 3.6
- eru 5y ago
  How do you determine the fudge factor? Eg if you compare (x + 1.2) + (x + 2.3) < (2*x + 3.6) that goes wrong for big enough x.
  - wheelerof4te 5y ago
    Use the round function in that case.
    - eru 5y ago
      How is that going to help? What precision do you want to round to? You know that at big enough values, floating point numbers are farther apart than integers?
kgm 5y ago
For another breakdown of the details of floating-point precision, I recommend qntm's "0.1 + 0.2 returns 0.30000000000000004": https://qntm.org/notpointthree https://qntm.org/notpointthree
cdavid 5y ago
This is not a very good article, and I would not recommend it beyond simple use cases. The problem is that there is no right way, it depends on the usecase and the magnitude of numbers you're comparing. See e.g. https://bitbashing.io/comparing-floats.html https://bitbashing.io/comparing-floats.html as a better referenc. The fundamental difficulty of comparing floats is that the format ensures a near constant number of digit of precision regardless of the scale. This is very useful for most calculations because it means you can calculate without worrying too much about the amplitude of your numbers. But it means that the smallest representable difference between two numbers gets bigger as numbers get bigger: that's why they are called floating point. That's why using tolerance, etc. is not so reliable: because the tolerance will depend on the magnitude of the numbers, even by doing those simple tricks. In particular, it is important to understand epsilon is only "correct" around 1, i.e. a + eps != a is only true of a is close to 1. More precisely, epsilon is the smallest number such as 1 + eps != 1.
- Tainnor 5y ago
  I mean, he's using relative tolerance, not absolute. Most common arithmetic operations should be well behaved w.r.t. relative tolerance, unless you're using denormalised numbers.
  - cdavid 5y ago
    relative tolerance is not always enough, especially near 0. This classic article gives a nice overview of absolute, relative, ulp-based and other methods for comparison of float numbers: https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ https://randomascii.wordpress.com/2012/02/25/comparing-float...
    - Tainnor 5y ago
      > especially near 0 Yes, these are denormalised numbers. They aren't as well behaved.
worik 5y ago
once again I am glad I am not a Python programmer. What is this: "math.isclose()"??? In computing floating point numbers cannot be compared for equality. It is obvious why. So "math.isclose()" is for people who cannot use "<" or ">" operators? Golly. Do any programmers need it explained to them why floating point numbers are not precise and equality does not apply? Decimal types are an abomination. The Fraction type blows my mind. Why? It is the sort of nonsense C++ programmers loved to do in the twentieth century, and we all (?) learnt from that and do not do such opaque things any more. This is all very elementary mathematics, and a language is wasting its time making easy things easy. Programmers must learn the easy things
- selcuka 5y ago
  > So "math.isclose()" is for people who cannot use "<" or ">" operators? No, it's a shortcut for `abs(a-b)<tolerance`. Is `math.sin` for people who cannot implement Taylor expansions themselves?
  - worik 5y ago
    You missed my point "math.isClose()" is making easy things easy. Not easier, a little bit harder, and it is hiding what is actually happening. Like the C++ nonsense I mentioned that did all that (and I was a sinner in those days) "math.sin(..)" makes a hard thing easy, which is a good thing. Do you see what I mean? math.colse()" increases the cognitive load for no benefit.
    - OrderlyTiamat 5y ago
      > Do you see what I mean? math.colse()" increases the cognitive load for no benefit. That seems to me to be a matter of perspective. It doesn't increase cognitive load for me (appropriate name, meaning is clear, checking the implementation, that's how I would've done it anyway, etc.) so I guess opinions differ. from your earlier comment: > Golly. Do any programmers need it explained to them why floating point numbers are not precise and equality does not apply? I think the answer is _yes_. Actually, all programmers need that explained to them. Especially python programmers, as python is used by a lot of beginning programmers. So I don't see the problem here.
opisthenar84 5y ago
This is a good introductory article, but I would not recommend it for more advanced use cases. Just because two numbers are "close" does not mean that they are "equal". If truly precise floating point math is needed, symbolic representations are a much better way to do things.
- eru 5y ago
  Python supports rational numbers just fine, too.
eyelidlessness 5y ago
> In other words, 0.1 gets converted from base 10 to base 2. This isn’t really a (helpful) explanation, but it’s ample setup for: oh I see your problem, you’re converting from base 10 to base 10.
eternityforest 5y ago
I think floats should be treated like noisy analog values. If you are doing something that requires you to reason about their precision beyond deciding how many bits is enough, you should probably not be using floats, because then you've added some more reasoning the next guy has to understand. There is a good reason to compare float equality though: when there's no calculation involved and you're basically just comparing object identity. Checking if a float timestamp has changed by comparing it to an old copy of exactly the same variable is fine. They're like analog values that can be exactly stored and copied.

137 points by EntICOnc 5y ago

I never noticed math.isclose() and have often done the naive a - b < eps There is also (for rolling your own fn) import sys sys.float_info.epsilon although I can't immediately think of an advantage over the module.

gotaquestion 5y ago
Why do you consider that naive? Also, I wouldn't use epsilon, because that is a very small number and is related to representation of numbers. I would instead use a larger "tolerance" value, in `if |a-b| < tol ...` because that indicates the expected accuracy of your algorithm (unless you're really working with 1e-80 tolerance?). But TFA explains that. But back to point (a) is that not reasonable? (If not, I've got some code to change... :)
- PennRobotics 5y ago
  Naive because it's just the easiest way. (Also, right, I forgot abs.) I imagine the downside with a chosen "epsilon" (which I used as an alias for any tolerance) is that it can be truncated on some machines, plus this equation on its own doesn't do relative accuracy. On an LEGO EV3, Python's float is 1e38 max with 7 significant digits. I've never run into problems, which is why I keep using it, too.

nabla9 5y ago

Not mentioned in the article but numpy.isclose() works for 0.0 case unlike math.isclose().

da12 5y ago
Could you explain what you mean the 0.0 case? math.isclose() has an abs_tol parameter for, among other things, handling comparisons to 0.
- macintux 5y ago
  I assume “out of the box” is the distinction.
- longemen3000 5y ago
  The Julia equivalent does a good job explaining the 0.0 case: https://docs.julialang.org/en/v1/base/math/#Base.isapprox https://docs.julialang.org/en/v1/base/math/#Base.isapprox
PennRobotics 5y ago
An explanation for the atol difference between numpy and math, by the author of math.isclose(): https://github.com/numpy/numpy/issues/10161#issuecomment-350021545 https://github.com/numpy/numpy/issues/10161#issuecomment-350... plus a ton of flogging/benchmarking common libraries (numpy, scipy, scikit-learn) and discussion about np.isclose() ... and also this gem: import numpy as np a = 1 b = 10 rtol = 1 atol = 0 np.isclose(a, b, rtol=rtol, atol=atol) # True np.isclose(b, a, rtol=rtol, atol=atol) # False (For the record:) import math math.isclose(a, b, rel_tol=rtol, abs_tol=atol) # True math.isclose(b, a, rel_tol=rtol, abs_tol=atol) # True

cuteboy19 5y ago

This is true for all languages, not just Python. Unfortunately C++ STL does not offer anything close to isclose (boost does however). Neither does Java. Float.compare is just == in disguise. But while going through the docs I stumbled upon this Gem > If f1 and f2 both represent Float.NaN, then the equals method returns true, even though Float.NaN==Float.NaN has the value false. https://docs.oracle.com/javase/7/docs/api/java/lang/Float.html https://docs.oracle.com/javase/7/docs/api/java/lang/Float.ht...

saurik 5y ago
> But while going through the docs I stumbled upon this Gem You make it sound like that's crazy or something but the goal of .compare is to be used as a "comparator" to provide an ordering of all possible values in the implementation of various data structures or sorting algorithms, while the goal of the various operators is to support programmatic logic.
- cuteboy19 5y ago
  We need a little bit of background for this. In Java the == operator checks if two variables are pointing to the same object. The .equals() method checks for actual value equality. new String("Hi")==new String("Hi"); // False new String("Hi").equals(new String("Hi")); // True Except for primitives like ints and floats where == does value comparison. 1==1;// True For NaN the value comparison is false. Even then .equals returns true Float.NaN==Float.NaN; // False new Float(0f/0f).equals(new Float(0f/0f)); // True!!!
mlochbaum 5y ago
> This is true for all languages, not just Python. Not quite: APL does comparisons with a configurable relative tolerance, which is nonzero by default. My article [0] about comparing one number to many quickly in this system starts with a discussion of the reasoning and mathematics there. It brings a lot of implementation difficulty, particularly with hash tables. It's possible to build a hash on tolerant doubles by making two hashes per lookup, and even complex numbers with four per lookup, but no one's figured out how to deal with entries that contain multiple numbers yet. Nonetheless I would say it does make it easier to write working programs overall. But not by much: I've been working on an APL derivative and initially left out comparison tolerance as there were some specifics I was unsure of. A year or so later I noticed I hardly ever needed it even for numerical work and dropped any plans to add it to the language. [0] https://www.dyalog.com/blog/2018/11/tolerated-comparison-part-1/ https://www.dyalog.com/blog/2018/11/tolerated-comparison-par...

leetrout 5y ago

I abuse `Decimal` all the time in python. I usually make a little helper with a very short name like def d(in: Any) -> Decimal Then use that everywhere I expect a float (or just anywhere). Decimal's constructor is so forgiving allowing strings, ints, floats, other decimals it is so convenient to use. Of course there is the perf penalty but I always think the precision is worth the tradeoff.

caturopath 5y ago
Decimal has the same fundamental representation issues that float does, albeit with greater configuration.
- bsder 5y ago
  It does, but those issues manifest themselves in ways that humans have been trained to operate. "Round to nearest even" for binary floating point is weird to anyone who doesn't have a numerics background. "Round 0.5 up" is normal to humans because that's what most have been taught. Future programming languages should probably default to Decimal Floating Point and allow people to opt-in to binary floating point on request.
  - caturopath 5y ago
    > It does, but those issues manifest themselves in ways that humans have been trained to operate. Not really. If you do computations like "convert Fahrenheit to Celsius" or "pay 6 days of interest at this APR" or a million other things, you run into the same basic faulty assumptions as ever
  - eru 5y ago
    Rounding to even makes sense for human-facing decimals as well. That's what they do in accounting, I think? At least Wikipedia tells me it's also called banker's rounding. Incidentally, it's also what Python's Decimal does by default. See https://docs.python.org/3/library/decimal.html#rounding-modes https://docs.python.org/3/library/decimal.html#rounding-mode...
5y ago

SloopJon 5y ago

See issue 1580 for a description of the change that displays floats using the shortestish decimal expansion that round trips: https://bugs.python.org/issue1580 https://bugs.python.org/issue1580 I thought there was a PEP for it, but evidently not. Although this is generally an improvement, it can contribute to confusion.

daenz 5y ago

Floating point numbers are a common source of issues in large open world games. Because of how a float is stored, there is more precision for values closer to the "origin" of the world (0, 0, 0). Which means there is less precision for very far away things. This can result in things like "z-fighting"[0], which I'm sure most gamers have seen, and also issues with physics. One solution to address some of the precision issues (not necessarily z-fighting, as it operates in a different coordinate system) is to dynamically re-adjust the player's world origin as they move throughout the world. This way, they are always getting the highest float precision for things near to them. 0. https://en.wikipedia.org/wiki/Z-fighting https://en.wikipedia.org/wiki/Z-fighting

cyber_kinetist 5y ago
I've heard that the Star Citizen engine devs had to change all of their math operations in CryEngine from single to double precision to add support for seamless large worlds (the player origin hack has limits...) Don't even want to imagine how awful of a nightmare that would have been.
- spywaregorilla 5y ago
  I believe unreal engine is doing the same thing for their next major release to support large open worlds
ryanianian 5y ago
This is manifested as "far lands" in java editions of minecraft. Location away from origin gets out to precision limitations in floats resulting in jittering and other rendering oddities https://www.youtube.com/watch?v=crAa9-5tPEI https://www.youtube.com/watch?v=crAa9-5tPEI
jbay808 5y ago
Is there a reason that game developers don't use fixed-point math for Cartesian coordinates? A 64 bit integer and a 64 bit float both chop up your coordinate system into the same number of points, but with the integer, those points are equally spaced which is the behaviour you'd expect from a Cartesian coordinate system (based on the symmetry group of translational invariance). And even a 32-bit integer is still fine enough resolution to support four kilometres at one-micrometer resolution. With 64 bits per axis you can represent the entire solar system with 15 nm resolution, while maintaining equal resolution at any location, and exact distance calculations between any points no matter how close or how far.

Veedrac 5y ago

Either people are doing very strange things with floating point numbers or this is generally pretty poor advice. You should not generally try to ham-fistedly pretend that approximate floating point operations are producing exact results. You should not generally be looking for exact equality of floating point values except in the case where you have a justified reason to believe that a floating point computation would arrive at an exact result. You should not generally be adding approximate equality terms to your directional comparisons because all that does is switch the precise point at which you are dividing the input space, except now it is at a location that isn't consistent with the opposite test and isn't specified explicitly in the program text. You also shouldn't treat exact floating point values as if they are approximate—floating point is not approximate, some floating point operations are (conditionally) approximate. A value of exactly 1.0, say from the result of min(1.0, f), is not going to arbitrarily change to a different value. Decimal is also not a solution to floating point approximation. Decimal has all of the same rounding properties that binary floating point has, and actually it's a bit less well behaved. The differences are that Decimal round trips between textual decimal representations, and that Decimal supports an arbitrary choice of bit length. There are cases where one does need to check for convergence to arbitrary values, but these are probably not what you are doing, and if they are you should probably know more about the precise numerical guarantees that you have.

cuteboy19 5y ago
> Decimal has all of the same rounding properties that binary floating point has This is not true. > Unlike hardware based binary floating point, the decimal module has a user alterable precision (defaulting to 28 places) which can be as large as needed for a given problem > Both binary and decimal floating point are implemented in terms of published standards. While the built-in float type exposes only a modest portion of its capabilities, the decimal module exposes all required parts of the standard. When needed, the programmer has full control over rounding and signal handling. This includes an option to enforce exact arithmetic by using exceptions to block any inexact operations. https://docs.python.org/3/library/decimal.html https://docs.python.org/3/library/decimal.html
- 5y ago

nealabq 5y ago

I have another problem with ==. I have some code that needs to know if two (nested) tuples of Any are equivalent. I'd like to test (tupleA == tupleB), but that doesn't work if NaN appears anywhere in tuples because (NaN != NaN). This is because == is an overloaded concept. Python has chosen to define == in the way that makes the most mathematical sense in the case of NaN. Maybe there's a metaprogramming way to override how == in interpreted for floats? with float_equality_test( math.isclose): assert (0.1 + 0.2) == 0.3 # This still allows (+0.0 == -0.0): def my_float_eq( a :float, b :float) ->bool: return (math.isnan( a) and math.isnan( b)) or (a == b) with float_equality_test( my_float_eq): assert (1, 2.0, math.nan) == (1, 2.0, math.nan)

eru 5y ago
If you use your own dataclasses (from dataclassy) instead of tuples, you can override this behaviour. Making NaNs compare unequal to themselves was a hack that dates back from the time modern floats were first introduced.

sacrosancty 5y ago

I'm not comfortable they recommend using an epsilon (isclose()). That's begging for mysterious bugs when the numbers are near the threshold. Better to just not compare floats for equality except when you're confident they can be exactly equal. Usually, that doesn't make sense, any more than comparing measurements of weight for equality.

deleted 5y ago

[deleted]

andrewmcwatters 5y ago

I remember having to write something similar for Lua, because there was no math.approximately() function and I was dealing with networking floats and comparing values over the wire to predicted movement in 2D space. https://github.com/Planimeter/grid-sdk/blob/master/engine/shared/mathlib.lua#L18 https://github.com/Planimeter/grid-sdk/blob/master/engine/sh... https://en.wikipedia.org/wiki/Machine_epsilon https://en.wikipedia.org/wiki/Machine_epsilon https://pubs.opengroup.org/onlinepubs/009696899/basedefs/float.h.html https://pubs.opengroup.org/onlinepubs/009696899/basedefs/flo...

petercooper 5y ago

I know the answer is "yes" because the majority tends to have a point.. but are there good reasons, beyond backwards compatibility, for languages to be using IEEE 754 floating point arithmetic nowadays rather than just storing decimals "precisely" (to a specific degree of resolution)? Or are any new languages eschewing IEEE 754 entirely? (I'm aware of BigDecimal, etc. but these still seem to be treated as a bonus feature rather than 'the way'.)

rich_sasha 5y ago
Floats have an enormous range, with fixed relative precision. Even a single-precision float can store numbers up to about 1.7e38 . Now of course you pay for that by losing absolute precision, but chances are that if you’re working with numbers like 1e20 you don’t much care about anything after the decimal point.
bee_rider 5y ago
They should have convenient hardware support (vector extensions for example). I'm sure you could design floating-point decimals that work nicely with vector extensions (which usually do cover integers), but it would be a significant project. If your language is going to plug into the numerical computing stack (BLAS+LAPACK then all the fun stuff built up on that) then it'll need to talk binary floats. All the annoying stuff that numericists understand but would like to not mess around with, like rounding directions and denormals, are handled nicely with 754 floats. These are all sort of backward compatibility/legacy issues in the sense that they are based on decisions made in the past, but the hardware and libraries aren't going anywhere I bet! Also, note that IEEE 754 does define a decimal interchange format. I bet they aren't handled as nicely in hardware, though.
- longemen3000 5y ago
  In Julia at least, there are packages that provide alternatives to Bigfloats: Logarithmic numbers: more range, less precision https://github.com/cjdoris/LogarithmicNumbers.jl https://github.com/cjdoris/LogarithmicNumbers.jl Double arithmetic: stitch 2 floats together https://github.com/JuliaMath/DoubleFloats.jl https://github.com/JuliaMath/DoubleFloats.jl Of course, if you are calling BLAS/LAPACK, you are constrained to use floats, but the recommendation on DoubleFloats is clear: if you know you algorithms, use the increased precision only in the parts that matter

wheelerof4te 5y ago

Step one: You don't. At least, not exact numbers. You almost always have to round the numbers up, or just aproximate the value. So, while 1.2 + 2.3 != 3.5 this is always correct: 1.2 + 2.3 < 3.6

eru 5y ago
How do you determine the fudge factor? Eg if you compare (x + 1.2) + (x + 2.3) < (2*x + 3.6) that goes wrong for big enough x.
- wheelerof4te 5y ago
  Use the round function in that case.
  - eru 5y ago
    How is that going to help? What precision do you want to round to? You know that at big enough values, floating point numbers are farther apart than integers?

kgm 5y ago

For another breakdown of the details of floating-point precision, I recommend qntm's "0.1 + 0.2 returns 0.30000000000000004": https://qntm.org/notpointthree https://qntm.org/notpointthree

cdavid 5y ago

This is not a very good article, and I would not recommend it beyond simple use cases. The problem is that there is no right way, it depends on the usecase and the magnitude of numbers you're comparing. See e.g. https://bitbashing.io/comparing-floats.html https://bitbashing.io/comparing-floats.html as a better referenc. The fundamental difficulty of comparing floats is that the format ensures a near constant number of digit of precision regardless of the scale. This is very useful for most calculations because it means you can calculate without worrying too much about the amplitude of your numbers. But it means that the smallest representable difference between two numbers gets bigger as numbers get bigger: that's why they are called floating point. That's why using tolerance, etc. is not so reliable: because the tolerance will depend on the magnitude of the numbers, even by doing those simple tricks. In particular, it is important to understand epsilon is only "correct" around 1, i.e. a + eps != a is only true of a is close to 1. More precisely, epsilon is the smallest number such as 1 + eps != 1.

Tainnor 5y ago
I mean, he's using relative tolerance, not absolute. Most common arithmetic operations should be well behaved w.r.t. relative tolerance, unless you're using denormalised numbers.
- cdavid 5y ago
  relative tolerance is not always enough, especially near 0. This classic article gives a nice overview of absolute, relative, ulp-based and other methods for comparison of float numbers: https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ https://randomascii.wordpress.com/2012/02/25/comparing-float...
  - Tainnor 5y ago
    > especially near 0 Yes, these are denormalised numbers. They aren't as well behaved.

worik 5y ago

once again I am glad I am not a Python programmer. What is this: "math.isclose()"??? In computing floating point numbers cannot be compared for equality. It is obvious why. So "math.isclose()" is for people who cannot use "<" or ">" operators? Golly. Do any programmers need it explained to them why floating point numbers are not precise and equality does not apply? Decimal types are an abomination. The Fraction type blows my mind. Why? It is the sort of nonsense C++ programmers loved to do in the twentieth century, and we all (?) learnt from that and do not do such opaque things any more. This is all very elementary mathematics, and a language is wasting its time making easy things easy. Programmers must learn the easy things

selcuka 5y ago
> So "math.isclose()" is for people who cannot use "<" or ">" operators? No, it's a shortcut for `abs(a-b)<tolerance`. Is `math.sin` for people who cannot implement Taylor expansions themselves?
- worik 5y ago
  You missed my point "math.isClose()" is making easy things easy. Not easier, a little bit harder, and it is hiding what is actually happening. Like the C++ nonsense I mentioned that did all that (and I was a sinner in those days) "math.sin(..)" makes a hard thing easy, which is a good thing. Do you see what I mean? math.colse()" increases the cognitive load for no benefit.
  - OrderlyTiamat 5y ago
    > Do you see what I mean? math.colse()" increases the cognitive load for no benefit. That seems to me to be a matter of perspective. It doesn't increase cognitive load for me (appropriate name, meaning is clear, checking the implementation, that's how I would've done it anyway, etc.) so I guess opinions differ. from your earlier comment: > Golly. Do any programmers need it explained to them why floating point numbers are not precise and equality does not apply? I think the answer is _yes_. Actually, all programmers need that explained to them. Especially python programmers, as python is used by a lot of beginning programmers. So I don't see the problem here.

opisthenar84 5y ago

This is a good introductory article, but I would not recommend it for more advanced use cases. Just because two numbers are "close" does not mean that they are "equal". If truly precise floating point math is needed, symbolic representations are a much better way to do things.

eru 5y ago
Python supports rational numbers just fine, too.

eyelidlessness 5y ago

> In other words, 0.1 gets converted from base 10 to base 2. This isn’t really a (helpful) explanation, but it’s ample setup for: oh I see your problem, you’re converting from base 10 to base 10.

eternityforest 5y ago

I think floats should be treated like noisy analog values. If you are doing something that requires you to reason about their precision beyond deciding how many bits is enough, you should probably not be using floats, because then you've added some more reasoning the next guy has to understand. There is a good reason to compare float equality though: when there's no calculation involved and you're basically just comparing object identity. Checking if a float timestamp has changed by comparing it to an old copy of exactly the same variable is fine. They're like analog values that can be exactly stored and copied.