6 ms·

Pitfalls of comparing BigDecimals in Java

53 points by Igor_Wiwi 2y ago

agluszak 2y ago
Interesting. I'd say that it's one of the places where using a non-trivial, non-structural comparison method would make sense, as opposed to the infamous URL equals method.
dgfitz 2y ago
I use bigInt and bigDec when I wrote a pokemonGo bot way back when the cracked api released in like… 2016? I didn’t bump into the particular issue, but not being much of a Java person, they were very helpful. My bot was pretty consistent at getting lvl30 in a night and usually something fun like a snorlax or charizard. When they broke the api I stopped playing, writing the code for a bit was more fun (for me) than playing as intended. As an aside, are there games like that, which require writing code? I imagine the market isn’t big for those, but I sure had a blast for a month or two.
- gleenn 2y ago
  Out of curiosity, why did that game require BigDecimals?
  - dgfitz 2y ago
    I’ll hafta see if I have the code somewhere. IIRC it had something to do with lat/lon, calculating the next “step in the walk” and making it appear human in nature.
- bongodongobob 2y ago
  Spacetraders. I've been meaning to get into it but just haven't found the time. Looks really fun.
- foobazgt 2y ago
  Zachtronics (https://www.zachtronics.com/ https://www.zachtronics.com/) is amazing, but their games are lower level programming. You're typically designing hardware or writing assembly. I haven't played them all, but I can vouch for last call BBS (design hardware), exapunks (software controlling bots), shenzen i/o (you design hardware and write the assembly to power it), and tis-100 (writing assembly for a distributed, cluster of processors). It sounds a lot more intimidating than reality. The puzzles start off easy enough that you can ramp into the difficulty., and they usually have many different possible solutions. You can "win" with easy solutions, and then later revisit them and try to produce the "best" solution. (Often pretty difficult). To be fair, I did come at them from somewhat of a background in hardware, but my son also loved them (with no such background).
auntienomen 2y ago
This seems like incredibly bad design. Is there any justification or use case where someone would want the abstraction to leak like this?
- cowanon55 2y ago
  It is likely because equals is usually “object equals”, not necessarily “math equals”. Since the internal representation is different, the objects are indeed different. Tricky though, and most 3rd party code would likely choose to make equals true in this kind of situation. Java has tons of examples where the core behavior is logically correct, but VERY unintuitive - I feel the pain daily.
  - ktm5j 2y ago
    I don't think that flies, because the whole point of the .equals() method in regards to string is to avoid "object equals" and make sure the strings say the same thing..
- Tao3300 2y ago
  It's supposed to be a mess. We have very little job security anymore.
- erik_seaberg 2y ago
  If one mass is 1.0 kg ± 40 g, and another mass is 1.000 kg ± 0.4 g, are they close enough to equal? Maybe, and if you round them to the same precision you get whichever answer you wanted.
- jjmarr 2y ago
  https://en.wikipedia.org/wiki/Significant_figures https://en.wikipedia.org/wiki/Significant_figures Trailing zeroes convey information about the measurement precision of a number. If you're doing math with real-world measurements, errors accumulate, and you're expected to keep track of them.
  - auntienomen 2y ago
gleenn 2y ago
Kinda makes me think that implementation is probably not very performant or optimized for the obvious case. A big pet peeve of mine is when languages do weird and very unintuitive things with equality. Like in Javascript and PHP you end up needing these crazy charts to determine if something == something else. Personally I think Clojure gets this really well and has some of the fewest surprises.
- csjh 2y ago
  No need to remember crazy charts to determine equality in Javascript, just blanketly use `===`
  - RodgerTheGreat 2y ago
    JavaScript == comparison isn't particularly strange; the "crazy charts" are mostly cases that you should never encounter if you're writing reasonable code. If you're going to use a language, it's a much better idea to familiarize yourself with its actual semantics than to pepper your codebase with superstitious rituals.
happytoexplain 2y ago
Why would the BigDecimal implementation of 'equals' not use semantic (numeric, in this case) equality? Is 'equals' always meant to be memberwise (or perhaps makes performance guarantees), and if so, shouldn't there be another function/operator that is always meant to be semantic? The article doesn't acknowledge these questions.
- lmm 2y ago
  > Why would the BigDecimal implementation of 'equals' not use semantic (numeric, in this case) equality? A BigDecimal with different precision is not really semantically equivalent - it will lead to different results when you use it in calculations.
  - happytoexplain 2y ago
    Oh really??? That would make sense, though it raises more questions with perhaps not so brief answers.
    - lmm 2y ago
      3 / 2 = 1, 3.0 / 2 = 1.5. So 3.equals(3.0) is false, as this "equals" is meant to have programming semantics rather than mathematical ones.
      - happytoexplain 2y ago
        It looks like you're comparing integer arithmetic and float arithmetic. Are you saying BigDecimal can represent both??
        lmm 2y ago
        That's a special case of decimal precision. 1/3 = 0, 1.0/3=0.3, 1.00/3=0.33 is probably a better example. (where by 1.0 I mean a BigDecimal with precision=2 - in fact the default is 40)
vitus 2y ago
Even more pitfalls result from this behavior: > Care should be exercised if BigDecimal objects are used as keys in a SortedMap or elements in a SortedSet since BigDecimal's natural ordering is inconsistent with equals. See Comparable, SortedMap or SortedSet for more information. https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi%2F%2F/java.base/java/math/BigDecimal.html https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi... "Consistency with equals" specifically means a.compareTo(b) == 0 is equivalent to a.equals(b). Java docs explicitly state that this is "strongly recommended (though not required)". https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi%2F%2F/java.base/java/lang/Comparable.html https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi...
doubloon 2y ago
whats funny is the divide between math theory in the 20th century and automated arithmetic. math world - "wouldn't it be great if we had a thing called a field, or even better a group, a bunch of rules everyone agrees on where operations are closed and you can make all these great discoveries and there is so much consistency and predictability..." computer world - "yes but wouldn't it also be great if we could add binary fractions with rounding...."
vitiral 2y ago
I feel like if they had simply named it FixedDecimal they would have avoided the confusion. It makes more sense that 1.0 differs from 1.00 as they are different precicions. As another person pointed out, dividing them by 3 will yield different results.
- definitelyauser 2y ago
  BigFixedDecimal? Sounds like yet another java naming pitfall.
klysm 2y ago
This seems like a massive API design mistake
- Viliam1234 2y ago
  What does? The different numbers being different, or some other part?

53 points by Igor_Wiwi 2y ago

Interesting. I'd say that it's one of the places where using a non-trivial, non-structural comparison method would make sense, as opposed to the infamous URL equals method.

dgfitz 2y ago

I use bigInt and bigDec when I wrote a pokemonGo bot way back when the cracked api released in like… 2016? I didn’t bump into the particular issue, but not being much of a Java person, they were very helpful. My bot was pretty consistent at getting lvl30 in a night and usually something fun like a snorlax or charizard. When they broke the api I stopped playing, writing the code for a bit was more fun (for me) than playing as intended. As an aside, are there games like that, which require writing code? I imagine the market isn’t big for those, but I sure had a blast for a month or two.

gleenn 2y ago
Out of curiosity, why did that game require BigDecimals?
- dgfitz 2y ago
  I’ll hafta see if I have the code somewhere. IIRC it had something to do with lat/lon, calculating the next “step in the walk” and making it appear human in nature.
bongodongobob 2y ago
Spacetraders. I've been meaning to get into it but just haven't found the time. Looks really fun.
foobazgt 2y ago
Zachtronics (https://www.zachtronics.com/ https://www.zachtronics.com/) is amazing, but their games are lower level programming. You're typically designing hardware or writing assembly. I haven't played them all, but I can vouch for last call BBS (design hardware), exapunks (software controlling bots), shenzen i/o (you design hardware and write the assembly to power it), and tis-100 (writing assembly for a distributed, cluster of processors). It sounds a lot more intimidating than reality. The puzzles start off easy enough that you can ramp into the difficulty., and they usually have many different possible solutions. You can "win" with easy solutions, and then later revisit them and try to produce the "best" solution. (Often pretty difficult). To be fair, I did come at them from somewhat of a background in hardware, but my son also loved them (with no such background).

auntienomen 2y ago

This seems like incredibly bad design. Is there any justification or use case where someone would want the abstraction to leak like this?

cowanon55 2y ago
It is likely because equals is usually “object equals”, not necessarily “math equals”. Since the internal representation is different, the objects are indeed different. Tricky though, and most 3rd party code would likely choose to make equals true in this kind of situation. Java has tons of examples where the core behavior is logically correct, but VERY unintuitive - I feel the pain daily.
- ktm5j 2y ago
  I don't think that flies, because the whole point of the .equals() method in regards to string is to avoid "object equals" and make sure the strings say the same thing..
Tao3300 2y ago
It's supposed to be a mess. We have very little job security anymore.
erik_seaberg 2y ago
If one mass is 1.0 kg ± 40 g, and another mass is 1.000 kg ± 0.4 g, are they close enough to equal? Maybe, and if you round them to the same precision you get whichever answer you wanted.
jjmarr 2y ago
https://en.wikipedia.org/wiki/Significant_figures https://en.wikipedia.org/wiki/Significant_figures Trailing zeroes convey information about the measurement precision of a number. If you're doing math with real-world measurements, errors accumulate, and you're expected to keep track of them.
- auntienomen 2y ago

gleenn 2y ago

Kinda makes me think that implementation is probably not very performant or optimized for the obvious case. A big pet peeve of mine is when languages do weird and very unintuitive things with equality. Like in Javascript and PHP you end up needing these crazy charts to determine if something == something else. Personally I think Clojure gets this really well and has some of the fewest surprises.

csjh 2y ago
No need to remember crazy charts to determine equality in Javascript, just blanketly use `===`
- RodgerTheGreat 2y ago
  JavaScript == comparison isn't particularly strange; the "crazy charts" are mostly cases that you should never encounter if you're writing reasonable code. If you're going to use a language, it's a much better idea to familiarize yourself with its actual semantics than to pepper your codebase with superstitious rituals.

happytoexplain 2y ago

Why would the BigDecimal implementation of 'equals' not use semantic (numeric, in this case) equality? Is 'equals' always meant to be memberwise (or perhaps makes performance guarantees), and if so, shouldn't there be another function/operator that is always meant to be semantic? The article doesn't acknowledge these questions.

lmm 2y ago
> Why would the BigDecimal implementation of 'equals' not use semantic (numeric, in this case) equality? A BigDecimal with different precision is not really semantically equivalent - it will lead to different results when you use it in calculations.
- happytoexplain 2y ago
  Oh really??? That would make sense, though it raises more questions with perhaps not so brief answers.
  - lmm 2y ago
    3 / 2 = 1, 3.0 / 2 = 1.5. So 3.equals(3.0) is false, as this "equals" is meant to have programming semantics rather than mathematical ones.
    - happytoexplain 2y ago
      It looks like you're comparing integer arithmetic and float arithmetic. Are you saying BigDecimal can represent both??
      - lmm 2y ago
        That's a special case of decimal precision. 1/3 = 0, 1.0/3=0.3, 1.00/3=0.33 is probably a better example. (where by 1.0 I mean a BigDecimal with precision=2 - in fact the default is 40)

vitus 2y ago

Even more pitfalls result from this behavior: > Care should be exercised if BigDecimal objects are used as keys in a SortedMap or elements in a SortedSet since BigDecimal's natural ordering is inconsistent with equals. See Comparable, SortedMap or SortedSet for more information. https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi%2F%2F/java.base/java/math/BigDecimal.html https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi... "Consistency with equals" specifically means a.compareTo(b) == 0 is equivalent to a.equals(b). Java docs explicitly state that this is "strongly recommended (though not required)". https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi%2F%2F/java.base/java/lang/Comparable.html https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi...

doubloon 2y ago

whats funny is the divide between math theory in the 20th century and automated arithmetic. math world - "wouldn't it be great if we had a thing called a field, or even better a group, a bunch of rules everyone agrees on where operations are closed and you can make all these great discoveries and there is so much consistency and predictability..." computer world - "yes but wouldn't it also be great if we could add binary fractions with rounding...."

vitiral 2y ago

I feel like if they had simply named it FixedDecimal they would have avoided the confusion. It makes more sense that 1.0 differs from 1.00 as they are different precicions. As another person pointed out, dividing them by 3 will yield different results.

definitelyauser 2y ago
BigFixedDecimal? Sounds like yet another java naming pitfall.

klysm 2y ago

This seems like a massive API design mistake

Viliam1234 2y ago
What does? The different numbers being different, or some other part?