4 ms·
You're making the same mistake but now it's less obvious because the change of scale. When you compare floating point numbers, simple == is not usually what you
by firethief 7y ago
You're making the same mistake but now it's less obvious because the change of scale. When you compare floating point numbers, simple == is not usually what you want; you need to compare them with a tolerance. Choosing the tolerance can be difficult, but in general when working with small numbers you need a small value and with large numbers you need a large value. This dataset involves datapoint(s) at 1e20; at that magnitude, whatever you're measuring, the error in your measurements is going to be way more than 1, so a choice of tolerance ≤ 1 is a mistake.
- civility 7y agoUgh, you're preaching to the choir. I wasn't trying to make a point about the equality operator, I was trying to make a point about x and y being completely different. I must be really bad at communicating with people.
- firethief 7y agoThat's the thing: x and y are not "completely different"; they are within the tolerance one would use to compare them correctly.
- civility 7y agoSo umm, more than a "difference in the realm of 6-8 significant digits"?
- firethief 7y agoThat construction can turn any residual N digits out into a first digit difference. It wouldn't matter without making comparisons with tolerance way under the noise floor of the dataset. But yes, you have technically invented a situation that differs from that real-world anecdote in regard to that property, in an extremely literal interpretation.
- civility 7y agoAnd here I was, worried I might be tedious or pedantic, trying to argue that floating point is just not that simple. You've really outdone me in that regard.