2 ms·
It may seem low, but it's really not. At 6 significant digits, you'd have a margin of error of a thousandth of an inch when measuring a football field.
by kstenerud 6y ago
It may seem low, but it's really not. At 6 significant digits, you'd have a margin of error of a thousandth of an inch when measuring a football field.
- 0xffff2 6y agoCounterpoint: At 6 significant digits you have a margin of error of thousandths of a millimeter when measuring degrees longitude near the prime meridian, but the error balloons to whole meters when measuring longitudes in Australia. I have a piece of software that I wrote ages ago where I looked at the max error in the US and concluded that 32bit floats were enough. This made sense since at the time I was reasonably confident that the tool would only be used with locations in the continental US and it simplified my architecture. Fast forward a few years and we ended up open sourcing the tool and people started using it all over the world. I'm still dealing with the fallout to this day. I'm dreading the day that the tool gets used in some inter-planetary context and I have to repeat all of the work to add support for 128 bit/arbitrary precision location data.
- T-hawk 6y agoSure, that's low, depending on context. At 6 significant digits, a million-dollar bank account would have a margin of error of up to +/- 5 dollars. And if you add up a million of any item with 6 significant digits, that's enough for inaccuracy to propagate all the way into your first significant digit. (It would require systematic inaccuracy in one direction, but there are certainly data sets and contexts where that would happen.)
- kstenerud 6y agohttps://dzone.com/articles/never-use-float-and-double-for-monetary-calculatio https://dzone.com/articles/never-use-float-and-double-for-mo...
- s9w 6y agoYou are also not able to represent your yearly salary to the cent. At least if you're American haha