4 ms·
I have some geospatial algorithms in my geogeometry library. Unit testing those, you often need to compare coordinates. A good way to do that is by calculating
by jillesvangurp 4y ago
I have some geospatial algorithms in my geogeometry library. Unit testing those, you often need to compare coordinates. A good way to do that is by calculating their distance. Equality is really not that relevant with many of these algorithms. Almost everything that you do to those numbers has rounding errors.
If you think about it, people use 64 bit doubles for representing latitude and longitude. They don't have a lot of precision. A second of a degree is about 25-30 meters (depending on where you are. that's 1/3600th of a degree. So five decimal points gets you there. With 64 bit numbers, you can probably get to cm/mm level accuracy. Which is plenty considering GPS is not accurate to more than about 5 meter.
And distance algorithms are tricky too; they are not that exact. The earth isn't a perfect sphere.
So, you are dealing with an inexact representation of doubles that weren't very exact to begin with and the algorithms add more errors. The solution is to not compare equality but distance with some margin of error. So, subtract your expected and actual values and assert that their difference should be less than whatever margin of error is acceptable to you.
- adgjlsfhk1 4y agonote that this is a perfect example where you should use scaled integers. you'd get 4000x more precision.
- jillesvangurp 4y agoData comes as doubles. No choice in the matter there. Conversions just add to the problem.