12 ms·
A fast, offline reverse geocoder in Python
- zetahunter 12y agoAwesome, one more thing that can be made standalone instead of using google maps service.
- thampiman 12y agoThanks! I'm the developer of this library and I hope you find it useful.
- dexterbt1 12y agoStarred. We're currently using nominatim + osm data + postgis on our own hosted servers. Can this be a good alternative?
- thampiman 12y agoI should think so. I've tried nominatim/osm data but it took forever to query a large set of coordinates. I was only interested in knowing the nearest city and admin regions 1/2. And this library is really fast... ~20s to lookup 10M coordinates on my MBP. If you'd however like to know the full address, then this is maybe not a good idea.
- dexterbt1 12y agoThat's fast. Though yes, our use case involves reverse geo of full street level addresses. Currently we do several 10s to 100s of req/s on nominatim. The sibling thread asked about using OSM data; it'd be awesome if street level OSM data is workable.
- ldng 12y agoFor reverse geocoding only as it does not seem to do geocoding proper. Pelias[1] might be a better alternative once they simplified the install process. I had to "reverse engineer" (aka read and understand) their chef cookbook as a Vagrant was not an option form me. Not that complicated but time consuming when don't know Chef. [0] - https://github.com/pelias https://github.com/pelias
- staz 12y agoWe are currently using Postgis + geonames (the same DB used by this project) and it works very well. I'm curious to see benchmarks comparing the two. edit: We are only interested in knowing the nearest big city though
- sandstrom 12y agoLooks really interesting! Would it be possible to use OpenStreetMap data? http://planet.openstreetmap.org/ http://planet.openstreetmap.org/
- thampiman 12y agoThat is interesting. I haven't looked at that data set. At the moment, this only looks at cities with a population > 1000 obtained from GeoNames.
- mtmail 12y agoOSM data doesn't contain an easy way to find the top 1000 cities. You'd end up with 100.000s. Looking for wikipedia tags, population (which often comes from Wikipedia) and 'admin' tags might be a good guide. (I work on a OSM geocoder, not offline but has a Python library http://geocoder.opencagedata.com/ http://geocoder.opencagedata.com/)
- mcbetz 12y agoVery good companion for Geocoder - https://github.com/DenisCarriere/geocoder https://github.com/DenisCarriere/geocoder. Glad to see Python getting more geo libraries for Non-GIS users.
- dheera 12y agoAre there any offline Geocoders that work for the whole world, even if not free? Nominatim doesn't work for a lot of Asian addresses.
- yellowbkpk 12y agoIf you're interested in making both forward and reverse geocoding better, please consider paying attention to a project I started and help maintain called OpenAddresses: http://openaddresses.io http://openaddresses.io The goal is to collect address datasets so that forward and reverse geocoding is an easier problem to solve. A contributor wrote an excellent overview of the project the other day: https://medium.com/colemanm/creating-an-open-database-of-addresses-73a7d0dc24c5 https://medium.com/colemanm/creating-an-open-database-of-add...
- thampiman 12y agoVery cool project. Out of curiosity, how do you test the validity of the addresses?
- yellowbkpk 12y agoAll of the addresses come from "authoritative" datasets (i.e. from local or state governments) so we're assuming them to be correct.
- Stratoscope 12y agoInteresting project! I assume you don't have a manual procedure to report and correct individual addresses that are in error, and the trick would be to contact the agency that provided the erroneous dataset, is that right? It would be useful to have a way to find out "where did the data for this location come from, and who do I contact to correct it?" For example, our house doesn't have an outline on your sample map, although there is a red dot there. Our street address shows up on the house next door, and the neighboring houses in that direction are shifted similarly. Most other online maps for our location have the same error or similar. I think the confusion happened because we are on a corner and some years ago a previous owner changed our street address from one street to the other. So whatever agency maintains this data goofed something up in the process. If there were an easy way to find out who this mystery agency is, then I could help them sort out this mixup. :-)
- 12y ago
- kelukelugames 12y agoHello, I read a little bit about geocoding on wikipedia but was hoping to learn more. Is a good beginner guide on geocdoer/geocoding?
- deleted 12y ago[deleted]
- natch 12y agoKudos for a very well done README (and it's not just cribbed from the original project, it explains the new stuff very well and tells what the project is, and gives credit back). So many projects neglect the README. One question - is it OK to put an MIT license on something that is based on LGPL code? I don't know enough about how the LGPL works (I do know it is less "infective" than plain GPL). Well two questions: python2, or python3?
- thampiman 12y agoThanks for that comment! Good question regarding the license. I'm not too sure about that. I'd appreciate it if someone could shed some light on it. Regarding the version, I've only tested it on python2. I should add that in the README. Thanks!
- stared 12y agoAs of now, for Python 3 it does not work, but it seams that fixes are not hard: https://github.com/thampiman/reverse-geocoder/issues/2 https://github.com/thampiman/reverse-geocoder/issues/2
- belorn 12y agoIf its based on LGPL code, then any original code need to retain it's license. Beyond that, everything regarding licenses look good if you ask me. MIT is both compatible if you modify the LGPL code or if you link with it, so one can use either of the two licenses.
- bzz01 12y agoYou can't really use KD trees with lat/lon coordinates, at least you can't use euclidean distance there for nearest neighbor search. First, longitude wraps from -180 to +180 at antimeridian, meaning distance calculations will fail there; second, and I'd say more importantly, one degree longitude length in meters differs a lot depending on latitude; meaning this library will be heavily biased towards longitudal neighbors when using it for locations far from equator.
- thampiman 12y agoGood point. I would need to use this distance function instead: http://www.movable-type.co.uk/scripts/latlong.html http://www.movable-type.co.uk/scripts/latlong.html On it!
- corford 12y agoHere's a python function we use to calculate haversine distances: import math from collections import namedtuple def haversine_distance(origin, destination): """ Haversine formula to calculate the distance between two lat/long points on a sphere """ radius = 6371 # FAA approved globe radius in km dlat = math.radians(destination.lat-origin.lat) dlon = math.radians(destination.lng-origin.lng) a = math.sin(dlat/2) * math.sin(dlat/2) + math.cos(math.radians(origin.lat)) \ * math.cos(math.radians(destination.lat)) * math.sin(dlon/2) * math.sin(dlon/2) c = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a)) d = radius * c # Return distance in km return int(math.floor(d)) LatLng = namedtuple('LatLng', 'lat, lng') origin = LatLng(51.507222, -0.1275) # London destination = LatLng(37.966667, 23.716667) # Athens print "Distance (km): %d" % haversine_distance(origin, destination)
- shawn-butler 12y agoI can recommend the excellent Geographic lib::Geodesic for this having used it in the past. http://geographiclib.sourceforge.net http://geographiclib.sourceforge.net There is a python implementation available as well. http://pypi.python.org/pypi/geographiclib http://pypi.python.org/pypi/geographiclib
- nickstefan12 12y agoNice! Shameless plug for a SQLite no network geocoder that uses (I believe) the same text files to seed everything. https://github.com/NickStefan/no-network-geocoder https://github.com/NickStefan/no-network-geocoder
- Animats 12y agoWhile we're on this subject, is there a good, free street address parser that will work for at least the US, Canada, UK, and the major EU countries? I've tried most of the available ones, and they can parse about 90-95% of business addresses. (Regular expressions don't work well for this. Neither does starting from the beginning of the address. Proper address parsing starts at the end of the address and works backwards, with the information found near the end, such as country name and postal code, used to disambiguate the information found earlier.)
- unclesaamm 12y agoThe most principled approach I've seen on this is at https://github.com/datamade/usaddress https://github.com/datamade/usaddress. They use tagged training data and conditional random fields. I haven't seen comparisons with other systems, but it's worked well enough for my projects. Though as the name suggests, it's only trained for US addresses.
- dheera 12y agoHave you tried Nominatim? I've found it to work for most of the above. I'm only disappointed it doesn't work well for developing countries, particularly a lot of Asia.
- alexcroox 12y agoThis is great, does anyone know of a js version? I'm currently using http://nominatim.openstreetmap.org/reverse http://nominatim.openstreetmap.org/reverse in my Node app but I'd rather not rely on a 3rd party, especially under heavy load.
- bronson 12y agoVery impressive, I'll be looking closer at K-D trees. I wrote a quick (500k lookups/sec) offline geocoder for Ruby: https://github.com/bronson/geolocal https://github.com/bronson/geolocal to comply with the silly EU cookie rules. It precompiles the statements you're interested in: Geolocal.in_eu?(request.ip) Geolocal.in_us?('8.8.8.8') Glad to see that my lib has a role model if it ever grows up. :)
- pjkundert 12y agoOn a related note: An efficient geolocation encoder/decoder with error correction using Reed-Solomon. 3m accuracy with error correction in 10 symbols. 20mm accuracy with 5-nines certainty in 15 symbols: https://github.com/pjkundert/ezpwd-reed-solomon https://github.com/pjkundert/ezpwd-reed-solomon
- thecodemonkey 12y agoThis is super cool! Shameless plug. If you're looking for street-level reverse (or forward) geocoding, we offer[1] a super affordable API and CSV upload tool. [1] http://geocod.io http://geocod.io