7 ms·
Redis Geo
- cg25 11y agoI wrote an introductory article to the new Geo API. Comments are welcome :) http://cristian.regolo.cc/2015/07/07/introducing-the-geo-api-in-redis.html http://cristian.regolo.cc/2015/07/07/introducing-the-geo-api...
- itamarhaber 11y agoIndeed - and as we speak antirez is busily refactoring, merging, testing and extending it in Redis... happy-meter is off the scales here.
- slowernet 11y agoFollow along at home: https://github.com/antirez/redis/commits/geo https://github.com/antirez/redis/commits/geo
- kyleblarson 11y agoWould love to know how this performs compared with analogous PostGIS operations on a similar sized set of geos.
- jandrewrogers 11y agoIt is a little bit apples and oranges. This is a simple point-in-plane index, PostGIS is a full-blown geospatial data modeling capability. PostGIS is going to be much slower but it is also doing much more as part of the process so that it is useful for sophisticated spatial applications. PostGIS is not fast relative to comparable systems but this particular comparison is not fair because PostGIS is expected to do so much more. That said, I have also implemented large-scale point-in-plane indexing systems, though not in a while. Per CPU core, you can encode and index around 20M geo-points per second if the implementation is good.
- marpstar 11y agoWe're using Redis Geo to track customers who are "online" so that we can easily retrieve a list of users who are online and in the area. The user's permanent location is stored in a SQL database for permanent storage.
- kbenson 11y agoHow do you deal with locations close to boundaries? Isn't that still a problem with comparing geohashes for proximity?
- antirez 11y agoThis is handled by the Redis implementation (and most others AFAIK). So it returns the right items regardless of the query position or the items position. Redis has tests in place for this.
- mikeatlas 11y agoIt's great to see you (jandrewrogers) of SpaceCurve reply to correctly to the question! Your blog post on why Geospatial Databases are Hard has long influenced me and a handful of others I've run into thinking about all of this. Geohashed (X+Y) points in Redis is only peripherally related to actual geospatial database concerns, particularly big-data or ("NoSQL") spatial database concerns - how do you efficiently index anything other than point data (polygons, multi_polygons), or ask spatial queries (e.g., nearest neighbors) is not even close to being addressed. Blog post here for reference: http://www.jandrewrogers.com/2015/03/02/geospatial-databases-are-hard/ http://www.jandrewrogers.com/2015/03/02/geospatial-databases...
- ppymou 11y agoI never knew that redis can be extended for geo. This is really cool. I researched geodb a while back and I only found PostGIS but I recently found out that elastic search also supports geo features. Shameless plug - This is the simple API I built on top of PostGIS https://github.com/moo-mou/GeoGo https://github.com/moo-mou/GeoGo
- Wouter33 11y agoDynamic Redis (which is a Redis Fork that lets you add modules to it) is a great find! Especially with the Geo module added. I needed a limit parameter for the Georadius method because i wanted a result set sorted by distance in a large radius, but not have thousands of results send from the redis server to my webservers. Anyone interested in such a parameter can use my little fork for the Geo module: https://github.com/wouter33/krmt https://github.com/wouter33/krmt
- antirez 11y agoGood idea, I'll add LIMIT. Makes sense to say I want N nearest elements with max distance of Y.
- Wouter33 11y agoGreat Salvatore! Could you make the max distance parameter optional? Sometimes i just want the nearest items, even though they could be on the other side of the world.
- Jake232 11y agoI feel like that is a fairly unusual usecase; plus you could just set the max distance parameter to the diameter of the Earth.
- Wouter33 11y agoOne of our applications has user generated content with a geolocation attached to each of the items. For several selections we want to show the top 10 nearest items. If they are only a few, the radius has to be very large to have some items in the result set. Currently the Geo module only supports a range up to 20.000 km. So your idea does not work at the moment. I agree that those items are not really "nearby", but to be consistent through out the application it would be nice to be able to select without a max distance.
- cwisecarver 11y ago
- hoprocker 11y agoThis is awesome. I've been using Redis -- naively, with hashes -- as a lat/lon cache for some path analysis on my website. Didn't even know this was out there, and can't wait to switch over!
- jerrysievert 11y agoi can only ask that this extension requires longitude and latitude in the correct order: x,y.
- antirez 11y agoSomebody opened an issue about this too. While in the GIS world apparently this is the standard, outside the GIS world everybody talks in terms of latitude,longitude (y,x). So I'm not sure what to do. I suspect the majority of the users were not exposed to GIS systems, and that the GIS exposed users will check the parameters better, so I'm inclining in taking y,x in favor of the average person that like me thinks latitude,longitude is normal.
- jerrysievert 11y agothe issue is that you are now entering a different sector of technology, where there are already standards in place, and those standards are x, y, z ordering: * mongodb geospatial extensions - http://docs.mongodb.org/manual/core/geospatial-indexes/ http://docs.mongodb.org/manual/core/geospatial-indexes/ * postgresql geospatial extensions - http://postgis.net/docs/manual-2.1/ http://postgis.net/docs/manual-2.1/ * geocouch - https://github.com/couchbase/geocouch/wiki/Spatial-Views-API https://github.com/couchbase/geocouch/wiki/Spatial-Views-API so, it isn't just GIS users who are doing this, it is also the convention across databases that support geo functionality. i hope that you reconsider this, and don't go against what is both standard by committee and standard by implementation.
- chriswgt 11y agoDamn, you're right about GIS databases. In my apps I always used lattitude first, as was taught in geography lessons, but you've convinced me to move from lattitude endian to longitude endian. Two choices - too many choices...
- jerrysievert 11y agoit gets worse. there are tons and tons of spatial references, and they can all be defined differently. see: http://spatialreference.org http://spatialreference.org oregon, for instance, has two official state spatial references (if top of memory is correct).
- antirez 11y agoHello, I'm currently integrating, refactoring and fixing the original code, in the context of this branch: https://github.com/antirez/redis/commits/geo https://github.com/antirez/redis/commits/geo So far the code was simplified and polished, fixed (it did not returned the right set of elements in all the cases), GEORADIUS speed was improved. On the other side I removed all the JSON things and the Pub/Sub (IMHO is interesting to subscribe to areas to see objects entering/exiting them, not much to position updates since it's the same of doing PUBLISH+GEOADD from the client side). Added: GEOHASH to return a Geohash standard string from of a given element, mainly for interoperability. I'm also adding GEODIST (to calculate distances) and GEOPOS (to retrieve latitude/longitude of objects without the need to query for radius). The work will be available in the context of Redis 3.2, unless I get mad and merge back into 3.0, which is unlikely but not impossible. EDIT: before of three days ago I was not aware of Geohashing at all, so if you have five mins to check the code, I'll be glad :-) Thanks. I'm pretty confident that it's ok now after I added a fuzzy test that replicates the feature (by brute forcing distance computation) in Tcl that checks against the Redis output, but still... EDIT2: wanted to add that I'm very glad Matt efforts are not going to be wasted and we are finally merging this.
- deleted 11y ago[deleted]
- arturventura 11y agoI have a question that I haven't seen address in the article. Does the code always assume the "sphere" is Earth or can you provide geometric information for other geometries?
- antirez 11y agoIt assumes the Earth is a sphere, also the code is not designed to work well towards the poles, at latitudes less or more than -/+ 85.
- moron4hire 11y agoI work on a system that uses GPS data to figure out some metrics about roads. At that scale, the spherical model was not a useful approximation. It's been useful for drawing on top of Google Maps, but that's obviously because Google Maps chose it to keep straight lines rectilinear. The spherical model is a visually convenient model, but I don't think you can really do any serious calculations with it.
- antirez 11y agoRedis Geo commands are mainly intended to solve the specific problem of indexing points into the earth and querying by radius. In this setup, the error we have currently is around 0.5% because Earth is not actually a sphere, but the good news is that fixing it should be just a very simple matter of using a more accurate distance formula: http://www.movable-type.co.uk/scripts/latlong-vincenty.html http://www.movable-type.co.uk/scripts/latlong-vincenty.html Since anyway the bounding boxes we calculate and the resulting areas we scan are larger than needed, so all the actual precision is in the filtering function that removes points outside the radius. TLDR: It will be trivial to fix that later in a backward compatible way. Btw adding an item in my TODO in order to check if I can do it right now without compromising performances.
- moron4hire 11y agoHaving done both Spherical Mercator and Universal Transverse Mercator in my own code, UTM is something like 25 times as many arithmetic operations as Spherical. That's not to say I have the most ideal implementation, though.
- sidarape 11y agoSome can ELI5 redis for me? And also this extension?
- doppel 11y agoRedis is an in-memory "data structure server", in that besides simple key-value (think memcached) storage you can also have sets, lists, sorted sets, hashes and more. See redis.io. This extension adds geo-specific commands and storage to the list of operations supported.
- sidarape 11y agoInteresting! Thank you for your answer. I have previously heard that people use that to do cache server for websites. How does that use case is covered by reddis? Also, what are other common use cases for reddis? Thank you.
- stevekemp 11y agoWhat reading have you done of http://redis.io/ http://redis.io/ ? What search terms have you used on your favourite search engine? People here are generous with their time, skills, and knowledge, but your initial question and your followup makes it sound like you've not even tried to educate yourself.
- chriswgt 11y agoSteve, you're probably talking to a bot, designed to collect some karma points, to be used for upvoting/downvoting some important article or comment in the future.
- sidarape 11y agoAhaha lol whut?! Seriously, I'm asking questions because all I see are features but no real use cases that I could say that Reddis is useful. I think you may be a litle too defensive.
- deleted 11y ago[deleted]
- har777 11y agoHell yeah ! I've been waiting for this to be integrated into Redis.
- krisdol 11y agoWe've been using this since day 1 at our startup (well, at this point it's more like early-stage free-time group project). Very happy to see support from antirez
- krisdol 11y agoWe've been using this since day 1 at our startup (well, at this point it's more like early-stage free-time group project). Very happy to see support from antirez
- emehrkay 11y agoRedis is damn cool. As an effort to better understand it and after reading through the Twitter in Redis example, I decided to see how much of a graph database I could create using it and Python (https://github.com/emehrkay/rgp https://github.com/emehrkay/rgp) Dynamic Redis seems like a decent project and would make hacking core functionality fun and simple.
- itamarhaber 11y agoLoading plugins into Redis, a-la Dynamic Redis approach or otherwise, has long been discussed in the Redis community. Presently, antirez prefers not to destablize Redis (and thus damage its solid stability reputation) by allowing external code be run in it. So, despite the obvious perks that we'll enjoy by side-loading our own junk, antirez's "Quality or DEATH" approach on this matter leaves no open questions about the possible inclusion of this functionality. There is, however, another way to get code running in Redis... it is called a GitHub Pull Request :) If you've thunk up something really amazing/useful and coded it, drop by the redis-dev mailing list to discuss it, maybe do an RCP and submit your code. No one promises that your PR will be accepted, but even if it won't make its way into the core, anyone will be able to use it nonetheless. Another point worth mentioning in this context is that in order to even contemplate such a mechanism, Redis' core will need heavy refactoring of the internal API. This is by all accounts a huge effort that, if taken by the developer, is likely to postpone a lot of other important work.
- rakoo 11y agoAs someone outside to redis development, I'd like to know: is Dynamic Redis that more performant than good ol' lua scripts that it's worth introducing potential incompatibilities between redis instances deployments ? I stumbled upon the old discussion about adding a scripting language to redis, and the number 1 requirement was that additional code should be "migratable" very easily from instance to instance. A related question: is there some kind of repo for "useful lua scripts that everybody could use but didn't make it in core redis (yet)" ?
- antirez 11y ago
- kposehn 11y agoVery interesting. We could have used this ourselves, but ended up writing a way (circa 2010) for Postgres to be queried by MGRS (Military Grid Reference System: https://en.wikipedia.org/wiki/Military_grid_reference_system https://en.wikipedia.org/wiki/Military_grid_reference_system)
- nchelluri 11y agoThis is unrelated, but I've been hearing great things about the readability and teaching ability of the Redis code for people interested in learning good C practices. I knew/know C but have been doing high level programming for years. Would anyone suggest if I wanted to visit this source tree, I check out an earlier version (like 1.x) or just jump straight to latest stable or master?
- urlgrey 11y agoI've got a Docker image for Dynamic Redis that includes all of the standard modules, including geo.so: https://registry.hub.docker.com/u/urlgrey/dynamic-redis/ https://registry.hub.docker.com/u/urlgrey/dynamic-redis/ Github repo: https://github.com/urlgrey/docker-dynamic-redis https://github.com/urlgrey/docker-dynamic-redis