8 ms·
Getting Started on Geospatial Analysis with Python, GeoJSON and GeoPandas
- tke248 9y agoWould it be possible with something like this to locate all the pools in a give area and find the street addresses?
- anc84 9y agoThe linked post is just a very basic tutorial. Your idea is quite complicated geospatial analysis depending on the source data and desired correctness.
- dagw 9y agoSure, but getting the necessary data would be far from trivial. How would you locate pools? Using areal photographs is an option, but you'd either have to do it manually which would be very time consuming or using image recognition which would be very error prone. Depending on where you live people might have to apply for a building permit to put in a pool on their property in which case the local municipality should have a database over which houses have pools, but there is no guarantee that it is up do date and getting a hold of that database is far from easy. Connecting points to street addresses is also a bit hit and miss. Things like Google's geocoding API works OK for buildings, but is tends to be quite hit and miss for points outside of buildings. Generally it will give you the address of the closest building rather than the address the plot of land actually belongs to. So if you want to be correct you have to get a map with actual property lines and who owns what property. So the ease of doing something like that is entirely dependent on what data you have access and how accurate you have to be. Basically the hard part of any GIS project is always data gathering/cleaing/pre-processing and never the actual analysis.
- dakics 9y agoSpeaking of pools, Greek government in their efforts to collect more tax, used to search for pools using Google Earth http://www.spiegel.de/international/europe/finding-swimming-pools-with-google-earth-greek-government-hauls-in-billions-in-back-taxes-a-709703.html http://www.spiegel.de/international/europe/finding-swimming-...
- Boothroid 9y agoThat's a very broad statement and not one I can agree with - some of the processes I've encountered have been very complex - in fact in certain situations the dynamics of a problem can be so complex that they defy conventional analysis. The field of Spatial Decision Support Systems arose in order to address these type of poorly structured spatial questions. Plenty of literature out there, not so trendy these days though.
- dagw 9y agoI'll admit I was being slightly factious and open data is making everything much easier these days. But geospatial analysis is mostly just math with a bit of programming while data collection often involves trying to deal with county and state level employees operating with a very high power Someone Else's Problem field and then trying to explain that scanned photocopies of old maps is not quite what I was expecting when they said that they had all their maps 'digitized' :) But I'm the sort of person that much prefers dealing with math to dealing with people.
- Boothroid 9y agoThe trend in GIS these days is away from writing code and towards configurable apps, only dropping into code where necessary. Why not just use QGIS instead? Far simpler.
- dagw 9y agoFor the case presented using, QGIS would be a far more obvious solution. However another trend in GIS is much larger scale analysis. Someone will do some analysis i QGIS by hand for a small area, then someone will go "wait a minute. We have the necessary data for the whole city/county/country. Why not run the same analysis over all that data?" Then knowing how to do reproduce the whole analysis in code becomes vital.
- Boothroid 9y agoIsh - in an enterprise of any reasonable size you may likely choose to divide your work by employee skills, hence perhaps most of your GIS analysts need not get their hands dirty with code (nor want to).
- sarcher 9y agoPerhaps our experiences are different, I've noticed the opposite. There will always be a strong role for GIS applications, but I rarely see geospatial problem solving these days that doesn't require some amount of code. I still do 50%+ of my work in QGIS and find the embedded python interpreter to be essential. There are very few projects where I don't open it up, or otherwise have organized/cleaned the data beforehand (often with python, I'm a one trick pony). In 2017 so far only one project has not required some coding, and that was a print map for a small transit agency. All the data could be easily hand-digitized.
- sbrother 9y ago<self promotion, but relevant> I just launched a product similar to this but a bit more powerful and aimed at a less technical audience (most of my customers are Excel users). There is a python client that does geocoding, census data lookup, and driving distance/time if you are looking for something more than the open source options provide. If you're interested, email me or check out https://cairngeographics.com/ https://cairngeographics.com/
- anc84 9y agoUsing geojson.io to visualise the GeoJSON seems like the wrong tool for the job; just use folium, it works perfectly in Jupyter Notebooks: https://github.com/python-visualization/folium https://github.com/python-visualization/folium
- deleted 9y ago[deleted]
- maurits 9y agoThank you, just what I was looking for.
- Iv 9y agoA friend just made a presentation on these exact tools at the Pycon JP 2017 a few days ago, I half expected to see Halfdan's face there: https://www.youtube.com/watch?v=Yd5oEIBFQ_E https://www.youtube.com/watch?v=Yd5oEIBFQ_E
- pb060 9y agoWhy isn't GRASS GIS more widely used in geospatial analysis (or it's just my impression that is not)? I tried several tools but usually GRASS proves to be the fastest and more flexible one.
- dagw 9y agoBecause GRASS doesn't play nice others. Calling GRASS from other apps isn't as easy as one might like and GRASS has a very GRASS-specific way it want to do things. Which is a shame because most of GRASS's algorithms are really best in class.
- sarcher 9y agoI agree with you. That being said, it's a go-to tool for me when I'm cleaning road networks and for the subsequent network analysis. And that's a hugely underwhelming use-case, I'm touching a fraction of what GRASS does. Very helpful community as well.
- dagw 9y agoIf someone made a friendly python GRASS wrapper that would let you use GRASS function in a pythonic manner with numpy/rasterio/fiona/shapely data i think GRASS usage would increase massively.
- Demiurge 9y agoYes, this is it for me too. Whenever I have a huge vector dataset I need to clean up, I would use GRASS simply because it is topological, and can re-interpret, fix, manipulate vectors efficiently and valid, as a single layer entity. Even though PostGIS added topology, I have not been able to figure out how to do the same manipulations, or achieve the same speed.
- aw3c2 9y agoA GUI from the 90s, documentation often indecipherable if you are not a subject expert, awkward UX. I like using it via QGIS but that's as near I can make myself go near it. It saddens me because it seems very powerful and featureful.
- jasongrout 9y agoAnother possibility for interactively manipulating maps is the ipyleaflet Jupyter widget library: https://github.com/ellisonbg/ipyleaflet https://github.com/ellisonbg/ipyleaflet. See examples of notebooks at https://github.com/ellisonbg/ipyleaflet/tree/master/examples https://github.com/ellisonbg/ipyleaflet/tree/master/examples (disclosure: I have helped with this library)
- danso 9y agoGreat tutorial, and thank goodness it's in 3.x. About a year ago it was difficult to find many 3.x examples of geospatial analysis. It felt like there was always some dependency that was not quite 3.x compatible, so it made sense that a lot of folks would just stick to 2.x just to get started.
- rkda 9y agoYou might also be interested in GeoNotebook, a Jupyter Notebook extension for geospatial analysis https://github.com/OpenGeoscience/geonotebook https://github.com/OpenGeoscience/geonotebook
- Waterluvian 9y agoI would also be sure to be aware of GDAL and OGR. Both are quite useful when you're doing anything GIS/Remote Sensing. http://www.gdal.org/ http://www.gdal.org/
- dagw 9y agoIf you're working with python it's also worth looking at rasterio and fiona which provide some nice wrappers around a subset of gdal and ogr.
- Waterluvian 9y agoShapely too!
- deleted 9y ago[deleted]
- Bedon292 9y agoIf you are interested in GIS, make sure to check out rasterio and Fiona to go with Shapely. All great tools for GIS in python.
- rkda 9y agoRasterio ftw! Way easier to use than gdal's Python bindings. Fun fact: Geopandas uses Fiona and shapely under the hood.
- wiredfool 9y agoWhere is a good place to start to take a set of lat/long data (e.g., bunch of ride/walk traces) and plot a neat looking map? It seems to be a little harder than hello world, but not worth a full blown GIS stack.
- rkda 9y agoFor JavaScript folks, there's Turfjs from Mapbox. http://turfjs.org/ http://turfjs.org/