10 ms·

Opinions I have formed about the “geospatial industry”

170 points by luu 6y ago

stickfigure 6y ago
This seems to have been written by someone unfamiliar with real estate, local government, natural resources, or any other traditional line of business that depends heavily on GIS. I don't know what my county pays for ArcGIS but it's probably substantial, and I suspect there are multiple IT employees devoted to maintaining the system. Replicate that across nearly every single political locality in the US, probably the world. The industry is huge. Also, "long form twitter" is dumb.
- SOLAR_FIELDS 6y ago
  I used to work for a local government (municipality) as part of their GIS department and we were spending about 10-15 grand a year on licenses IIRC. This was for an ArcGIS server license and 3 “power users” (ArcInfo), 5 “editors” (ArcEdit) and 5 “viewers” (ArcView). We were pretty small compared to most orgs, basically 3-4 people in the department at any given time. I distinctly remember the ArcGIS server license being a very significant portion of that number, maybe even half of it. Things may have changed since then (this was nearly a decade ago, before ArcGIS Pro licensing style was implemented), but probably not by much, you can probably extrapolate that ballpark and figure how much any actually decent sized org spends on Arc suite of products (a lot)
- campchase 6y ago
  Hi stickfigure, this is Joe from the dumb long form twitter thread you just wrote five sentences critiquing. Thanks for your feedback, I really appreciate it.
Waterluvian 6y ago
Two geography degrees meant I saw a lot of ESRI. In fact I won a national award and was flown out by them to meet Jack in San Diego. Everyone at ESRI I interacted with were all wonderful people. But their software is just brutally corporate and slow to respond to market changes. FOSS GIS software is pretty good but suffers from the usual lack of budget. Nevertheless, I’ve been able to push robotics + GIS packages for QGIS with ease. Reaching out to my ESRI contacts about “where’s your indoor and outdoor robotics suite offerings” received a response of only crickets. As for Google Maps. The biggest tell is that there’s almost no geospatial capability in the software. They don’t want you to analyze and understand. They just want you to consume “where is X”. It’s a standard Google vector for ads approach. I’m going to rant for a long time if I don’t stop here so I’ll leave it at: stop teaching so damn much ESRI and start teaching way more programming. The biggest regret of my education was that I was mostly taught how to use Geospatial tools rather than how to make my own.
- craftinator 6y ago
  When you say, "robotics + GIS packages", what exactly do you mean? I've had a little experience with ESRI products, and a lot of experience with robots, but I don't understand the connection here.
  - Waterluvian 6y ago
    Mobile robots are highly instrumented spatial data collection platforms. Here’s a very brief high level view. Regrettably I only touch on the surface of what is considerably deeper. https://roscon.ros.org/2018/presentations/ROSCon2018_UnleashingGISToolbox.pdf https://roscon.ros.org/2018/presentations/ROSCon2018_Unleash... https://github.com/locusrobotics/qgis_ros https://github.com/locusrobotics/qgis_ros
    - craftinator 6y ago
      Ah I see, thank you for the follow-up!
- chrisbroadfoot 6y ago
  > As for Google Maps. The biggest tell is that there’s almost no geospatial capability in the software. They don’t want you to analyze and understand. They just want you to consume “where is X”. It’s a standard Google vector for ads approach. It was never meant for that. That is/was Google Earth Pro.
  - samplatt 6y ago
    Rio Tinto runs some inhouse software that utilises (amongst other things) Google Earth Pro for analysis of open-pit minesite walls, constantly scanning for movement or changes that could lead to instability of a wall or road.
  - campchase 6y ago
    A similar product, Google Earth Enterprise is now open source, amazingly. (http://www.opengee.org/ http://www.opengee.org/) Not to be confused with Google Earth Engine, which is the most shocking piece of technology I’ve come across in my career and is absolutely free if you can get access: https://earthengine.google.com/ https://earthengine.google.com/
artificialidiot 6y ago
I agree to all points. For point 6. Mapbox is a big contender for ESRI's current position. For 8. Openstreetmap, GDAL and associated open source ecosystem offers a solution to proprietary format and API of the month problem to a degree. It is not surprising that there is no money flowing if mapping is not a core business of the users. Aerial and satellite imagery is the heavily regulated and lucrative part of any mapping solution. If you are allowed and able to obtain that, you can probably get by just selling the raw imagery without geospatial processing.
- redis_mlc 6y ago
  I write and maintain some OpenLayers API (OSM) Javascript, used in a product. OpenLayers was selected after Google started charging 10x for their maps API, grayed out the existing tiles, then required a credit card on file. Since the API key can be "view sourced", that was totally a non-starter, along with the always-changing ToS. Way too many moving goal-posts. (Part of the reason non-free is a problem is that any testing ends up costing money, and the more testing you do, the more $$, which is a disincentive to do testing. But just staying on top of lopsided ToS gets old fast when you can just replace it.) Using a combination of OpenLayers and SO sample code, I've been able to do everything I wanted to do and am happy with the final result, but usually it's some of the toughest programming of the month.
- jaakl 6y ago
  Mapbox tries to replace Google maps, ESRI is different category which is only slowly challenged by enterprise vendors (Azure and other clouds) and FOSS: PostGIS and other sptially enabled databases which have more and more geo features, making specialized GiS to a niche.
TheSpiciestDev 6y ago
I've enjoyed working on an enterprise, GIS-focused application over the past few years (with no prior, formal geographic background) and one of the take-aways I always enjoy sharing with family and friends is the "point in polygon" question [0] and it's simplicity. I'm fun at parties. I have certainly learned how much more intuitive data is for users to explore when you offer (if you can) spatially. [0] https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_algorithm https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_a...
- deleted 6y ago
  [deleted]
- CyberDildonics 6y ago
  Are you really doing the test in the wikipedia article over and over? Why not convert a complex polygon to triangles? Things like this were established in computer graphics in the 70s.
jandrewrogers 6y ago
As someone deep into the cutting edge parts of the geospatial industry that few ever see, I’d quibble on some of the points but it is a broadly accurate description of the current state. The industry has been comfortably stagnant for many years. A critical distinction that explains much of what we see today is that geospatial analytics has almost no relationship to making maps at a technical level. There is a superficial similarity because they sometimes share presentation layer. All of the popular infrastructure and tooling was designed, either implicitly or explicitly, for making maps. If you want to analyze some physical world behavior using multimodal sensor data, the paucity of viable data infrastructure is insurmountable for most companies. Unfortunately, this is what most organizations want to do these days.
jld 6y ago
As a mid-career software engineer who would be interested in getting into the "geospatial industry", A) What technologies should I learn to allow me to get my foot into the industry? I have historically focused on being a UX focused web engineer. I'm sure that experience could surely transfer, but am interested in learning the technologies and/or understanding that is more core to geospatial problems. B) As someone who loves maps and imagery, but does not want to work for the military industrial complex, how do I avoid point #3. "In geo, you either die a hero or live long enough to make the majority of your revenue from defense and intelligence."
- campchase 6y ago
  A: Break it into two parts, vector and raster. For the former, focus on Mapbox and associated libraries/standards (VectorTiles, Leaflet, GeoJSON, etc.). JS is the name of the game in vector land—it’s amazing what you can do all on the front end now. For backend/DB tech, look at PostGIS. For the latter, I’d focus on Python to start. Look at GDAL (and it’s various Python interfaces/wrappers like rasterio), Cloud Optimized Geotiffs, and the SpatioTemporal Asset Catalog spec. In both cases, watch what the leading companies are putting out (Mapbox, Azavea where I work, Development Seed, etc.) and not what OGC is doing. One is a leading indicator, the other is a lagging one. Sorry OGC, I still love you and you play an important role. B: Get comfortable with the idea of working with Defense. For better or worse, they’re the ones with the budget to take on humanitarian work, disaster response work, even climate change adaptation work. That’s my advice. Possible to avoid it altogether, but only if you work on an app that is domain-focused (eg Trulia) and incidentally geospatial.
- vharuck 6y ago
  B) Look into healthcare systems or public health. Defense holds sway because of its budget size, but it's not the only fat cat in town. Epidemiologists are nice, but they're expensive, rare, and usually bad at GIS programming. Most governments will happily hire a GIS analyst. They can't get enough maps and data dashboards. And most of the ones they do have are terrible (misleading, slow, convoluted, or just plain not useful).
campchase 6y ago
Hey, this is Joe from twitter thread linked. I didn’t post this to HN but it’s nice to see it on here. I was wondering why a weeks-old tweet suddenly started popping off... One thing I’d like to note—-I have learned a tremendous amount from other people responding to the opinions enumerated in that thread. Some things I’ve even started to question. So I’d love to hear your ideas—-especially if you disagree with me. Just please do it constructively. Sometimes seeing your ideas thrust onto HN without being prepared for it can be a harrowing experience and I’d rather this time around be a productive one. Much love to the brilliant minds of HN -Joe
- sankalp221 6y ago
  I think your tweet is taking off here because Facebook's acquisition of Mapillary is currently trending on the front page. My guess is lots of people here are trying to understand the industry. https://news.ycombinator.com/item?id=23605362 https://news.ycombinator.com/item?id=23605362
  - campchase 6y ago
    Wait what? I didn’t know about that post either.
    - campchase 6y ago
      I don’t think anyone has ever posted something I’ve written to HN, let alone two things in the same day.
- atupis 6y ago
  Any good resourse to start with geospastial stuff? personally I am from data science background.
  - Stevvo 6y ago
    I found this to be an easy & fun project; https://blog.patricktriest.com/game-of-thrones-map-node-postgres-redis/ https://blog.patricktriest.com/game-of-thrones-map-node-post... If not a game of thrones fan, it's adaptable to something real world.
csteubs 6y ago
I'd argue another point; satellites are a major bottleneck to GIS as we're familiar with. With only ~300 or so commercial imaging satellites in orbit, current revisit rates are sufficient for most use cases. Maybe one or two images/day for X point depending on weather. Low earth orbit is getting crowded, fuel to raise and lower orbit is limited onboard, expensive maintenance cycles, etc. I'm working on a startup trying to solve this problem now. The idea is to "crowdsource" aerial imagery using a network of camera arrays on 80,000 commercial flights/day. We stream the video feed back to earth using in-flight wifi and overlay the visual diff to create a map that shows minute-level changes. There are huge built-in cost savings that come with not having to develop, launch and maintain a satellite constellation, and with a vastly higher revisit rate (as high as 4,532/day in some areas). We're at notasatellite.com
- oh_sigh 6y ago
  I'm clueless in this space but I'd imagine the cost of adding these sensors(installation, ongoing maintenance, certification etc) to airplanes is non-negligible? I assume you're targeting small personal aircraft for the sensor array and not some airline with a fleet of 737s?
  - csteubs 6y ago
    The sensors are mounted internally to the window in order to avoid any sort of external retrofit work, tunnel testing, FAA approval, etc. The MVP is actually just an app that uses the phone's camera to the same effect. We're talking to a number of carriers and exploring all the other deployment options as well (including a frequent flyer revenue share program).
- sgt 6y ago
  Streaming the video feed using the onbord wifi? I think you are overestimating the quality of the Internet connections available on flights.
  - notahacker 6y ago
    Having worked on inflight connectivity apps and Earth Observation respectively, for a couple of startups I'm inclined to agree overall. Workarounds would be: [i] ground sync, which would be fine for a lot of use cases where higher revisit rate and sometimes less cloud cover is sufficient to make the imagery valuable, and open up non-wifi flights. Ground sync can be over regular 4G or even swapping flash drives [some W-IFE boxes update this way, though infrequently]. [ii] Getting dedicated bandwidth from the satellite/a2g internet provider, who can potentially allow you a lot more than is made available to an individual consumer, but obviously at a hefty charge. [iii] using streaming codecs designed for low kbps rates - though this is obviously lossy, and pixel perfection matters for a lot of EO use cases.
jhoechtl 6y ago
ESRI is essentially the Oracle of GIS. To big to fail and to powerful to be wiped away. On the sweet spot of acceptably behind the trends. Yet nobody go ever fired because of acquiring ESRI Software.
littlestymaar 6y ago
> The most successful and ambitious mapping project of all time, Google Maps Google map is indeed a commercial success, but it's far from “the most successful and ambitious” when it comes to mapping itself, quite the opposite. State projects (even those developed before satellites where a thing[1]) are way better, and even openstreetmap achieves a better coverage and higher maps quality. It's the “most successful and ambitious” commercial project, which is coherent with the rest of the thread. [1]: for instance: https://www.sovietmaps.com/ https://www.sovietmaps.com/
- ramraj07 6y ago
  Do you have a citation for OSM? My experience in first world countries especially where Google had ground truth project are still much better in Google than osm.
  - morsch 6y ago
    Casually browsing OSM in my area, I see individual park benches, garbage cans including type, road surface material, doorways to buildings, power lines, location of springs. And that's just what Osmand is rendering, I'm sure I'm just scratching the surface. Google doesn't even attempt to map most or all of these things here, much less expose this information. OSM attempts to build a much better map, and for my region, easily succeeds. That's not to discredit Google Maps. Almost all of the time, I don't need the best map, I need the kind of yellow pages, yelp, sightseeing guide, live traffic and transport, navigation mashup Google is projecting onto their map with great skill. I think it's their best product.
  - littlestymaar 6y ago
    Maybe if you limit your definition of “first world countries” to urban areas, but I live in a somewhat rural place (not the middle of nowhere, I have 4G network and 200Mbs internet) and my street, built in 2011, doesn't appear on Google Maps. In fact it looks like everything built after 2010 in my region is missing from Google Maps. And never ever use Google maps if you're planning a bike trip, unless you really want to get into troubles. As a host on warmshowers.org for the past four years, I've heard tons of horror stories…
atrocious 6y ago
My academic and work history is with GIS, but I've spent the last 4 years in fintech. What does the cutting edge of GIS look like in tech nowadays?
- growlist 6y ago
  Take a look at what OGC are working on.
notagoodidea 6y ago
During my academic history with GIS/remote sensing/Geospatial (and still going on), I have seen how the same tools around the various fields in got plugged to in, mostly for the data pre-processing, processing and analyst. The sacro-saint trilogy, for me at leat, is GDAL/OGR, PROJ and GEOS. They are the building blocks behind QGIS, PostGIS and most of spatial libraries in various programming languages (Rasterio and Fiona in Python for example). On a more GIS analyst perspective, most of it can be done with ESRI or QGIS and if you can get you hand dirty PostGIS is an amazing tool that I which I had the time to invest way more (or even Spatialite, sqlite flavour of GIS in database). I really which than Shapefiles goes away for good and be superseded by GeoPackage coupled with GeoJSON. The web had a hard time to be a proper tool for visualisation on GIS/remote sensing due to the big size of the various files but Cloud-optimized GeoTiff and Vector Tiles are the game changers but came too late. Commercial vendors tends to go for the cloud and platform lock down as Planet and their super-powered jupyter notebook and access to the catalog. Same for Google Eart Engine. We see also stuff like CartoDB and the Database As A Service for geodata. In a world when Geodata are highly available with better resolution than never, they are so big to store somewhere than it can be crazy. Working on time series at country scale (or even world scale) require now a lot of store space and calculation power for high resolution images (Even just Sentinel or Landsat at 10m and 30m resolution respectively). Small-scale is easier than ever and lower resolution can be managed on a laptop. I remember when I had to work with time series dataset of Sentinel-1 and -2 images when we were preprocessing 10-12 Tb of data with a pretty long chain of operations to do before handing it out to another team for furter data analysis. Some academic lab are storing Petabytes of remote sensing data gather during the time. "Geospatial Industry" is way more than looking at ESRI or Google Maps/Earth. A lot of things can be and are done with various research and industry projects than require high integration with the spatial field and the temporal analysis. And the quality of what you have as tools vary highly between field, actually I am working for a PhD on a Bayesian spatio-temporal modelling with lattice data. It is the wild west and you need to cure a lot of R libraries to get what you need. I don't know how much you can do in Python because we had to avoid any MCMC method due to constraints on processing power available : low in fund and no easy access to calculator, we settled for R-INLA who is very interesting on its own.
perrygeo 6y ago
I started my career as a GIS analyst and was steeped in the "geospatial industry" for a few years. As I grew my career into software development, I found an entire world of innovative practices and technologies that had been largely hidden from view. Linux, Databases, HTTP, legit programming languages, software best practices... these things are largely absent from the GIS curriculum. In their place are bespoke, proprietary systems that solve the Geographic part well but are painfully isolated from the Information Systems that the rest of the world are working on. My biggest realizations came using tools like PostGIS, GeoPandas, Rasterio, and Numpy where geography is just another data type (albeit with some specialized functions to work with them) and well integrated with the rest of the software ecosystem. You can see it reflected in the salaries - A "GIS Programmer" will often make a fraction of what a "Software Engineer" makes. There's a reason for that - for most use cases, geospatial isn't unique enough to warrant a walled garden of technology.
- newyankee 6y ago
  R also has some decent spatial libraries.
geogra4 6y ago
I used to be heavily involved in geospatial and a whiz with the esri toolset. I worked at a software company that made a custom GIS tool for visualization. The problem is that it's underpaid and underappreciated. I had little transferrable experience and most GIS related jobs don't pay that much. For me my GIS job was mostly a time to learn programming and get a real it job
jonny383 6y ago
I really wish this trend of writing blog posts in 240 character chunks on Twitter would pass. It's so damb frustrating to read them.
- simonw 6y ago
  Weirdly I found I really like them. The thread format forces people to break their thoughts up into chunks - a little like bullet points on slides but 280 characters makes the chunks much more information-full than just a bullet. I can like or retweet specific interesting points as I read them. Individual tweets in a thread can include images or link previews or link to other tweets. What platform are you reading them on? I find them fine to read in the Twitter iPhone app - almost indistinguishable from a article elsewhere. Is that not the case outside of the app?
ReactiveJelly 6y ago
I mostly work on desktop programming, and I've dabbled in web stuff... And I don't know what GIS is. I see it talked about _a lot_ but I don't understand how there can be so much money and interest in geography. I understand web stuff being huge, because I'm online almost all day. But what _is_ GIS? Is it something businesses do and consumers don't see?
- deleted 6y ago
  [deleted]
- thinkingemote 6y ago
  https://en.wikipedia.org/wiki/GIS https://en.wikipedia.org/wiki/GIS GIS is everything location. From modelling pandemics to rainforest loss, to logistics, to Tinder and Pokemon Go.
  - Crespyl 6y ago
    Uber, Lyft, etc are other useful examples. I think some of the sort-of-obscurity/confusion comes from the fact that, as the OP tweet gets at, you don't really see a very visible "GIS Industry"/products most of the time, you just see tools that happen to have maps embedded in them, and those often require a lot of backend GIS work to power them. Finding a rideshare and planning your next oil well don't, at first, seem to have a lot in common, but on the backend there may be a surprising amount of overlap.
- PLenz 6y ago
  Everything has to happen somewhere, even online things, and GIS is they way to analyze it.
- rurban 6y ago
  GIS is when you rotate your x with y coordinates for your CAD drawings or maps. And less details, much less objects, more grander scale. Only points, polygons, polylines and markers.
zeku 6y ago
I entered college to get a Human Geography B.A. and go on to get my PhD. After realizing this meant I must leave my home area and my family(I live in a major US Metro but not one of the biiiig ones) AND the only jobs were Professor or Think Tank(very cool, very walled garden and I went to a state school) I decided to switch to Physical Geography B.S. My curriculum had 3 GIS classes and a Remote sensing class among other physical geo classes. After almost finishing this degree I realized it would leave me with only GIS Tech and hopefully GIS analyst positions to have which is a smallish market and might get "automated away". Also it doesn't pay super great and I was worried about my prospects if I didn't like it. So I dug bag in and also got my B.S. in Computer Science. I wanted to be that Geospatially focused programmer. So I graduate with my B.S. in Geography and another in CS. I get interviews with a couple of random defense contracts and directly for federal government, but I turn them down because honestly I don't want my work going to wars I might not agree with and I would need to leave my family. I also happened to get an interview at ESRI for fun when I still was deciding if I should leave my metro or not and they laughed at me because I didn't have an internship--something that is very common in Toronto, LA, NYC, but not common in my mid-US metro CS scene. So I now have a nice programming career solving "big data" problems completely away from geospatial, because the industry is tiny. I don't really know where I was going with this but this industry lacks opportunities for programmers compared to all the other stuff they could be doing. I'm someone who put years into wanting to break into the industry and do the tech work, if I wasn't the prime candidate for a junior geospatial programmer role then I don't know who is and the industry still couldn't get me in it.
- maw 6y ago
  > the only jobs were Professor or Think Tank Mostly offtopic: I initially misread that as "Professor of Think Tank" and I immediately understood it to mean something within the disciplines of military affairs, macroeconomics, some areas of law, a few others. This actually seems like an interestingly useful grouping, possibly more so than "STEM."
- vharuck 6y ago
  The disconnect is from one of the beliefs listed in the Twitter thread: "There is no 'geospatial industry,' only industries with spatial problems." I'm a statistician with a state health department and do some GIS work. If I were more skilled with geospatial analysis, there's no doubt the decision makers I work with would be delighted. If somebody wants to use GIS skills, they should keep an eye on IT or analysis roles. They might not specifically ask for it, but you can still sell it in the interview if it'll help solve their problems. If you're hoping to make the big bucks or work on cutting edge problems, then staying local will be difficult.
edleranalytics 6y ago
I have spent a fair amount of time learning GIS in undergraduate (urban studies program) and applying those skills in local government/real estate development. I have to agree with a lot of what he is saying. Many of the GIS skills I learned in ESRI during University pigeonholed me into the same few glorified data entry/basic spatial analysis roles. There are also a lot of people with this niche skill set, which makes getting these positions really challenging. There is an emerging field called Civic Technology that focuses more on building out geospatial applications for the public good. I hope more public policy/urban studies degree programs shift in this direction of building out tools for social change rather than learning the same few applications of ESRI ArcGIS.

170 points by luu 6y ago

This seems to have been written by someone unfamiliar with real estate, local government, natural resources, or any other traditional line of business that depends heavily on GIS. I don't know what my county pays for ArcGIS but it's probably substantial, and I suspect there are multiple IT employees devoted to maintaining the system. Replicate that across nearly every single political locality in the US, probably the world. The industry is huge. Also, "long form twitter" is dumb.

SOLAR_FIELDS 6y ago
I used to work for a local government (municipality) as part of their GIS department and we were spending about 10-15 grand a year on licenses IIRC. This was for an ArcGIS server license and 3 “power users” (ArcInfo), 5 “editors” (ArcEdit) and 5 “viewers” (ArcView). We were pretty small compared to most orgs, basically 3-4 people in the department at any given time. I distinctly remember the ArcGIS server license being a very significant portion of that number, maybe even half of it. Things may have changed since then (this was nearly a decade ago, before ArcGIS Pro licensing style was implemented), but probably not by much, you can probably extrapolate that ballpark and figure how much any actually decent sized org spends on Arc suite of products (a lot)
campchase 6y ago
Hi stickfigure, this is Joe from the dumb long form twitter thread you just wrote five sentences critiquing. Thanks for your feedback, I really appreciate it.

Waterluvian 6y ago

Two geography degrees meant I saw a lot of ESRI. In fact I won a national award and was flown out by them to meet Jack in San Diego. Everyone at ESRI I interacted with were all wonderful people. But their software is just brutally corporate and slow to respond to market changes. FOSS GIS software is pretty good but suffers from the usual lack of budget. Nevertheless, I’ve been able to push robotics + GIS packages for QGIS with ease. Reaching out to my ESRI contacts about “where’s your indoor and outdoor robotics suite offerings” received a response of only crickets. As for Google Maps. The biggest tell is that there’s almost no geospatial capability in the software. They don’t want you to analyze and understand. They just want you to consume “where is X”. It’s a standard Google vector for ads approach. I’m going to rant for a long time if I don’t stop here so I’ll leave it at: stop teaching so damn much ESRI and start teaching way more programming. The biggest regret of my education was that I was mostly taught how to use Geospatial tools rather than how to make my own.

craftinator 6y ago
When you say, "robotics + GIS packages", what exactly do you mean? I've had a little experience with ESRI products, and a lot of experience with robots, but I don't understand the connection here.
- Waterluvian 6y ago
  Mobile robots are highly instrumented spatial data collection platforms. Here’s a very brief high level view. Regrettably I only touch on the surface of what is considerably deeper. https://roscon.ros.org/2018/presentations/ROSCon2018_UnleashingGISToolbox.pdf https://roscon.ros.org/2018/presentations/ROSCon2018_Unleash... https://github.com/locusrobotics/qgis_ros https://github.com/locusrobotics/qgis_ros
  - craftinator 6y ago
    Ah I see, thank you for the follow-up!
chrisbroadfoot 6y ago
> As for Google Maps. The biggest tell is that there’s almost no geospatial capability in the software. They don’t want you to analyze and understand. They just want you to consume “where is X”. It’s a standard Google vector for ads approach. It was never meant for that. That is/was Google Earth Pro.
- samplatt 6y ago
  Rio Tinto runs some inhouse software that utilises (amongst other things) Google Earth Pro for analysis of open-pit minesite walls, constantly scanning for movement or changes that could lead to instability of a wall or road.
- campchase 6y ago
  A similar product, Google Earth Enterprise is now open source, amazingly. (http://www.opengee.org/ http://www.opengee.org/) Not to be confused with Google Earth Engine, which is the most shocking piece of technology I’ve come across in my career and is absolutely free if you can get access: https://earthengine.google.com/ https://earthengine.google.com/

artificialidiot 6y ago

I agree to all points. For point 6. Mapbox is a big contender for ESRI's current position. For 8. Openstreetmap, GDAL and associated open source ecosystem offers a solution to proprietary format and API of the month problem to a degree. It is not surprising that there is no money flowing if mapping is not a core business of the users. Aerial and satellite imagery is the heavily regulated and lucrative part of any mapping solution. If you are allowed and able to obtain that, you can probably get by just selling the raw imagery without geospatial processing.

redis_mlc 6y ago
I write and maintain some OpenLayers API (OSM) Javascript, used in a product. OpenLayers was selected after Google started charging 10x for their maps API, grayed out the existing tiles, then required a credit card on file. Since the API key can be "view sourced", that was totally a non-starter, along with the always-changing ToS. Way too many moving goal-posts. (Part of the reason non-free is a problem is that any testing ends up costing money, and the more testing you do, the more $$, which is a disincentive to do testing. But just staying on top of lopsided ToS gets old fast when you can just replace it.) Using a combination of OpenLayers and SO sample code, I've been able to do everything I wanted to do and am happy with the final result, but usually it's some of the toughest programming of the month.
jaakl 6y ago
Mapbox tries to replace Google maps, ESRI is different category which is only slowly challenged by enterprise vendors (Azure and other clouds) and FOSS: PostGIS and other sptially enabled databases which have more and more geo features, making specialized GiS to a niche.

TheSpiciestDev 6y ago

I've enjoyed working on an enterprise, GIS-focused application over the past few years (with no prior, formal geographic background) and one of the take-aways I always enjoy sharing with family and friends is the "point in polygon" question [0] and it's simplicity. I'm fun at parties. I have certainly learned how much more intuitive data is for users to explore when you offer (if you can) spatially. [0] https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_algorithm https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_a...

deleted 6y ago
[deleted]
CyberDildonics 6y ago
Are you really doing the test in the wikipedia article over and over? Why not convert a complex polygon to triangles? Things like this were established in computer graphics in the 70s.

jandrewrogers 6y ago

As someone deep into the cutting edge parts of the geospatial industry that few ever see, I’d quibble on some of the points but it is a broadly accurate description of the current state. The industry has been comfortably stagnant for many years. A critical distinction that explains much of what we see today is that geospatial analytics has almost no relationship to making maps at a technical level. There is a superficial similarity because they sometimes share presentation layer. All of the popular infrastructure and tooling was designed, either implicitly or explicitly, for making maps. If you want to analyze some physical world behavior using multimodal sensor data, the paucity of viable data infrastructure is insurmountable for most companies. Unfortunately, this is what most organizations want to do these days.

jld 6y ago

As a mid-career software engineer who would be interested in getting into the "geospatial industry", A) What technologies should I learn to allow me to get my foot into the industry? I have historically focused on being a UX focused web engineer. I'm sure that experience could surely transfer, but am interested in learning the technologies and/or understanding that is more core to geospatial problems. B) As someone who loves maps and imagery, but does not want to work for the military industrial complex, how do I avoid point #3. "In geo, you either die a hero or live long enough to make the majority of your revenue from defense and intelligence."

campchase 6y ago
A: Break it into two parts, vector and raster. For the former, focus on Mapbox and associated libraries/standards (VectorTiles, Leaflet, GeoJSON, etc.). JS is the name of the game in vector land—it’s amazing what you can do all on the front end now. For backend/DB tech, look at PostGIS. For the latter, I’d focus on Python to start. Look at GDAL (and it’s various Python interfaces/wrappers like rasterio), Cloud Optimized Geotiffs, and the SpatioTemporal Asset Catalog spec. In both cases, watch what the leading companies are putting out (Mapbox, Azavea where I work, Development Seed, etc.) and not what OGC is doing. One is a leading indicator, the other is a lagging one. Sorry OGC, I still love you and you play an important role. B: Get comfortable with the idea of working with Defense. For better or worse, they’re the ones with the budget to take on humanitarian work, disaster response work, even climate change adaptation work. That’s my advice. Possible to avoid it altogether, but only if you work on an app that is domain-focused (eg Trulia) and incidentally geospatial.
vharuck 6y ago
B) Look into healthcare systems or public health. Defense holds sway because of its budget size, but it's not the only fat cat in town. Epidemiologists are nice, but they're expensive, rare, and usually bad at GIS programming. Most governments will happily hire a GIS analyst. They can't get enough maps and data dashboards. And most of the ones they do have are terrible (misleading, slow, convoluted, or just plain not useful).

campchase 6y ago

Hey, this is Joe from twitter thread linked. I didn’t post this to HN but it’s nice to see it on here. I was wondering why a weeks-old tweet suddenly started popping off... One thing I’d like to note—-I have learned a tremendous amount from other people responding to the opinions enumerated in that thread. Some things I’ve even started to question. So I’d love to hear your ideas—-especially if you disagree with me. Just please do it constructively. Sometimes seeing your ideas thrust onto HN without being prepared for it can be a harrowing experience and I’d rather this time around be a productive one. Much love to the brilliant minds of HN -Joe

sankalp221 6y ago
I think your tweet is taking off here because Facebook's acquisition of Mapillary is currently trending on the front page. My guess is lots of people here are trying to understand the industry. https://news.ycombinator.com/item?id=23605362 https://news.ycombinator.com/item?id=23605362
- campchase 6y ago
  Wait what? I didn’t know about that post either.
  - campchase 6y ago
    I don’t think anyone has ever posted something I’ve written to HN, let alone two things in the same day.
atupis 6y ago
Any good resourse to start with geospastial stuff? personally I am from data science background.
- Stevvo 6y ago
  I found this to be an easy & fun project; https://blog.patricktriest.com/game-of-thrones-map-node-postgres-redis/ https://blog.patricktriest.com/game-of-thrones-map-node-post... If not a game of thrones fan, it's adaptable to something real world.

csteubs 6y ago

I'd argue another point; satellites are a major bottleneck to GIS as we're familiar with. With only ~300 or so commercial imaging satellites in orbit, current revisit rates are sufficient for most use cases. Maybe one or two images/day for X point depending on weather. Low earth orbit is getting crowded, fuel to raise and lower orbit is limited onboard, expensive maintenance cycles, etc. I'm working on a startup trying to solve this problem now. The idea is to "crowdsource" aerial imagery using a network of camera arrays on 80,000 commercial flights/day. We stream the video feed back to earth using in-flight wifi and overlay the visual diff to create a map that shows minute-level changes. There are huge built-in cost savings that come with not having to develop, launch and maintain a satellite constellation, and with a vastly higher revisit rate (as high as 4,532/day in some areas). We're at notasatellite.com

oh_sigh 6y ago
I'm clueless in this space but I'd imagine the cost of adding these sensors(installation, ongoing maintenance, certification etc) to airplanes is non-negligible? I assume you're targeting small personal aircraft for the sensor array and not some airline with a fleet of 737s?
- csteubs 6y ago
  The sensors are mounted internally to the window in order to avoid any sort of external retrofit work, tunnel testing, FAA approval, etc. The MVP is actually just an app that uses the phone's camera to the same effect. We're talking to a number of carriers and exploring all the other deployment options as well (including a frequent flyer revenue share program).
sgt 6y ago
Streaming the video feed using the onbord wifi? I think you are overestimating the quality of the Internet connections available on flights.
- notahacker 6y ago
  Having worked on inflight connectivity apps and Earth Observation respectively, for a couple of startups I'm inclined to agree overall. Workarounds would be: [i] ground sync, which would be fine for a lot of use cases where higher revisit rate and sometimes less cloud cover is sufficient to make the imagery valuable, and open up non-wifi flights. Ground sync can be over regular 4G or even swapping flash drives [some W-IFE boxes update this way, though infrequently]. [ii] Getting dedicated bandwidth from the satellite/a2g internet provider, who can potentially allow you a lot more than is made available to an individual consumer, but obviously at a hefty charge. [iii] using streaming codecs designed for low kbps rates - though this is obviously lossy, and pixel perfection matters for a lot of EO use cases.

jhoechtl 6y ago

ESRI is essentially the Oracle of GIS. To big to fail and to powerful to be wiped away. On the sweet spot of acceptably behind the trends. Yet nobody go ever fired because of acquiring ESRI Software.

littlestymaar 6y ago

> The most successful and ambitious mapping project of all time, Google Maps Google map is indeed a commercial success, but it's far from “the most successful and ambitious” when it comes to mapping itself, quite the opposite. State projects (even those developed before satellites where a thing[1]) are way better, and even openstreetmap achieves a better coverage and higher maps quality. It's the “most successful and ambitious” commercial project, which is coherent with the rest of the thread. [1]: for instance: https://www.sovietmaps.com/ https://www.sovietmaps.com/

ramraj07 6y ago
Do you have a citation for OSM? My experience in first world countries especially where Google had ground truth project are still much better in Google than osm.
- morsch 6y ago
  Casually browsing OSM in my area, I see individual park benches, garbage cans including type, road surface material, doorways to buildings, power lines, location of springs. And that's just what Osmand is rendering, I'm sure I'm just scratching the surface. Google doesn't even attempt to map most or all of these things here, much less expose this information. OSM attempts to build a much better map, and for my region, easily succeeds. That's not to discredit Google Maps. Almost all of the time, I don't need the best map, I need the kind of yellow pages, yelp, sightseeing guide, live traffic and transport, navigation mashup Google is projecting onto their map with great skill. I think it's their best product.
- littlestymaar 6y ago
  Maybe if you limit your definition of “first world countries” to urban areas, but I live in a somewhat rural place (not the middle of nowhere, I have 4G network and 200Mbs internet) and my street, built in 2011, doesn't appear on Google Maps. In fact it looks like everything built after 2010 in my region is missing from Google Maps. And never ever use Google maps if you're planning a bike trip, unless you really want to get into troubles. As a host on warmshowers.org for the past four years, I've heard tons of horror stories…

atrocious 6y ago

My academic and work history is with GIS, but I've spent the last 4 years in fintech. What does the cutting edge of GIS look like in tech nowadays?

growlist 6y ago
Take a look at what OGC are working on.

notagoodidea 6y ago

During my academic history with GIS/remote sensing/Geospatial (and still going on), I have seen how the same tools around the various fields in got plugged to in, mostly for the data pre-processing, processing and analyst. The sacro-saint trilogy, for me at leat, is GDAL/OGR, PROJ and GEOS. They are the building blocks behind QGIS, PostGIS and most of spatial libraries in various programming languages (Rasterio and Fiona in Python for example). On a more GIS analyst perspective, most of it can be done with ESRI or QGIS and if you can get you hand dirty PostGIS is an amazing tool that I which I had the time to invest way more (or even Spatialite, sqlite flavour of GIS in database). I really which than Shapefiles goes away for good and be superseded by GeoPackage coupled with GeoJSON. The web had a hard time to be a proper tool for visualisation on GIS/remote sensing due to the big size of the various files but Cloud-optimized GeoTiff and Vector Tiles are the game changers but came too late. Commercial vendors tends to go for the cloud and platform lock down as Planet and their super-powered jupyter notebook and access to the catalog. Same for Google Eart Engine. We see also stuff like CartoDB and the Database As A Service for geodata. In a world when Geodata are highly available with better resolution than never, they are so big to store somewhere than it can be crazy. Working on time series at country scale (or even world scale) require now a lot of store space and calculation power for high resolution images (Even just Sentinel or Landsat at 10m and 30m resolution respectively). Small-scale is easier than ever and lower resolution can be managed on a laptop. I remember when I had to work with time series dataset of Sentinel-1 and -2 images when we were preprocessing 10-12 Tb of data with a pretty long chain of operations to do before handing it out to another team for furter data analysis. Some academic lab are storing Petabytes of remote sensing data gather during the time. "Geospatial Industry" is way more than looking at ESRI or Google Maps/Earth. A lot of things can be and are done with various research and industry projects than require high integration with the spatial field and the temporal analysis. And the quality of what you have as tools vary highly between field, actually I am working for a PhD on a Bayesian spatio-temporal modelling with lattice data. It is the wild west and you need to cure a lot of R libraries to get what you need. I don't know how much you can do in Python because we had to avoid any MCMC method due to constraints on processing power available : low in fund and no easy access to calculator, we settled for R-INLA who is very interesting on its own.

perrygeo 6y ago

I started my career as a GIS analyst and was steeped in the "geospatial industry" for a few years. As I grew my career into software development, I found an entire world of innovative practices and technologies that had been largely hidden from view. Linux, Databases, HTTP, legit programming languages, software best practices... these things are largely absent from the GIS curriculum. In their place are bespoke, proprietary systems that solve the Geographic part well but are painfully isolated from the Information Systems that the rest of the world are working on. My biggest realizations came using tools like PostGIS, GeoPandas, Rasterio, and Numpy where geography is just another data type (albeit with some specialized functions to work with them) and well integrated with the rest of the software ecosystem. You can see it reflected in the salaries - A "GIS Programmer" will often make a fraction of what a "Software Engineer" makes. There's a reason for that - for most use cases, geospatial isn't unique enough to warrant a walled garden of technology.

newyankee 6y ago
R also has some decent spatial libraries.

geogra4 6y ago

I used to be heavily involved in geospatial and a whiz with the esri toolset. I worked at a software company that made a custom GIS tool for visualization. The problem is that it's underpaid and underappreciated. I had little transferrable experience and most GIS related jobs don't pay that much. For me my GIS job was mostly a time to learn programming and get a real it job

jonny383 6y ago

I really wish this trend of writing blog posts in 240 character chunks on Twitter would pass. It's so damb frustrating to read them.

simonw 6y ago
Weirdly I found I really like them. The thread format forces people to break their thoughts up into chunks - a little like bullet points on slides but 280 characters makes the chunks much more information-full than just a bullet. I can like or retweet specific interesting points as I read them. Individual tweets in a thread can include images or link previews or link to other tweets. What platform are you reading them on? I find them fine to read in the Twitter iPhone app - almost indistinguishable from a article elsewhere. Is that not the case outside of the app?

ReactiveJelly 6y ago

I mostly work on desktop programming, and I've dabbled in web stuff... And I don't know what GIS is. I see it talked about _a lot_ but I don't understand how there can be so much money and interest in geography. I understand web stuff being huge, because I'm online almost all day. But what _is_ GIS? Is it something businesses do and consumers don't see?

deleted 6y ago
[deleted]
thinkingemote 6y ago
https://en.wikipedia.org/wiki/GIS https://en.wikipedia.org/wiki/GIS GIS is everything location. From modelling pandemics to rainforest loss, to logistics, to Tinder and Pokemon Go.
- Crespyl 6y ago
  Uber, Lyft, etc are other useful examples. I think some of the sort-of-obscurity/confusion comes from the fact that, as the OP tweet gets at, you don't really see a very visible "GIS Industry"/products most of the time, you just see tools that happen to have maps embedded in them, and those often require a lot of backend GIS work to power them. Finding a rideshare and planning your next oil well don't, at first, seem to have a lot in common, but on the backend there may be a surprising amount of overlap.
PLenz 6y ago
Everything has to happen somewhere, even online things, and GIS is they way to analyze it.
rurban 6y ago
GIS is when you rotate your x with y coordinates for your CAD drawings or maps. And less details, much less objects, more grander scale. Only points, polygons, polylines and markers.

zeku 6y ago

I entered college to get a Human Geography B.A. and go on to get my PhD. After realizing this meant I must leave my home area and my family(I live in a major US Metro but not one of the biiiig ones) AND the only jobs were Professor or Think Tank(very cool, very walled garden and I went to a state school) I decided to switch to Physical Geography B.S. My curriculum had 3 GIS classes and a Remote sensing class among other physical geo classes. After almost finishing this degree I realized it would leave me with only GIS Tech and hopefully GIS analyst positions to have which is a smallish market and might get "automated away". Also it doesn't pay super great and I was worried about my prospects if I didn't like it. So I dug bag in and also got my B.S. in Computer Science. I wanted to be that Geospatially focused programmer. So I graduate with my B.S. in Geography and another in CS. I get interviews with a couple of random defense contracts and directly for federal government, but I turn them down because honestly I don't want my work going to wars I might not agree with and I would need to leave my family. I also happened to get an interview at ESRI for fun when I still was deciding if I should leave my metro or not and they laughed at me because I didn't have an internship--something that is very common in Toronto, LA, NYC, but not common in my mid-US metro CS scene. So I now have a nice programming career solving "big data" problems completely away from geospatial, because the industry is tiny. I don't really know where I was going with this but this industry lacks opportunities for programmers compared to all the other stuff they could be doing. I'm someone who put years into wanting to break into the industry and do the tech work, if I wasn't the prime candidate for a junior geospatial programmer role then I don't know who is and the industry still couldn't get me in it.

maw 6y ago
> the only jobs were Professor or Think Tank Mostly offtopic: I initially misread that as "Professor of Think Tank" and I immediately understood it to mean something within the disciplines of military affairs, macroeconomics, some areas of law, a few others. This actually seems like an interestingly useful grouping, possibly more so than "STEM."
vharuck 6y ago
The disconnect is from one of the beliefs listed in the Twitter thread: "There is no 'geospatial industry,' only industries with spatial problems." I'm a statistician with a state health department and do some GIS work. If I were more skilled with geospatial analysis, there's no doubt the decision makers I work with would be delighted. If somebody wants to use GIS skills, they should keep an eye on IT or analysis roles. They might not specifically ask for it, but you can still sell it in the interview if it'll help solve their problems. If you're hoping to make the big bucks or work on cutting edge problems, then staying local will be difficult.

edleranalytics 6y ago

I have spent a fair amount of time learning GIS in undergraduate (urban studies program) and applying those skills in local government/real estate development. I have to agree with a lot of what he is saying. Many of the GIS skills I learned in ESRI during University pigeonholed me into the same few glorified data entry/basic spatial analysis roles. There are also a lot of people with this niche skill set, which makes getting these positions really challenging. There is an emerging field called Civic Technology that focuses more on building out geospatial applications for the public good. I hope more public policy/urban studies degree programs shift in this direction of building out tools for social change rather than learning the same few applications of ESRI ArcGIS.