10 ms·
Migrating away from Google Maps and cutting costs
- ckastner 8y ago> In under 30 minutes we migrated our code to Stadiamaps without prior knowledge of their API, including deployment. This surprised me. I'm completely ignorant with respects to geoinformation providers, but for some reason, I somewhat expected vendor lock-in to be much tighter.
- _Codemonkeyism 8y agoGeo coding took longer as the meta information about city districts etc. which needs to be mapped is different, but changing maps was easy. That said Open Cage was nice as their data was cleaner and we didn't have to manually change data as we had to do with Google.
- mtmail 8y agoOpenCage (https://opencagedata.com/ https://opencagedata.com/) founder here if anybody has questions. Putting country hierarchies in a set of keys is a challenge for all geocoders and the output differs slightly (though we have a google-compat mode). Sometimes it's just road vs street or town vs city. England has parishes, Paris has arrondissements (equivalent to a city within a city), Berlin is a city, county and state (same outer boundaries) for example.
- everybodyknows 8y agoFAQ has: >Just pass the coordinates as a latitude and a longitude, separated by either a comma or a (URL encoded) space and the API will automagically work out that you want to reverse geocode. Is the reverse geocoding based on 1. government-source boundary data, or 2. proximity to tagged points from OSM or other non-authoritative sources?
- mtmail 8y agoBoth where government provides open boundary data, though in most cases that would've been imported into OpenStreetMap already https://opencagedata.com/credits https://opencagedata.com/credits
- megous 8y agoThey probably have a nicely decoupled code and not too complex an integration. If you're just showing some icons on the map, all you need to know is where to place them on the overlay layer and then query and set the bounding box for the map. That's pretty much it. The rest can be in your own code.
- _Codemonkeyism 8y agoExactly.
- josephmosby 8y agoThe core principles of geoinformation are the same across providers. Here is a map which is functionally geographic locations in specific locations, you place your own special bits in "layers" on the top. Leaflet.js is a great example of a library that works pretty much the same regardless of which provider you choose. The competitive set is differentiated in some of their niche work (like specific-device-embedded maps, such as the ones in Tesla), but the first principles for the common developer are all the same.
- dmortin 8y agoAnother problem with Google that you cannot really experiment with it for free. You have to add your credit card and there is no concept of using only the free quota. E.g. I'd put up an experimental site which uses google maps free qouta only and I don't want to pay anything, since it's only an experiment, I'd be happy with my access blocked automatically for the rest of the month after the free quota is exhausted. But AFAIK it cannot be done. You have to track you spendings and shut down the site manually when you are near the end of the free quota. Google in the past helped people experimenting with its platform. This is not the case anymore with google maps.
- deleted 8y ago[deleted]
- tom-- 8y agoRight, and to even get an API key for experimentation you have to provide a credit card number, which a lot of students wouldn't have.
- i_cant_speel 8y agoAt this point I don't think they're really interested in accommodating students.
- BrandoElFollito 8y agoCouldn't you provide an e-card with a 1€ limit? You would not be charged more but then would maybe be in breach of the contract passed with Google (?)
- discreditable 8y agoSounds like a good way to get your Google account banned.
- Yizahi 8y agoWhich will then proceed to ban all related accounts that they suspect are yours too (so they try to ban a person, not an account). And god help you if it was work account or a linked one.
- _Codemonkeyism 8y agoAuthor here ( @codemonkeyism ), ready to answer questions. Blog seems down, we're haven't been slashdotted for a long time, sorry! [Added some more caching, site works again slowly]
- mabbo 8y ago> Thanks to Google for giving us free Maps for some years This attitude is something I truly appreciate. So often posts like these are angry rants about 'how dare they do this to me'. You seem to presume good intentions, which should be the default reaction in life.
- ed_balls 8y agoOne could argue that Google used search revenue for predatory pricing, which breaks anti-trust laws, to subsidise Maps and other services in order to build a monopoly.
- rejschaap 8y agoIt seemed a bit passive aggressive to me, but maybe I'm reading into it too much.
- _Codemonkeyism 8y agoSorry English isn't my native tongue, didn't intend it to be passive aggressive but meant it.
- magduf 8y agoIt kinda reminds me of someone thanking a cocaine dealer for giving them some free hits.
- rashomon 8y agoAny way you could copy the text in here or are there more relevant graphics?
- deleted 8y ago
- blantonl 8y agoI've migrated much of our mapping capabilities to MapBox's API.. but we're still facing mapping bills in the hundreds of dollars. My next step is going to be to setup a caching Open Street Map tile server and using https://leafletjs.com/ https://leafletjs.com/ javascript library. The costs now for mapping solutions are going to drive us to run our own GIS infrastructure.
- Waterluvian 8y agoBe sure you've considered OpenLayers if you're going to be doing GIS and not just mapping. It's my go-to for the front end of all my Geomatics-Robotics tools.
- linsomniac 8y agoWe also looked at MapBox but ended up going with local tile server from OpenStreetMap data. We are using the geofabrik.de data bundles, mod_tile and renderd to serve up the tiles, and we pre-generate the tiles so all our requests are served from disk rather than being rendered (which really slows down mapping). We are lucky that we are only dealing with one state, so our data is around 25GB for map tiles and another 10GB for "street overlay" tiles that we lay on top of aerial tiles that we generate from NAIP data. We customized the CartoCSS a bit to get the maps closer to the way we wanted them. For example, we took out building outlines, and reduced the frequency of highway shields, which required removing exit numbers. It's a good solution for us, but we already had some mapping experience and lots of system deployment experience, so YMMV.
- thesimon 8y agoGoogle Cache: https://webcache.googleusercontent.com/search?q=cache:https://www.eventsofa.de/campus/migrating-away-from-google-maps-and-cutting-costs-by-99/&rlz=1C1GCEA_enDE769DE769&strip=0&vwsrc=0 https://webcache.googleusercontent.com/search?q=cache:https:...
- _Codemonkeyism 8y agoThanks.
- imaginenore 8y agoYeah, but Stadia Maps aren't that good. They have really bizarre issues with labeling. For instance, they would display "Bridgeport", but not "New York" at some zoom levels. And Long Island looks completely disconnected, and Manhattan is missing altogether. https://i.imgur.com/O6v1X03.png https://i.imgur.com/O6v1X03.png
- wlkr 8y agoI don't think that's their fault but issues with the map services they rely on (which is OSM from what I gather [0]). I remember this being discussed recently on HN but can't find an article/thread right now. [0]: https://news.ycombinator.com/item?id=17010831 https://news.ycombinator.com/item?id=17010831
- ianthetechie 8y agoYes, It's a mix of underlying data and Mapbox GL. This is an active area of development both on our end and in the Mapbox open source community.
- dbatten 8y agoIn case it helps anybody else - we were recently looking for alternatives to Google Maps for reverse geocoding. We took property data from a local county, ran the centroids of a bunch of randomly selected parcels through 4 different reverse geocoding services, and compared the addresses we got back to the actual address of the parcel, as listed by the county. Google was 97% accurate, at $4 per 1000 requests. Mapbox was 95% accurate, at $.50 per 1000 requests. TomTom was 73% accurate, at $.42 per 1000 requests. Location IQ (one of many providers simply running the open-source Nominatim project) was 12% accurate, at $.03 per 1000 requests. To be fair to Location IQ / Nominatim, they had the right street the vast majority of the time, but were usually wrong about house number due to interpolating between address boundaries at cross streets defined in Census Data. If you need exact addresses, Nominatim probably isn't for you, but if you need a general location, it might work fine. Also - this was one county of test data, so take it with a grain of salt. Nevertheless, it gave us confidence that we could move away from the Google APIs, save 90% of our costs, and still have a high level of accuracy.
- perfmode 8y agowould it be possible for you to generate an “accuracy map” and select a provider based on an cost/accuracy trade-off, using a cheap provider in the common case, but using the most accurate provider in regions where the best provider is “much better”. probably not worth the effort
- dbatten 8y agoTo make a nation-wide accuracy map, you'd need to compile property data from every county (or equivalent) in the nation. Assuming every county publishes it publicly (I'm sure some don't), it would still be a ton of work... The real catch is this, though: if you actually pulled it off, you would have just built a nearly flawless reverse geocoder, and you wouldn't need to use an external API at all. You could just look locations up in your huge property data set.
- perfmode 8y agocouldn’t the OP use the same method to determine accuracy that they used to compare providers? it doesn’t need to be an accurate decider, just “good enough” in a majority of cases.
- narrowtux 8y agoWe did something similar and used Maptiler.com's vector tile server offering + MapboxGL.js on the frontend. Using mapbox's own backend would've been too expensive for us since our app is able to do asset tracking, a use case almost all of the bigger mapping providers (we checked google, bing, here, mapbox) want to earn an extra buck on.
- lukeqsee 8y agoStadia Maps founder here to answer any questions!
- pg_bot 8y agoAny reason why you wouldn't self host your tiles using something like open map tiles?[0] If you're geocoding mostly US addresses it's actually not impossible to roll your own geocoder* . If you're insane enough, I'm sure you could find some German spatial data and follow along with how the TIGER geocoder loads and normalizes information if you wanted to run an entire GIS operation in-house. [0]: https://openmaptiles.org https://openmaptiles.org * Provided you're familiar with PostgreSQL, and PostGIS.
- snarfy 8y agoIt makes sense if it's your core competency, but when it's not it's usually worth it to pay someone else to deal with it, like email.
- _Codemonkeyism 8y agoNot VC funded therefor no money (sadly) for such engineering, we focus on core features for our customers and only write or setup stuff we could not get anywhere else. But would love to do some deeper GIS and mapping on top of PG.
- phirschybar 8y agoOur experience with the Google cost increase was similar. We were paying around $550/mo for the GMaps Platform pre-price-change. Over 80% of this cost was for dynamic maps usage. The rest was for use of the Google Places API. When I got the notification in May that the rates were increasing, I didn't take it that seriously. We have a profitable company and I would have been more than happy to pay Google twice their rate for the premium services they offer. Nobody ever expects a sudden rate increase to be more than 20%, 30% or 40%, right???! It was in July that the gravity of the situation hit me. I was seeing tweets and articles lamenting the rate increase. I thought to myself "huh, I better check this out". I did some quick math using the new rate card and nearly had a heart attack! Our bill was estimated to be ~$14,500 and the CLOCK WAS TICKING. We were facing a 2600% rate increase and had only 2 months to figure out a game plan. Our business was on the line! I immediately determined that we were eligible for bulk pricing. However, Google will not sell you the bulk rates directly. You have to contract with an authorized 3rd party (re-seller, basically) to get the rates. So, we found a reseller and locked in the bulk rates. That brought our estimate down to around $12,000/mo. Better, but still a huge shock. The next step was optimization. There was no way for us to reduce dynamic maps usage because it is such a core part of our products. So, we cut off almost all of our Places API usage and started using other services. Our estimated bill was now down to $9,000. That's where we are today. We just got our first full month bill. It was a huge hit to our business, wiping out a significant portion of our profitability. To be frank, I am pissed about this. I was more than happy to pay Google far more than they were charging us. We were always under the impression that rates would increase one day. But, to force an increase of this magnitude with such a short amount of lead time is pretty f&^%ing sh*&$tty coming from a billion dollar giant. We're starting to test other mapping services. I met with a team member from Mapbox last week and am planning on testing their platform in December. Their quote for our usage needs? $550/mo.
- dracodoc 8y agoIn case it's not known by everybody, you can geocode(maybe also reverse geocode, I didn't check) US addresses in batch with TIGER census data and PostGIS. It's not directly relate to topic (dynamic maps, online geocoding), but somewhat relevant. With about $300 in Amazon EC2 I geocoded 18 million addresses in 3 months. You probably can do it with your own PC with similar time (the only hardware requirement is RAM and SSD). http://dracodoc.github.io/tags/Geocoding/ http://dracodoc.github.io/tags/Geocoding/
- wyldfire 8y agoDid you do any verification to show the effective accuracy/spatial resolution?
- dracodoc 8y agoThere is an accuracy score in postgis tiger geocoder plugin. Usually a low value mean a pretty good match. Depend on address format and quality, 70-80% of addresses got a good match. For resolution you cannot expect too much because street numbers are usually interpolated, and TIGER database definitely will not be as good as best commercial database. However you don't have many other solution if you have millions address to geocode.
- BattyMilk 8y agoPiggy backing off this thread for my own question. I've a small webapp to show me my elevation based on my lat/lon. I built it with Google's API as I've used it before and it's what I know. It doesn't get much traffic so it's unlikely to ever cost much once my free credit expires but I'd like to explore other options. What alternative (preferably free) services are there out there? I need to be able to pass lat/lon and get back the elevation above sea level of that point on earth.
- mtmail 8y agoWe (https://opencagedata.com/ https://opencagedata.com/) have such an API in beta. We're not happy with the performance yet, but might be enough for your use-case.
- Doctor_Fegg 8y agoI've been doing some interesting experiments with elevation compression recently that make it realistic to have large swathes of the earth memory-resident. If you're interested you know where to find me!
- ryandrake 8y agoThat would depend on what your resolution and accuracy requirements are. Ground level or treetop? If you are willing to DIY there are freely available DEM (Digital Elevation Model) datasets that might work for you. For example ASTER[1] has global 90 meter resolution and 30 meter resolution in the USA. There are smaller, embeddable DEMs with ~1km resolution. Lots of options out there. Grab a coffee and start with an internet search for DEM. 1: https://en.m.wikipedia.org/wiki/Advanced_Spaceborne_Thermal_Emission_and_Reflection_Radiometer#ASTER_Global_Digital_Elevation_Model https://en.m.wikipedia.org/wiki/Advanced_Spaceborne_Thermal_...
- BattyMilk 8y agoCheers, I'll take a read. I'm looking for ground level (I use it to calibrate the barometer on my watch at the start of a hike/trail run/climb)
- linsomniac 8y agoHas anyone figured out WHY Google maps did this double-digit multiplier price increase? Were they vastly under pricing it before? Did they want to stop providing it so they priced it at a point that would make almost everyone go away? I'm just curious about their motivation and surprised that this doesn't seem to have been figured out or even discussed.
- ianthetechie 8y agoMy guess is that they realized the enterprise market was more profitable for them, so they decided to focus more exclusively there. They know their enterprise customers aren't about to jump ship, so they can do pretty much as they please due to their dominant market positioning.
- pmontra 8y agoWhat most (all?) of those alternatives are missing is the satellite view and street view. I look at it almost every time I search for a place to get an idea of the surroundings for when I'll be there. However links to the same location on Google Maps can fix that.
- mtmail 8y agoGoogle invested 100s of millions driving streetview cars around the world. That's hard to match. "Street View cars have traveled almost 10 million miles, covering every single continent and 83 countries in total" (http://geoawesomeness.com/googles-street-view-turns-10/ http://geoawesomeness.com/googles-street-view-turns-10/) Mapbox has satellite view