9 ms·

Replacing Mapbox with open source solutions

613 points by gaws 4y ago

tpmx 4y ago
Wasn't Mapbox once the low-cost option?
- usrusr 4y ago
  And since the state of the open source map rendering art is consisting almost exclusively of pre-licence-change forks of mapbox code these days, they kind of still are. I wonder if Strava's recent acquisition of fatmap.com eventually boils down to them cutting ties with mapbox via acqui-hiring some mb-libre excellence. I could imagine Strava, due to the extreme map-centric nature of their product, bleeding quite considerable money to mapbox.
  - kylebarron 4y ago
    I didn't think Fatmap was based on mapbox at all? I thought it was their own custom renderer
    - gorbypark 4y ago
      I’m interpreting the comment above to mean Strava was/is a user of Mapbox and bought Fatmap in an effort to wean themselves off of Mapbox and use “inhouse” (Fatmap) mapping tech.
    - usrusr 4y ago
      I took a superficial glance at what the site is loading and iirc there were plenty of the usual suspects in there. And it would be silly to do it any other way, because their product is what they do with maps, not winning some NIH purity award.
  - lukeqsee 4y ago
    MapLibre has made really good strides (especially on the web) since the fork—3d-terrain support, more tile sources, other DX improvements (e.g., conversion to TypeScript). https://maplibre.org/maplibre-gl-js-docs/api/ https://maplibre.org/maplibre-gl-js-docs/api/
- deleted 4y ago
  [deleted]
- Operyl 4y ago
  Yup, which disrupted the market, and now it's not.
  - klysm 4y ago
    1. Subsidize low costs with venture capital and take on a ton of customers at a loss 2. Lock those customers into your API somehow. Enough code is effectively lock in for a while. 3. Change your pricing to be profitable. 4. Hopefully get acquired before all your customers leave
    - tpmx 4y ago
      They seem to be at step 3 at the moment.
      - Nemo_bis 4y ago
        Usage seems to have peaked in April 2022 https://trends.builtwith.com/mapping/Mapbox https://trends.builtwith.com/mapping/Mapbox
- gorbypark 4y ago
  They used to be low-cost _and_ open source. Now they're source available and not interested in anything but big clients. I was using Mapbox for all my clients before their unopen sourcing, and I tried reaching out three times to ask about options for one of my clients. It wouldn't have been a huge amount of money, probably a few hundred per month, but they never, ever reply. That client wasn't even using any of their server resources, they are 100% self hosted, it's just that Mapbox now charges even for using their client. Anyways, I was trying to give them money, and they wouldn't even acknowledge I exist. I switched that client over to Maplibre and never looked back. The other thing that annoys me is that they've more or less abandoned all their open source repos without even as much of a notice in the README. All these amazing tools sitting there with open pull requests and no one responding for years. If they just did a clean break, put a notice on everything they weren't going to touch again, it would have been a lot better/easier for the community to fork and move on with their lives.
  - shawn-butler 4y ago
    Mapbox was born from open source tools & culture. They were always active and supporting at FOSS4G. I wonder what has changed?
    - jonas21 4y ago
      VC funding? I think they were funded mostly by grants and their own revenues for the first few years. Then they started taking VC money (over $300M to date).
      - dataviz1000 4y ago
        They were once Development Seed [0], a company that specialized in making geo spatial visualizations for the UN, NGOs and nonprofits. Development Seed was a large contributor of code and modules to Drupal 6 which they left. In the beginning, their open source mapping and geo spatial visualizations projects were mostly funded by grants from the Knight Foundation which funded news related projects. MapBox came from an in-house tool they developed to create and manage map overlay tiles. (I could be making this all up.) [0] https://developmentseed.org/ https://developmentseed.org/
lukeqsee 4y ago
This is really awesome to see. I'm involved in many of these projects (helped get MapLibre started, wrote a library supporting PMTiles, have used OpenMapTiles for years), and to see open-source, open-data win in the end is really satisfying. If anyone has any questions about how you could do this for yourself, I'm happy to answer! (I have a company offering SaaS maps, but we know SaaS isn't for everyone, and are happy to point people the right direction when self-hosting is what you want.)
- jddj 4y ago
  Is there a standard folder tree of tiles -> PMTiles archiver or is that the wrong question to be asking?
  - lukeqsee 4y ago
    I don't think so. There's this: https://github.com/protomaps/go-pmtiles https://github.com/protomaps/go-pmtiles to get from MBTiles and there's a few xyz folder tree tools to get mbtiles from that structure (e.g., https://pypi.org/project/mbutil/ https://pypi.org/project/mbutil/). So you can get there, but I don't know of a good straight-line right now.
    - jddj 4y ago
      Thanks a lot. I developed and support a standalone desktop application which contains customer-specific tiled maps, and it can get unwieldy. A pivot at some point to these types of archives is an option, but I'd like to roll forward existing customers as well.
      - bdon 4y ago
        If you have access to SQLite in your desktop application then there isn't any big wins from using PMTiles over MBTiles; PMTiles does have some special design around tile-deduplication but this is possible in MBTiles too by using table views.
  - bdon 4y ago
762236 4y ago
If all of the tiles are coming from a single file (via PMTiles), how are they cached by the client? Do clients cache byte ranges, or do you configure things to give the appearance of serving separate files?
- lukeqsee 4y ago
  You typically use byte-range caching in libraries—either using an edge worker (a la Cloudflare) to create separate GET requests from the client's perspective or directly in the client.
  - cancan 4y ago
    i think you can also directly call them from the maplibre lib itself using the pmtiles adapter https://github.com/protomaps/PMTiles https://github.com/protomaps/PMTiles https://github.com/protomaps/PMTiles/blob/main/js/examples/maplibre.html#L24 https://github.com/protomaps/PMTiles/blob/main/js/examples/m...
- bdon 4y ago
  The renderer used in the linked article (MapLibre) has an internal tile cache used when you return to a previously loaded area of the map. For the non-tiled parts of the PMTiles file, like metadata and the directories that store Z/X/Y to offset information, those are LRU cached by the PMTiles implementation. The most widely deployed right now is for TypeScript/Node. The "client" can either be web browser or inside a runtime like Lambda/Cloudflare Workers. On those the cache will persist across invocations using the ephemeral memory of the serverless function. A level deeper, browsers vary in how they cache Range requests; IIRC Firefox does not cache while others will treat Range as if it was any other HTTP header.
kabes 4y ago
Worth mentioning that openlayers is also really extending their webgl rendering capabilities lately.
- lukeqsee 4y ago
  There's also support for MapLibre GL JS style specs in the works! Excited to see another stable vector tile & style renderer added to the ecosystem.
  - deleted 4y ago
    [deleted]
T3RMINATED 4y ago
[dead]
wipfli 4y ago
If you like PMTiles you should consider supporting Brandon financially as a GitHub Sponsor. Brandon is a solo-developer who builds Protomaps alone and we get all his great stuff for free... https://github.com/sponsors/protomaps https://github.com/sponsors/protomaps
- lukeqsee 4y ago
  You can also sponsor MapLibre: https://github.com/sponsors/maplibre https://github.com/sponsors/maplibre (Disclaimer: I'm on the governing board, but derive no financial benefit from it.)
  - pella 4y ago
    And https://donate.openstreetmap.org/ https://donate.openstreetmap.org/ "OpenStreetMap is the largest open geographic database in the world, the data infrastructure for multitudes of mapping projects around the globe. Your donation to the OpenStreetMap Foundation will cover our core operational expenses in supporting the OpenStreetMap project: hardware costs, legal fees, administrative assistant and other expenses of our working groups and administration. We currently run extremely lean for an operation for a project the size and importance of OpenStreetMap. The OpenStreetMap Foundation relies on revenue from individual and corporate membership dues, profits generated by the annual State of the Map conferences, and donations. We must keep our income sources diversified, as these vary from year to year, but our modest needs stay the same. For this, we need your support."
timwis 4y ago
Also see valhalla for an open source routing engine: https://github.com/valhalla/valhalla/ https://github.com/valhalla/valhalla/
- lukeqsee 4y ago
  Valhalla is great (I've contributed a few patches and run it for a while)! Also check: http://project-osrm.org/ http://project-osrm.org/ https://www.graphhopper.com/open-source/ https://www.graphhopper.com/open-source/
notyourday 4y ago
Open source use we will, pay for development we won't, paywall for our "reporting" we have
- ARandomerDude 4y ago
  Write like Yoda you do.
- margorczynski 4y ago
  That's why I actually support paid licensing for commercial use. I'm all for helping out people but not a fan of getting taken advantage off - you make money off my or someones else's work then pay up.
0xDEF 4y ago
Leaflet is also an interesting solution in this space: https://github.com/Leaflet/Leaflet https://github.com/Leaflet/Leaflet
- lights0123 4y ago
  Although I used to exclusively use it, Canvas-based maps just run so much smoother and support things like rotation that are really helpful for changing perspective.
- aendruk 4y ago
  Vector tiles?
  - lukeqsee 4y ago
    https://github.com/maplibre/maplibre-gl-leaflet https://github.com/maplibre/maplibre-gl-leaflet exists! It's not perfect, and you don't see the full benefit of a WebGL renderer, but if you want to keep using a Leaflet API, it's great.
flakeoil 4y ago
It's weird, 10 years ago I implemented using Google Maps API a website with dynamic maps, distance matrix, geolocation, map overlays and address autocomplete completely free also at high traffic. It was also easy to use. Five years ago Google started to charge an outrageous amount > $1000/month and we could not continue so we switched to something that cost $100/month. And that was for only the maps. We had to scrap the distance matrix and address autocomplete. Today it's still the same status, and if you want something cheaper then you have to work on putting pieces together like the Washington Post does.
- lukeqsee 4y ago
  We offer a very competitive solution that is quickly becoming a one-stop shop for what you mentioned: https://stadiamaps.com https://stadiamaps.com. All that functionality for a lot less than $1000 / month.
  - countvonbalzac 4y ago
    you should put a price comparison for stadia vs. google vs. mapbox
    - lukeqsee 4y ago
      Yes. We should, and we're literally working on it right now!
  - notatoad 4y ago
    and as a bonus you've got alidade smooth, the ideal map tile for overlaying colorful data on top of: https://stadiamaps.com/themes/ https://stadiamaps.com/themes/
    - lukeqsee 4y ago
      Thanks for the kind words! :)
  - vinibrito 4y ago
trynewideas 4y ago
In the tools space, cf. open source GeoJSON-to-vector tile tool Tippecanoe, which recently got a new non-Mapbox home at Felt: https://felt.com/blog/erica-fischer-tippecanoe-at-felt https://felt.com/blog/erica-fischer-tippecanoe-at-felt, https://github.com/felt/tippecanoe https://github.com/felt/tippecanoe
Doctor_Fegg 4y ago
> But for most of our use cases, we don’t need the latest and greatest. This is the crux of this. Let’s not pretend that much of this is anything other than “open source clones of Mapbox”. PMTiles is genuinely innovative and I’m delighted to see it getting traction. Maplibre GL (which I use and like very much) is a fork of the last open source version of Mapbox GL, with not many code changes between now and then. Maputnik is an open source reimplementation of Mapbox Studio. Mapbox went on to hire its original dev. OpenMapTiles started life as basically a how-to/makefile pulling together the disparate, poorly documented parts of Mapbox’s open source tile generation code. It’s now mostly known for a reasonable, not particularly exceptional vector tile schema. It’s great that people are now shipping production vector tiles without paying the Mapbox dollar. But, PMTiles aside, this is still Mapbox’s world. The next challenge is to evolve the tech stack to something beyond what Mapbox worked up five/ten years ago.
- lukeqsee 4y ago
  > PMTiles aside, this is still Mapbox’s world. Isn't this a bit like saying we're still in Google's world because most maps still use Web Mercator? :) Good tech builds on what came before. Mapbox did a lot of ground-breaking work in building tooling around OSM, but so have many others. The fact that they named it Mapbox Vector Tiles is genius in hindsight, because even though we may use tons of tooling they didn't create to build and render them, their name is still there. > The next challenge is to evolve the tech stack to something beyond what Mapbox worked up five/ten years ago. Agreed, and I think we've seen a lot of iterative work in the open since then. The next challenge is likely building a OSS stack to do proper 3D: open data (including OSM) to pixels, and that work is already beginning across a lot of organizations: https://github.com/nyurik/future-mvt/discussions https://github.com/nyurik/future-mvt/discussions, Overture Maps, MapLibre, etc.
  - twelvechairs 4y ago
    I get the parent comment. There are good alteratives to the mapbox ecosystem though they arent always used because people see mapbox as default DeckGL for instance is a very innovative js library for front end (really developed by Uber I believe), with better 3d and movement, better with dealing data at scale, etc. Prettymaps is a pure python approach to rendering OSM data. Etc.
cancan 4y ago
[founder of felt, maintainer of some of the libraries mentioned] this is amazing to see! we are happy to see our OS contributions like tippecanoe are being used by people all over the place, and i personally really love to work with journalists. happy to answer any questions!
- lukeqsee 4y ago
  Thanks for taking up the maintenance of Tippecanoe! I haven't needed to use it too much, but when I have, it fills a nice void.
sgt 4y ago
Geoserver is also good for serving spatial data and tiles. A bit dated look but fast and stable
- wiredfool 4y ago
  I find geoserver to be a bit of a headache to run, and have been moving more towards client side rendering using map ox style tools.
  - guwop 4y ago
    Do you know of any good alternatives to geoserver if i have a PostGIS db with some feature gis data and i need to expose it to other people through a Web Feature Service?
    - wiredfool 4y ago
      If you're just looking for a WFS (geojson/data) and not WMS/raster output, you might look at PyGeoAPI. (https://pygeoapi.io/ https://pygeoapi.io/) I haven't used it, but have looked at it a bit for a potential project to export geo data as an API. Geoserver does fill a pretty big hole in the capabilities space -- it's pretty easy to get going with a bunch of layers and style them, but ultimately they're implementing a RDBMS in xml files, and it's a big, complicated, java system that's been one of the more troublesome portions of the stack (IME).
      - guwop 4y ago
        Thanks a bunch, pygeoapi sounds very simple, i'll take a look.
preya2k 4y ago
Is there an OSS way to use vector tiles, without using Mapbox?
- mtmail 4y ago
  The article covers that. See the "maplibre-gl-js for client-side rendering" section.
- rhodysurf 4y ago
  Yes, just use tippecanoe to create your tiles, and serve them up via s3 or howerver else you want. Then client side render them with maplibregl
- Demiurge 4y ago
  Check this out for backend: https://postgis.net/docs/ST_AsMVT.html https://postgis.net/docs/ST_AsMVT.html if you can run ogr2ogr to import your data into PostGIS, and serve the binary data returned by a query, you're all set. You literally only need PostGIS for a dynamic vector tile service. Then mapboxgl or libremapjs on the frontend, or even OpenLayers.
  - guwop 4y ago
    So to get it straight: OSM.pbf file -> ogr2ogr -> PostGIS? I currently use a .mbtiles file and plug it into tileserver-gl, but i guess in theory you could just import OSM to PostGIS and then add the PostGIS as a source in tileserver-gl? That would be amazing
didip 4y ago
Is Mapbox dead now? It used to be awesome many years ago.
__turbobrew__ 4y ago
Titiler works pretty well for raster data in my experience. It is a step up from using an out of the box solution and involves a bit more development work but pretty easily allows customization. Using titiler on lambda with COGs stored on S3 we could get about 200ms load of a retina resolution map. We wanted to do things even faster so we started migrating to ECS which when using a network optimized instance we could render a full page map in sub 100ms. The author states that having all the data in a single file for PMTiles is advantageous, but it is a tradeoff. First of all you cannot modify parts of a S3 file so if you want to modify/update your dataset you need to re-process and re-upload the entire dataset. This can be a real pain if you are working with many TB datasets and you just want to update a subset of the data. Also, I thought that there is a global throughput limit on S3 files so wouldn’t putting all data in a single S3 file potentially hit bottlenecks?
- bdon 4y ago
  Yes, PMTiles is a tradeoff that isn't appropriate for transactional use cases. SQLite is pretty good for that already. There is a throughput limit on S3 files of approximately 5500 GETs/sec per key. Bare archives on S3 is an appropriate choice for small-scale, zero maintenance deployments. If your application demands any thing close to that level of throughput, you're probably either: * Serving individual tiles over the internet: you should use the CDN integration http://protomaps.com/docs/cdn http://protomaps.com/docs/cdn ; most tile requests will be cached and only misses will interact with the S3 bottleneck. * Bulk accessing a spatial subset of tiles: You shouldn't be requesting HTTP GETs for single tiles, but instead entire subsets of tiles with a single Range request made possible by the internal Hilbert curve ordering. This is still WIP here: https://github.com/protomaps/go-pmtiles/issues/31 https://github.com/protomaps/go-pmtiles/issues/31
- neuronexmachina 4y ago
  > Using titiler on lambda with COGs stored on S3 we could get about 200ms load of a retina resolution map. Since Lambda presumably has a separate GDAL RAM cache for each request, I'm guessing this wasn't able to take advantage of any of GDAL's header or block caching? It's pretty impressive you were able to get latencies that low with that approach.
pierotofy 4y ago
OpenMapTiles is open source just as much as OpenAI is open source.
- goplayoutside 4y ago
  While it might be nice if there we pre-rendered tiles available for download, and openmaptiles.org constantly steers users to "buy now" pages, instead of offering a simple button to the self-service docs, they do in fact offer a significant amount of documentation on how to generate and serve your own tiles[1], including a page with fairly step-by-step instructions on how to create a tileset "from scratch" without sending them a cent[2]. 1. https://openmaptiles.org/docs/ https://openmaptiles.org/docs/ 2. https://openmaptiles.org/docs/generate/generate-openmaptiles/ https://openmaptiles.org/docs/generate/generate-openmaptiles...
time4tea 4y ago
I think you can do this quite easily. Use docker compose to start an nginx and maptiler/tileserver-gl. Use an nginx config to send requests upstream to tileserver and cache. Download a region file from maptiler, or make one. That's it. It's about 2h work, using certbot for certificates. If you want to create your own styles, it's slightly more fiddly, but essentially it's https://maputnik.github.io/editor/#0.41/0/0 https://maputnik.github.io/editor/#0.41/0/0 Host on hetzner for €3/month.
jillesvangurp 4y ago
We use maplibre and map tiler for the map tiles in the startup that I run. We switched to that from leaflet and I've been pretty happy with it. I might invest some time at some point in self hosting tiles. But honestly, map tiler is good value for money for now. I actually used to work with Nokia and Nokia Maps (in Berlin), which is now Here Maps. So, I know a thing or two about this market and still know some people working with various map companies, including at mapbox. Mapbox is increasingly an alternative to Here, Tom Tom and a few others that offer a wide range of services. The most simple one of which is showing maps. Think of indoor maps, traffic services, routing, geo coding, address search (surprisingly hard problem with open data), etc. Customers for these services tend to spend a lot of money on getting all sorts of bespoke solutions. Maps are a commodity at this point and there's very little money in providing just some basic maps. It's all the other stuff that's built on top of that that is a lot harder and more valuable. The recently announced overture collaboration could actually shake things up a little in this space. Openstreetmap is nice but also has some pretty significant limitations. Basically what the above mentioned companies do is creating proprietary services and data (with and without open data typically) that they then sell access to for a lot of money. The big companies behind overture are each big customers for this stuff. And now they are pooling resources to create a cheaper, more open alternative to all that.
mapnostic 4y ago
This discussion is great to see! We’ve been talking about this article at work, the only thing I’d point out that hasn’t been said is that there is some complexity to designing with and managing the other renderer assets besides tiles (i.e. styles, markers, textures, icons, fonts). Most of this complexity is well documented if you know where to look, but will trip you up at least the first time you set up a serverless mapping stack, so buyer beware. The trickiest of these assets is the stylesheet, necessary for the renderer to know how to draw the data in tiles. The article mentions Maputnik, which works great for quick prototyping and the production deadlines of newsrooms, but limiting once your map, application, and or tiles get to a certain level of complexity.
ecf 4y ago
Currently using Mapbox in an open-source project for their static map API and I wholeheartedly recommend against it. Everything but the simplest of polylines look jagged since they refuse to switch to POST requests with a json body and instead rely on you fitting in as much detail into URL query parameter length constraints. Going to take a look at replacing them with PMTiles and my own polyline rendering.
- tylershuster 4y ago
  I've switched over a fairly large transit client to PMTiles hosted on CloudFlare for their dynamic map and am switching another large transit client to using https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl with MBTiles and the same style.json to generate static maps. It's all gone swimmingly and we've cut costs by about $1,000 a month.
- lukeqsee 4y ago
  There's actually a reason why most of these APIs support POST—most use-cases are hot-linking, where you don't control how the client makes the request. (Often using an <img>.) Here's one alternative, if you don't want to build from scratch: https://docs.stadiamaps.com/static-maps/ https://docs.stadiamaps.com/static-maps/ We don't have much in the way of query string limits, so have at it!
- capitainenemo 4y ago
  Rather curious about the URL query parameter length constraint. Excluding IE11/Edge, don't most major browsers have at least 64k query lengths these days? Could you just serve the low quality jaggy poly to few percent still on IE11/Edge and a high detail one to the rest?
  - lukeqsee 4y ago
    A lot of servers impose a constraint on URL length, so I suspect Mapbox does on their APIs.
- danpat 4y ago
  You always have the option of creating a map style and using Mapbox's Raster Tile API - although that does mean you probably can't dynamically adjust your raster images in time for a request to be served. How are you using the Static Maps API currently where you would be able to send a POST request and display the image back to the user? Most use-cases for the Static Maps API involve the HTML <img> tag or something similar, which all send GET requests.
jeffbee 4y ago
The Post has been doing great data journalism lately. The maps from the article were really well done. Yesterday they ran this story that has all its graphics done in D3 and provides links to notebooks at ObservableHQ with evidence supporting the arguments made and conclusions drawn in the article. https://www.washingtonpost.com/climate-environment/interactive/2023/hot-cold-extreme-temperature-deaths/ https://www.washingtonpost.com/climate-environment/interacti...
jackson1442 4y ago
Looks great. Been using Mapbox as my go-to for small projects since it's much smoother than Leaflet and has better DX in my opinion. This seems to capture all of that with an even better DX so definitely will be trying it out.
- lukeqsee 4y ago
  https://maplibre.org/ https://maplibre.org/ definitely attempts to preserve the DX with free selection of tile providers.
  - tacker2000 4y ago
    DX == Developer Experience? Thats a new one.
    - deleted 4y ago
      [deleted]
    - mkl 4y ago
      No, it's quite old now. Well over a decade.
jokoon 4y ago
I don't want to sound annoying, but it's a bit hard to generate raster tiles. I mean openstreetmap data is open and free, but it doesn't seem they distribute any software to build the raster tiles they host. I don't know what they use, but it doesn't seem there is any software that is simple enough to use. I just want to configure a subregion, call generate-tiles.py planet.xml or whatever, and be done with it. I like open source, but there will always be open source software that is needlessly complicated because, well, it lets companies like mapbox and others to generate income for their expertise. That's not the best open source spirit, in my view.
- deleted 4y ago
  [deleted]
- livinglist 4y ago
  Test
- uneekname 4y ago
  Open-source geospatial tools in general have developed rapidly over the last couple of decades. Mapbox and friends have laid groundwork, both good and "needlessly complicated." But there are just too many people who care about this to think it'll always be hard. I assure you, the OSS (and in this case, OSM) community will steadily build out better, faster software that includes more and more of the features a modern map needs. It's already looking like companies like WashPo are seeing the potential here. Come join us and make the world a better place :)
- deleted 4y ago
  [deleted]
- tylershuster 4y ago
  https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl makes it really easy. I have a pretty simple pipeline set up to download an OSM extract, convert it to MBTiles, host it in CloudFlare, and render raster tiles with this software.
- lukeqsee 4y ago

613 points by gaws 4y ago

tpmx 4y ago

Wasn't Mapbox once the low-cost option?

usrusr 4y ago
And since the state of the open source map rendering art is consisting almost exclusively of pre-licence-change forks of mapbox code these days, they kind of still are. I wonder if Strava's recent acquisition of fatmap.com eventually boils down to them cutting ties with mapbox via acqui-hiring some mb-libre excellence. I could imagine Strava, due to the extreme map-centric nature of their product, bleeding quite considerable money to mapbox.
- kylebarron 4y ago
  I didn't think Fatmap was based on mapbox at all? I thought it was their own custom renderer
  - gorbypark 4y ago
    I’m interpreting the comment above to mean Strava was/is a user of Mapbox and bought Fatmap in an effort to wean themselves off of Mapbox and use “inhouse” (Fatmap) mapping tech.
  - usrusr 4y ago
    I took a superficial glance at what the site is loading and iirc there were plenty of the usual suspects in there. And it would be silly to do it any other way, because their product is what they do with maps, not winning some NIH purity award.
- lukeqsee 4y ago
  MapLibre has made really good strides (especially on the web) since the fork—3d-terrain support, more tile sources, other DX improvements (e.g., conversion to TypeScript). https://maplibre.org/maplibre-gl-js-docs/api/ https://maplibre.org/maplibre-gl-js-docs/api/
deleted 4y ago
[deleted]
Operyl 4y ago
Yup, which disrupted the market, and now it's not.
- klysm 4y ago
  1. Subsidize low costs with venture capital and take on a ton of customers at a loss 2. Lock those customers into your API somehow. Enough code is effectively lock in for a while. 3. Change your pricing to be profitable. 4. Hopefully get acquired before all your customers leave
  - tpmx 4y ago
    They seem to be at step 3 at the moment.
    - Nemo_bis 4y ago
      Usage seems to have peaked in April 2022 https://trends.builtwith.com/mapping/Mapbox https://trends.builtwith.com/mapping/Mapbox
gorbypark 4y ago
They used to be low-cost _and_ open source. Now they're source available and not interested in anything but big clients. I was using Mapbox for all my clients before their unopen sourcing, and I tried reaching out three times to ask about options for one of my clients. It wouldn't have been a huge amount of money, probably a few hundred per month, but they never, ever reply. That client wasn't even using any of their server resources, they are 100% self hosted, it's just that Mapbox now charges even for using their client. Anyways, I was trying to give them money, and they wouldn't even acknowledge I exist. I switched that client over to Maplibre and never looked back. The other thing that annoys me is that they've more or less abandoned all their open source repos without even as much of a notice in the README. All these amazing tools sitting there with open pull requests and no one responding for years. If they just did a clean break, put a notice on everything they weren't going to touch again, it would have been a lot better/easier for the community to fork and move on with their lives.
- shawn-butler 4y ago
  Mapbox was born from open source tools & culture. They were always active and supporting at FOSS4G. I wonder what has changed?
  - jonas21 4y ago
    VC funding? I think they were funded mostly by grants and their own revenues for the first few years. Then they started taking VC money (over $300M to date).
    - dataviz1000 4y ago
      They were once Development Seed [0], a company that specialized in making geo spatial visualizations for the UN, NGOs and nonprofits. Development Seed was a large contributor of code and modules to Drupal 6 which they left. In the beginning, their open source mapping and geo spatial visualizations projects were mostly funded by grants from the Knight Foundation which funded news related projects. MapBox came from an in-house tool they developed to create and manage map overlay tiles. (I could be making this all up.) [0] https://developmentseed.org/ https://developmentseed.org/

lukeqsee 4y ago

This is really awesome to see. I'm involved in many of these projects (helped get MapLibre started, wrote a library supporting PMTiles, have used OpenMapTiles for years), and to see open-source, open-data win in the end is really satisfying. If anyone has any questions about how you could do this for yourself, I'm happy to answer! (I have a company offering SaaS maps, but we know SaaS isn't for everyone, and are happy to point people the right direction when self-hosting is what you want.)

jddj 4y ago
Is there a standard folder tree of tiles -> PMTiles archiver or is that the wrong question to be asking?
- lukeqsee 4y ago
  I don't think so. There's this: https://github.com/protomaps/go-pmtiles https://github.com/protomaps/go-pmtiles to get from MBTiles and there's a few xyz folder tree tools to get mbtiles from that structure (e.g., https://pypi.org/project/mbutil/ https://pypi.org/project/mbutil/). So you can get there, but I don't know of a good straight-line right now.
  - jddj 4y ago
    Thanks a lot. I developed and support a standalone desktop application which contains customer-specific tiled maps, and it can get unwieldy. A pivot at some point to these types of archives is an option, but I'd like to roll forward existing customers as well.
    - bdon 4y ago
      If you have access to SQLite in your desktop application then there isn't any big wins from using PMTiles over MBTiles; PMTiles does have some special design around tile-deduplication but this is possible in MBTiles too by using table views.
- bdon 4y ago

762236 4y ago

If all of the tiles are coming from a single file (via PMTiles), how are they cached by the client? Do clients cache byte ranges, or do you configure things to give the appearance of serving separate files?

lukeqsee 4y ago
You typically use byte-range caching in libraries—either using an edge worker (a la Cloudflare) to create separate GET requests from the client's perspective or directly in the client.
- cancan 4y ago
  i think you can also directly call them from the maplibre lib itself using the pmtiles adapter https://github.com/protomaps/PMTiles https://github.com/protomaps/PMTiles https://github.com/protomaps/PMTiles/blob/main/js/examples/maplibre.html#L24 https://github.com/protomaps/PMTiles/blob/main/js/examples/m...
bdon 4y ago
The renderer used in the linked article (MapLibre) has an internal tile cache used when you return to a previously loaded area of the map. For the non-tiled parts of the PMTiles file, like metadata and the directories that store Z/X/Y to offset information, those are LRU cached by the PMTiles implementation. The most widely deployed right now is for TypeScript/Node. The "client" can either be web browser or inside a runtime like Lambda/Cloudflare Workers. On those the cache will persist across invocations using the ephemeral memory of the serverless function. A level deeper, browsers vary in how they cache Range requests; IIRC Firefox does not cache while others will treat Range as if it was any other HTTP header.

kabes 4y ago

Worth mentioning that openlayers is also really extending their webgl rendering capabilities lately.

lukeqsee 4y ago
There's also support for MapLibre GL JS style specs in the works! Excited to see another stable vector tile & style renderer added to the ecosystem.
- deleted 4y ago
  [deleted]

T3RMINATED 4y ago

[dead]

wipfli 4y ago

If you like PMTiles you should consider supporting Brandon financially as a GitHub Sponsor. Brandon is a solo-developer who builds Protomaps alone and we get all his great stuff for free... https://github.com/sponsors/protomaps https://github.com/sponsors/protomaps

lukeqsee 4y ago
You can also sponsor MapLibre: https://github.com/sponsors/maplibre https://github.com/sponsors/maplibre (Disclaimer: I'm on the governing board, but derive no financial benefit from it.)
- pella 4y ago
  And https://donate.openstreetmap.org/ https://donate.openstreetmap.org/ "OpenStreetMap is the largest open geographic database in the world, the data infrastructure for multitudes of mapping projects around the globe. Your donation to the OpenStreetMap Foundation will cover our core operational expenses in supporting the OpenStreetMap project: hardware costs, legal fees, administrative assistant and other expenses of our working groups and administration. We currently run extremely lean for an operation for a project the size and importance of OpenStreetMap. The OpenStreetMap Foundation relies on revenue from individual and corporate membership dues, profits generated by the annual State of the Map conferences, and donations. We must keep our income sources diversified, as these vary from year to year, but our modest needs stay the same. For this, we need your support."

timwis 4y ago

Also see valhalla for an open source routing engine: https://github.com/valhalla/valhalla/ https://github.com/valhalla/valhalla/

lukeqsee 4y ago
Valhalla is great (I've contributed a few patches and run it for a while)! Also check: http://project-osrm.org/ http://project-osrm.org/ https://www.graphhopper.com/open-source/ https://www.graphhopper.com/open-source/

notyourday 4y ago

Open source use we will, pay for development we won't, paywall for our "reporting" we have

ARandomerDude 4y ago
Write like Yoda you do.
margorczynski 4y ago
That's why I actually support paid licensing for commercial use. I'm all for helping out people but not a fan of getting taken advantage off - you make money off my or someones else's work then pay up.

0xDEF 4y ago

Leaflet is also an interesting solution in this space: https://github.com/Leaflet/Leaflet https://github.com/Leaflet/Leaflet

lights0123 4y ago
Although I used to exclusively use it, Canvas-based maps just run so much smoother and support things like rotation that are really helpful for changing perspective.
aendruk 4y ago
Vector tiles?
- lukeqsee 4y ago
  https://github.com/maplibre/maplibre-gl-leaflet https://github.com/maplibre/maplibre-gl-leaflet exists! It's not perfect, and you don't see the full benefit of a WebGL renderer, but if you want to keep using a Leaflet API, it's great.

flakeoil 4y ago

It's weird, 10 years ago I implemented using Google Maps API a website with dynamic maps, distance matrix, geolocation, map overlays and address autocomplete completely free also at high traffic. It was also easy to use. Five years ago Google started to charge an outrageous amount > $1000/month and we could not continue so we switched to something that cost $100/month. And that was for only the maps. We had to scrap the distance matrix and address autocomplete. Today it's still the same status, and if you want something cheaper then you have to work on putting pieces together like the Washington Post does.

lukeqsee 4y ago
We offer a very competitive solution that is quickly becoming a one-stop shop for what you mentioned: https://stadiamaps.com https://stadiamaps.com. All that functionality for a lot less than $1000 / month.
- countvonbalzac 4y ago
  you should put a price comparison for stadia vs. google vs. mapbox
  - lukeqsee 4y ago
    Yes. We should, and we're literally working on it right now!
- notatoad 4y ago
  and as a bonus you've got alidade smooth, the ideal map tile for overlaying colorful data on top of: https://stadiamaps.com/themes/ https://stadiamaps.com/themes/
  - lukeqsee 4y ago
    Thanks for the kind words! :)
- vinibrito 4y ago

trynewideas 4y ago

In the tools space, cf. open source GeoJSON-to-vector tile tool Tippecanoe, which recently got a new non-Mapbox home at Felt: https://felt.com/blog/erica-fischer-tippecanoe-at-felt https://felt.com/blog/erica-fischer-tippecanoe-at-felt, https://github.com/felt/tippecanoe https://github.com/felt/tippecanoe

Doctor_Fegg 4y ago

> But for most of our use cases, we don’t need the latest and greatest. This is the crux of this. Let’s not pretend that much of this is anything other than “open source clones of Mapbox”. PMTiles is genuinely innovative and I’m delighted to see it getting traction. Maplibre GL (which I use and like very much) is a fork of the last open source version of Mapbox GL, with not many code changes between now and then. Maputnik is an open source reimplementation of Mapbox Studio. Mapbox went on to hire its original dev. OpenMapTiles started life as basically a how-to/makefile pulling together the disparate, poorly documented parts of Mapbox’s open source tile generation code. It’s now mostly known for a reasonable, not particularly exceptional vector tile schema. It’s great that people are now shipping production vector tiles without paying the Mapbox dollar. But, PMTiles aside, this is still Mapbox’s world. The next challenge is to evolve the tech stack to something beyond what Mapbox worked up five/ten years ago.

lukeqsee 4y ago
> PMTiles aside, this is still Mapbox’s world. Isn't this a bit like saying we're still in Google's world because most maps still use Web Mercator? :) Good tech builds on what came before. Mapbox did a lot of ground-breaking work in building tooling around OSM, but so have many others. The fact that they named it Mapbox Vector Tiles is genius in hindsight, because even though we may use tons of tooling they didn't create to build and render them, their name is still there. > The next challenge is to evolve the tech stack to something beyond what Mapbox worked up five/ten years ago. Agreed, and I think we've seen a lot of iterative work in the open since then. The next challenge is likely building a OSS stack to do proper 3D: open data (including OSM) to pixels, and that work is already beginning across a lot of organizations: https://github.com/nyurik/future-mvt/discussions https://github.com/nyurik/future-mvt/discussions, Overture Maps, MapLibre, etc.
- twelvechairs 4y ago
  I get the parent comment. There are good alteratives to the mapbox ecosystem though they arent always used because people see mapbox as default DeckGL for instance is a very innovative js library for front end (really developed by Uber I believe), with better 3d and movement, better with dealing data at scale, etc. Prettymaps is a pure python approach to rendering OSM data. Etc.

cancan 4y ago

[founder of felt, maintainer of some of the libraries mentioned] this is amazing to see! we are happy to see our OS contributions like tippecanoe are being used by people all over the place, and i personally really love to work with journalists. happy to answer any questions!

lukeqsee 4y ago
Thanks for taking up the maintenance of Tippecanoe! I haven't needed to use it too much, but when I have, it fills a nice void.

sgt 4y ago

Geoserver is also good for serving spatial data and tiles. A bit dated look but fast and stable

wiredfool 4y ago
I find geoserver to be a bit of a headache to run, and have been moving more towards client side rendering using map ox style tools.
- guwop 4y ago
  Do you know of any good alternatives to geoserver if i have a PostGIS db with some feature gis data and i need to expose it to other people through a Web Feature Service?
  - wiredfool 4y ago
    If you're just looking for a WFS (geojson/data) and not WMS/raster output, you might look at PyGeoAPI. (https://pygeoapi.io/ https://pygeoapi.io/) I haven't used it, but have looked at it a bit for a potential project to export geo data as an API. Geoserver does fill a pretty big hole in the capabilities space -- it's pretty easy to get going with a bunch of layers and style them, but ultimately they're implementing a RDBMS in xml files, and it's a big, complicated, java system that's been one of the more troublesome portions of the stack (IME).
    - guwop 4y ago
      Thanks a bunch, pygeoapi sounds very simple, i'll take a look.

preya2k 4y ago

Is there an OSS way to use vector tiles, without using Mapbox?

mtmail 4y ago
The article covers that. See the "maplibre-gl-js for client-side rendering" section.
rhodysurf 4y ago
Yes, just use tippecanoe to create your tiles, and serve them up via s3 or howerver else you want. Then client side render them with maplibregl
Demiurge 4y ago
Check this out for backend: https://postgis.net/docs/ST_AsMVT.html https://postgis.net/docs/ST_AsMVT.html if you can run ogr2ogr to import your data into PostGIS, and serve the binary data returned by a query, you're all set. You literally only need PostGIS for a dynamic vector tile service. Then mapboxgl or libremapjs on the frontend, or even OpenLayers.
- guwop 4y ago
  So to get it straight: OSM.pbf file -> ogr2ogr -> PostGIS? I currently use a .mbtiles file and plug it into tileserver-gl, but i guess in theory you could just import OSM to PostGIS and then add the PostGIS as a source in tileserver-gl? That would be amazing

didip 4y ago

Is Mapbox dead now? It used to be awesome many years ago.

__turbobrew__ 4y ago

Titiler works pretty well for raster data in my experience. It is a step up from using an out of the box solution and involves a bit more development work but pretty easily allows customization. Using titiler on lambda with COGs stored on S3 we could get about 200ms load of a retina resolution map. We wanted to do things even faster so we started migrating to ECS which when using a network optimized instance we could render a full page map in sub 100ms. The author states that having all the data in a single file for PMTiles is advantageous, but it is a tradeoff. First of all you cannot modify parts of a S3 file so if you want to modify/update your dataset you need to re-process and re-upload the entire dataset. This can be a real pain if you are working with many TB datasets and you just want to update a subset of the data. Also, I thought that there is a global throughput limit on S3 files so wouldn’t putting all data in a single S3 file potentially hit bottlenecks?

bdon 4y ago
Yes, PMTiles is a tradeoff that isn't appropriate for transactional use cases. SQLite is pretty good for that already. There is a throughput limit on S3 files of approximately 5500 GETs/sec per key. Bare archives on S3 is an appropriate choice for small-scale, zero maintenance deployments. If your application demands any thing close to that level of throughput, you're probably either: * Serving individual tiles over the internet: you should use the CDN integration http://protomaps.com/docs/cdn http://protomaps.com/docs/cdn ; most tile requests will be cached and only misses will interact with the S3 bottleneck. * Bulk accessing a spatial subset of tiles: You shouldn't be requesting HTTP GETs for single tiles, but instead entire subsets of tiles with a single Range request made possible by the internal Hilbert curve ordering. This is still WIP here: https://github.com/protomaps/go-pmtiles/issues/31 https://github.com/protomaps/go-pmtiles/issues/31
neuronexmachina 4y ago
> Using titiler on lambda with COGs stored on S3 we could get about 200ms load of a retina resolution map. Since Lambda presumably has a separate GDAL RAM cache for each request, I'm guessing this wasn't able to take advantage of any of GDAL's header or block caching? It's pretty impressive you were able to get latencies that low with that approach.

pierotofy 4y ago

OpenMapTiles is open source just as much as OpenAI is open source.

goplayoutside 4y ago
While it might be nice if there we pre-rendered tiles available for download, and openmaptiles.org constantly steers users to "buy now" pages, instead of offering a simple button to the self-service docs, they do in fact offer a significant amount of documentation on how to generate and serve your own tiles[1], including a page with fairly step-by-step instructions on how to create a tileset "from scratch" without sending them a cent[2]. 1. https://openmaptiles.org/docs/ https://openmaptiles.org/docs/ 2. https://openmaptiles.org/docs/generate/generate-openmaptiles/ https://openmaptiles.org/docs/generate/generate-openmaptiles...

time4tea 4y ago

I think you can do this quite easily. Use docker compose to start an nginx and maptiler/tileserver-gl. Use an nginx config to send requests upstream to tileserver and cache. Download a region file from maptiler, or make one. That's it. It's about 2h work, using certbot for certificates. If you want to create your own styles, it's slightly more fiddly, but essentially it's https://maputnik.github.io/editor/#0.41/0/0 https://maputnik.github.io/editor/#0.41/0/0 Host on hetzner for €3/month.

jillesvangurp 4y ago

We use maplibre and map tiler for the map tiles in the startup that I run. We switched to that from leaflet and I've been pretty happy with it. I might invest some time at some point in self hosting tiles. But honestly, map tiler is good value for money for now. I actually used to work with Nokia and Nokia Maps (in Berlin), which is now Here Maps. So, I know a thing or two about this market and still know some people working with various map companies, including at mapbox. Mapbox is increasingly an alternative to Here, Tom Tom and a few others that offer a wide range of services. The most simple one of which is showing maps. Think of indoor maps, traffic services, routing, geo coding, address search (surprisingly hard problem with open data), etc. Customers for these services tend to spend a lot of money on getting all sorts of bespoke solutions. Maps are a commodity at this point and there's very little money in providing just some basic maps. It's all the other stuff that's built on top of that that is a lot harder and more valuable. The recently announced overture collaboration could actually shake things up a little in this space. Openstreetmap is nice but also has some pretty significant limitations. Basically what the above mentioned companies do is creating proprietary services and data (with and without open data typically) that they then sell access to for a lot of money. The big companies behind overture are each big customers for this stuff. And now they are pooling resources to create a cheaper, more open alternative to all that.

mapnostic 4y ago

This discussion is great to see! We’ve been talking about this article at work, the only thing I’d point out that hasn’t been said is that there is some complexity to designing with and managing the other renderer assets besides tiles (i.e. styles, markers, textures, icons, fonts). Most of this complexity is well documented if you know where to look, but will trip you up at least the first time you set up a serverless mapping stack, so buyer beware. The trickiest of these assets is the stylesheet, necessary for the renderer to know how to draw the data in tiles. The article mentions Maputnik, which works great for quick prototyping and the production deadlines of newsrooms, but limiting once your map, application, and or tiles get to a certain level of complexity.

ecf 4y ago

Currently using Mapbox in an open-source project for their static map API and I wholeheartedly recommend against it. Everything but the simplest of polylines look jagged since they refuse to switch to POST requests with a json body and instead rely on you fitting in as much detail into URL query parameter length constraints. Going to take a look at replacing them with PMTiles and my own polyline rendering.

tylershuster 4y ago
I've switched over a fairly large transit client to PMTiles hosted on CloudFlare for their dynamic map and am switching another large transit client to using https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl with MBTiles and the same style.json to generate static maps. It's all gone swimmingly and we've cut costs by about $1,000 a month.
lukeqsee 4y ago
There's actually a reason why most of these APIs support POST—most use-cases are hot-linking, where you don't control how the client makes the request. (Often using an <img>.) Here's one alternative, if you don't want to build from scratch: https://docs.stadiamaps.com/static-maps/ https://docs.stadiamaps.com/static-maps/ We don't have much in the way of query string limits, so have at it!
capitainenemo 4y ago
Rather curious about the URL query parameter length constraint. Excluding IE11/Edge, don't most major browsers have at least 64k query lengths these days? Could you just serve the low quality jaggy poly to few percent still on IE11/Edge and a high detail one to the rest?
- lukeqsee 4y ago
  A lot of servers impose a constraint on URL length, so I suspect Mapbox does on their APIs.
danpat 4y ago
You always have the option of creating a map style and using Mapbox's Raster Tile API - although that does mean you probably can't dynamically adjust your raster images in time for a request to be served. How are you using the Static Maps API currently where you would be able to send a POST request and display the image back to the user? Most use-cases for the Static Maps API involve the HTML <img> tag or something similar, which all send GET requests.

jeffbee 4y ago

The Post has been doing great data journalism lately. The maps from the article were really well done. Yesterday they ran this story that has all its graphics done in D3 and provides links to notebooks at ObservableHQ with evidence supporting the arguments made and conclusions drawn in the article. https://www.washingtonpost.com/climate-environment/interactive/2023/hot-cold-extreme-temperature-deaths/ https://www.washingtonpost.com/climate-environment/interacti...

jackson1442 4y ago

Looks great. Been using Mapbox as my go-to for small projects since it's much smoother than Leaflet and has better DX in my opinion. This seems to capture all of that with an even better DX so definitely will be trying it out.

lukeqsee 4y ago
https://maplibre.org/ https://maplibre.org/ definitely attempts to preserve the DX with free selection of tile providers.
- tacker2000 4y ago
  DX == Developer Experience? Thats a new one.
  - deleted 4y ago
    [deleted]
  - mkl 4y ago
    No, it's quite old now. Well over a decade.

jokoon 4y ago

I don't want to sound annoying, but it's a bit hard to generate raster tiles. I mean openstreetmap data is open and free, but it doesn't seem they distribute any software to build the raster tiles they host. I don't know what they use, but it doesn't seem there is any software that is simple enough to use. I just want to configure a subregion, call generate-tiles.py planet.xml or whatever, and be done with it. I like open source, but there will always be open source software that is needlessly complicated because, well, it lets companies like mapbox and others to generate income for their expertise. That's not the best open source spirit, in my view.

deleted 4y ago
[deleted]
livinglist 4y ago
Test
uneekname 4y ago
Open-source geospatial tools in general have developed rapidly over the last couple of decades. Mapbox and friends have laid groundwork, both good and "needlessly complicated." But there are just too many people who care about this to think it'll always be hard. I assure you, the OSS (and in this case, OSM) community will steadily build out better, faster software that includes more and more of the features a modern map needs. It's already looking like companies like WashPo are seeing the potential here. Come join us and make the world a better place :)
deleted 4y ago
[deleted]
tylershuster 4y ago
https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl makes it really easy. I have a pretty simple pipeline set up to download an OSM extract, convert it to MBTiles, host it in CloudFlare, and render raster tiles with this software.
lukeqsee 4y ago