7 ms·

2024: The year of the OpenStreetMap vector maps

345 points by RicoElectrico 3y ago

timmg 3y ago
I hope they come up with a nice, clean, standard for a vector tile format. I haven't spent a lot of time looking into it. But the last time I looked at commonly-used vector tile formats, the one I found was clunky to generate/parse. (And/or, maybe I just missed the right one. Please correct me if there is something out there that is clean and simple -- and has common library support.)
- philips 3y ago
  Have you tried pmtiles? https://docs.protomaps.com/ https://docs.protomaps.com/
- philips 3y ago
  Also a post from Paul Normans, mentioned in the parent post, covers some options. https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/ https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/
  - somishere 3y ago
    Not an expert, however (lol) - even if the recommendations here are solid - this doesn't feel like a particularly well-reasoned comparison. IMHO maintaining a large number of files on disk (be it 16m or 1.5b) should be a non-starter (not to mention that zipping them to slightly reduce the count removes one of the benefits of having them in the first place). A database makes sense - as does SOC between generation and distribution if it brings benefit - but immediately writing off SQLite because of write performance[1]? Similarly the arguments against pmtiles feel outdated and ignorant of the developing nature of the protocol (appreciate it's a 2yo post!). OSM deciding on a format here has potential to significantly shape things to come. Hopefully Paul has progressed his thinking a bit since this was written. [1] https://blog.wesleyac.com/posts/consider-sqlite https://blog.wesleyac.com/posts/consider-sqlite
    - cldellow 3y ago
      The SQLite limitations might also be solvable by sharding the databases (so each sees 1/N of the write volume), tiering them (so tiles that are written most recently are in file A, then file B, then file C), or a combination of these approaches. I'm assuming the requirement is not really to build a single .mbtiles file, but to have some way for a process to serve the correct tile when given a z/x/y tuple.
    - bdon 3y ago
      Most of the development for pmtiles has happened in the last 2 years, including a maturing server implementation (http://github.com/protomaps/go-pmtiles http://github.com/protomaps/go-pmtiles) but some key parts are still missing like the ability to decode an archive in native mobile applications. sqlite (mbtiles) already has ~10+ years of integration into the mapping ecosystem so that still works better if you want to move tilesets around to desktop applications and mobile devices. The situation Paul is addressing is one unique to OpenStreetMap itself, which is minute-level updates of a global scale tileset. This is a use case pmtiles is designed explicitly not to address, where using a database is a better fit.
      - somishere 3y ago
        Appreciate you weighing in directly Brandon! To be clear though, I didn't mean to say that protomaps was potentially a better solution (of course an uncompressed DB makes much more sense), simply that the developing nature of the protocol meant that it's inclusion within the "ecosystem" shouldn't be a closed case :) Which is to say, distribution of OSM data feels like a large part of the process. Of course there are various bottlenecks / considerations around edits / writes, but in practice surely reads are the bigger vector. I wonder how many OSM-external use cases rely on "minutely" updates or need the full fidelity of the raw data source. Feels like there is a solid case for providing less frequent (hourly, daily?) official updates via official "single-file" formats that could be widely distributed to the benefit of all, and e.g. allow OSM to loosen up their hot-linking policies and ensure continued investment in the chosen protocols. But mainly I was questioning how a somewhat proven format like SQLite, with its many benefits (interoperability, distribution, etc.), would be so easily dropped from consideration without even a test having been run. Just my thoughts of course!
- penorman 3y ago
  Mapbox Vector Tiles are, for better or worse, the industry standard. They have good library support and are good enough that no one is rushing to reinvent them. If you want to distribute sets of tiles to people, PMTiles is the best container format as you can just throw the file on hosting that supports HTTP Range requests (e.g. S3) and add the appropriate plugins to your client library. If you have to support clients which can't do the range requests from pmtiles, you can throw a thin layer in front of it to turn HTTP z/x/y requests into the pmtiles range requests. There are no good file formats for distributing minutely updates of tiles but there's no need for this. If you are keeping your tiles up to date up to the minute you have a reasonable internet connection and you can just download individual tiles instead.
  - timmg 3y ago
    Thanks! I just browsed the spec again (https://github.com/mapbox/vector-tile-spec/tree/master/2.1 https://github.com/mapbox/vector-tile-spec/tree/master/2.1) -- I didn't have as bad of a reaction as I did the first time I looked at it. I guess the idea of using (Google) protobufs for the "packaging" and then a super tight, custom, byte-encoded format for drawing commands seems... weird to me. But, honestly, it probably makes sense.
ajsnigrutin 3y ago
Slightly offtopic I have OsmAnd ( https://github.com/osmandapp/OsmAnd https://github.com/osmandapp/OsmAnd ) on my phone, download the basemaps, download my (small) country data (both sourced from open street maps), and with an app + ~1GB of data, I get the maps and full navigation within my country, POIs, etc., and can add other countries when needed. Is there something similar for a PC? I can download data from open street maps, but then I need postgres, postgis, a tile server and styles and apache running just to generate the tiles. Is there anything portable (short of running osmand in an android virtualbox) for offline navigation on a linux pc? QGIS can display vectors, but I wasn't able to easily style the data... navigation is a no-go there too. anything else?
- dividedbyzero 3y ago
  Not Linux, so it's probably not going to help you much, but the OsmAnd iPad app seems to work just fine on macOS and afaik those run as native apps on M1/2/3 machines, and this particular one feels very similar to the macOS-native Maps app. I guess that's pretty close for those on a Macbook, though turn-by-turn navigation is going to suck with no onboard GPS. I just got it from the App Store to test this and I think I'm going to keep it, it's actually pretty neat.
- lol768 3y ago
  You can download an extract of your country from Geofabrik, run it through Tilemaker (https://github.com/systemed/tilemaker https://github.com/systemed/tilemaker) to get a nice mbtiles file, and then use the built-in Ruby server to give you something you can load in your web-browser locally.
  - stereo 3y ago
    Post author here. Geofabrik has even started providing extracts in the Shortbread vector schema, the same schema we'll be using for our vector tiles.
- maxerickson 3y ago
  There's a port of OSMAnd, looks like it has stagnated. https://sourceforge.net/projects/offroadosm/files/ https://sourceforge.net/projects/offroadosm/files/
clementmas 3y ago
Which tech stack are they planning to use? It seems like Planetiler and tilemaker are popular options but do they scale for OSM's needs? Protomaps PMtiles are also very interesting
- maxerickson 3y ago
  Probably something else that is straightforward to incrementally update (applies to all 3 cases from your comment). https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/ https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/
  - clementmas 3y ago
    Thanks. I found this article on Paul's blog which answers my question: https://www.paulnorman.ca/blog/2024/01/minutely-updated-tiles https://www.paulnorman.ca/blog/2024/01/minutely-updated-tile...
- stereo 3y ago
  Post author here! We’re going to reuse a lot of existing bricks like osm2pgsql, postgis’s ST_AsMVT, and the shortbread schema to start with. Tilekiln, the project Paul is working on, is python to orchestrate it all, and regenerate tiles on database updates.
nzeid 3y ago
As a religious OsmAnd user I welcome this. I'll be donating directly to the effort.
- sorenjan 3y ago
  But OsmAnd already uses their own vector tiles.
neomantra 3y ago
I happened to be using OpenStreetMaps for the last couple weeks and released a Golang OpenAPI library for OpenChargeMap [1] In that, I released an IPython notebook and used that to visualize the data on Vision Pro [2]. And I did get it working! But it was not near as cool as it could be because of two dramatic events: OSM Foundation violated the trust (nice way to put it) of a participant, OSMBuildings.org, in the search for sponsorship [3] (four years ago, such lost progress :( ) The other is that LeafletJS’s developer is Ukrainian and can’t advance his project [4]. Please consider those two issues when contemplating donating to OSM due to this press release. While I applaud OSM abstractly, it exists because of a massive amounts of public contributions of IP, not because of $$ from a commercial sponsor. [1] https://github.com/neomantra/go-openchargemap https://github.com/neomantra/go-openchargemap [2] https://github.com/neomantra/go-openchargemap/blob/main/examples/ocm_3d.ipynb https://github.com/neomantra/go-openchargemap/blob/main/exam... [3] https://medium.com/@osmbuildings/why-were-not-going-to-support-a-multimillion-dollar-company-bde6116a954d https://medium.com/@osmbuildings/why-were-not-going-to-suppo... [4] https://leafletjs.com/ https://leafletjs.com/
- cldellow 3y ago
  It seems unlikely to me that the OSMF did something shady in exchange for the promise of 6,000 Euros from Cesium. The OSM Buildings folks called themselves "OSM Buildings", using the OSM trademark. They offered a commercial service. The OSMF was OK with this. It seems the OSMF is OK with a lot of things -- including permitting Cesium to use the OSM mark in their own commercial project. Live and let live, may a thousand flowers bloom, etc. Rather than reflect on the fact that their name was not really theirs, but relied on the goodwill of the OSM mark, the OSM Buildings folks decided to ragequit and shut down their project, dramatically claiming that their name was "seized by a multimillion dollar company"? The whole thing is bizarre to me.
  - neomantra 3y ago
    I was trying to understand it from both sides and not getting much. Some of your color helps too (like the euro amounts). It all was bizarre and sadly the whole thing blocked some progress. I had come back to temper my response because for sure this is great work by the tech team.
Humphrey 3y ago
I'm excited for this! I have noticed that most apps that use OSM data provide offline vector downloads for each region. Are most of these apps building their own vector databases? I assume that OSM themselves providing vector tiles will be game changing, allowing app developers to skip creating their own vector downloads as they could just hook into the live vector tiles. No doubt this will make providing OSM maps a much simpler process.
- cldellow 3y ago
  You often want to build your own tiles for specialized applications. Any map schema is a tradeoff -- if you include every feature in every tile at every zoom level so that the renderer can pick and choose what to show, you bloat the tile file size. The blog post is very light on details. My intuition is that this is primarily focused on powering the map on osm.org in service of people who edit OSM getting to see faster updates. This would be in keeping with their current raster tiles, which are not meant for use in "real" applications. There are enough technical challenges in rolling out vector tiles with minutely updates for editors on OSM -- in my opinion, they should solve it for OSM first before thinking about committing to a specific schema for public use by other apps. Still, anything that incentivizes innovation on vector tiles is pretty exciting. One option that could be interesting: provide "fat" builds of tiles that have every feature, and create tooling to do something akin to treeshaking so people can simplify the resulting tiles to only contain the features they need for their use case.
- orblivion 3y ago
  I recently made a self-hostable web app* that works the same way; the user downloads regions with a point and click, I have the data on S3. I used Protomaps, which is vector library for the browser. I did look into seeing if I could just download the maps straight from Protomaps. I didn't end up doing so, I think it's because they didn't seem to have an API that I felt like I could scrape wholesale. But also, it turns out that there's a question of how that data is generated. There's a certain schema of what sorts of things show up in the browser, perhaps with styles giving options for how to display them. But which raw OSM data you decide to include, and which visual element it corresponds to, is up to the producer of that data. So thinking of Organic Maps and OsmAnd, I bet they choose slighly different things and probably change it over time. * https://sandstorm.org/news/2023-12-05-osm-on-sandstorm https://sandstorm.org/news/2023-12-05-osm-on-sandstorm
deleted 3y ago
[deleted]
dewey 3y ago
Slightly related to OSM, can anyone recommend a good and polished iOS app? I recently tried to use OSM more but the web view on mobile was impossible to use. I was not able to navigate from point A to B with it.
- tsunagatta 3y ago
  I’ve had good luck with Organic Maps [1] on iOS. [1] https://apps.apple.com/us/app/organic-maps-offline-hike-bike/id1567437057 https://apps.apple.com/us/app/organic-maps-offline-hike-bike...
  - dewey 3y ago
    Looks good, thank you!
- Tagbert 3y ago
  I’m curious why you wouldn’t use the Apple Maps app or something else? Is it to promote open software?
  - dewey 3y ago
    There’s some hiking trails in the city that are only on OSM as they are not “official”, that’s why sometimes it’s nice to have a choice. I’m back on Google Maps as the POI data in Apple Maps is unfortunately very incomplete (Opening times, reviews, new restaurants etc.) even in bigger cities like Berlin.
    - Tagbert 3y ago
      Makes sense. POI data coverage varies from place to place.
- class3shock 3y ago
  Having tried multiple times to move away from Gmaps on Android, all the OSM based apps I've tried are barely usable at best (sadly).
simonw 3y ago
If anyone wants to experiment with vector OSM data today I can very firmly recommend Protomaps - I wrote about that here https://til.simonwillison.net/gis/pmtiles https://til.simonwillison.net/gis/pmtiles Here's a vector map I built with it of Half Moon Bay California https://simonw.github.io/hmb-map/ https://simonw.github.io/hmb-map/
- jakelazaroff 3y ago
  Oh this is awesome! Bookmarking for later, thank you for the link!
agumonkey 3y ago
OSM is a great project
jillesvangurp 3y ago
The maps on openstreetmap are nice of course. But most websites using openstreetmap don't get their maps from there but from companies like mapbox, maptiler, and others that typically already provided vector maps. Openstreetmap itself has been lagging behind a bit. Protomaps, mentioned elsewhere in this thread, is awesome for self hosting vector maps. The way it works is that you upload a single large pbf (protobuf) file with all map content and then use lambda functions to extract vector tiles from there. Put a CDN in front of it and you have cheap map hosting. Amazing stuff. I met Brandon Liu, the person that built this, some time ago at a meetup. Well worth checking out his stuff. There's also https://demo.f4map.com/#lat=47.5933049&lon=12.1748491&zoom=18&camera.phi=119.462 https://demo.f4map.com/#lat=47.5933049&lon=12.1748491&zoom=1..., which is a 3D rendering engine filled with openstreetmap data. Pretty nice stuff and be sure to fiddle with the options to change the time of day, turn on the moving ships, etc.
- hokkos 3y ago
  you don't need a lambda function if it can use the range header to get what it needs, also it is an aggregated pbf on a pyramid of space filling hilbert curve.
  - globular-toast 3y ago
    Indeed, you can use pmtiles files on a completely static site which is the main benefit for me. I wrote it up on my blog: https://blog.gpkb.org/posts/hugo-static-site-maps/ https://blog.gpkb.org/posts/hugo-static-site-maps/ I have since started making my own pmtiles from pbf extracts using planetiler, though. I should add that as an addendum...
    - jillesvangurp 3y ago
      We're actually looking into using hugo for our main website currently. However, running pm-tiles at build/deploy time for the whole world is a bit expensive and time consuming. The point of using lambda functions is to run it on demand for just the tiles that are needed and then cache the results via a CDN.
tobilg 3y ago
Have a look at https://www.serverlessmaps.com/ https://www.serverlessmaps.com/ which uses PMTiles to generate the tiles from OSM data. It's also possible to host the PMTiles directly on S3 and use HTTP range requests to retrieve the desired bounding boxes. Under the hood, it uses CloudFront as CDN, Lambda@Edge functions and S3 (see architecture at https://github.com/serverlessmaps/serverlessmaps/#architecture https://github.com/serverlessmaps/serverlessmaps/#architectu...) It's a very cost-effective solution as CloudFront has a 1TB/Month free tier, compared to hosted solutions.
Freak_NL 3y ago
Way overdue. OpenStreetMap's website at openstreetmap.org is its calling card, and for the past few years the default style shown (called Carto) has all but stagnated in development. Accepted features like highway=busway (introduced three years ago) are not rendered there because the maintainers can no longer be bothered, or dislike the tag personally despite broad community backing. What worries me for this new effort is that Paul Norman is one of the two remaining Carto sometimes-active maintainers who refuse to merge contributed PRs or even provide alternative minimal support for features like highway=busway, leading to awkward gaps on the baseline map shown on openstreetmap.org. I would love to be surprised in a positive way about this new effort, but I'm not holding my hopes up. Thankfully OpenStreetMap can be thoroughly useful in apps like OsmAnd and OrganicMaps, and the tile-based Tracestrack Topo layer on openstreetmap.org is getting quite decent: https://www.openstreetmap.org/#layers=P https://www.openstreetmap.org/#layers=P
- RicoElectrico 3y ago
  Apparently even Paul Norman is tired of contributing at openstreetmap-carto. Honestly of the maintainers imagico is the one I'd say is impossible to reason with.
  - Freak_NL 3y ago
    Oh absolutely. I have given up on engaging with him, it is pointless (I am jdhoek on GitHub, and have contributed various commits to Carto in the past). While I can understand that being a maintainer for a project like Carto can be taxing due to the high amount of feature requests which are often not suitable for the general purpose map, lack usage, or are otherwise unclear in their definition, I find that the way Carto has been maintained for the past few years has been harmful to the OpenStreetMap project in general. I think Christopher Hormann (imagico), despite his contributions in the past, is actively harming OpenStreetMap with his maintainership and (lack of) actions. Although to be fair the OpenStreetMap Foundation shares some of the blame here by shrugging 'not our problem' when the state of the default layer on openstreetmap.org is, repeatedly, pointed out, and letting this state of affairs persist. The typical response of 'Oh, OpenStreetMap is a do-ocracy, haha, if you want to change this, just create your own layer!' does not befit a project which has achieved so much as OpenStreetMap.
7e 3y ago
I don't use OpenStreeMap because their data quality is poor.
- habi 3y ago
  That really depends what you plan to use where. Roads and foot navigation is generally quite good, POIs is good to use in large cities. Where do you see OSM data lacking?
  - 7e 3y ago
    Nope. It took three years to update a tunnel in my local area. Even now, other streets I've sampled which less than ten years old are wrong. What you're saying is that government data is good, which OpenStreetMap has taken. But that data doesn't update very frequently.
    - Doctor_Fegg 3y ago
      > It took three years to update a tunnel in my local area. The idea is that you can update it yourself?

345 points by RicoElectrico 3y ago

I hope they come up with a nice, clean, standard for a vector tile format. I haven't spent a lot of time looking into it. But the last time I looked at commonly-used vector tile formats, the one I found was clunky to generate/parse. (And/or, maybe I just missed the right one. Please correct me if there is something out there that is clean and simple -- and has common library support.)

philips 3y ago
Have you tried pmtiles? https://docs.protomaps.com/ https://docs.protomaps.com/
philips 3y ago
Also a post from Paul Normans, mentioned in the parent post, covers some options. https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/ https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/
- somishere 3y ago
  Not an expert, however (lol) - even if the recommendations here are solid - this doesn't feel like a particularly well-reasoned comparison. IMHO maintaining a large number of files on disk (be it 16m or 1.5b) should be a non-starter (not to mention that zipping them to slightly reduce the count removes one of the benefits of having them in the first place). A database makes sense - as does SOC between generation and distribution if it brings benefit - but immediately writing off SQLite because of write performance[1]? Similarly the arguments against pmtiles feel outdated and ignorant of the developing nature of the protocol (appreciate it's a 2yo post!). OSM deciding on a format here has potential to significantly shape things to come. Hopefully Paul has progressed his thinking a bit since this was written. [1] https://blog.wesleyac.com/posts/consider-sqlite https://blog.wesleyac.com/posts/consider-sqlite
  - cldellow 3y ago
    The SQLite limitations might also be solvable by sharding the databases (so each sees 1/N of the write volume), tiering them (so tiles that are written most recently are in file A, then file B, then file C), or a combination of these approaches. I'm assuming the requirement is not really to build a single .mbtiles file, but to have some way for a process to serve the correct tile when given a z/x/y tuple.
  - bdon 3y ago
    Most of the development for pmtiles has happened in the last 2 years, including a maturing server implementation (http://github.com/protomaps/go-pmtiles http://github.com/protomaps/go-pmtiles) but some key parts are still missing like the ability to decode an archive in native mobile applications. sqlite (mbtiles) already has ~10+ years of integration into the mapping ecosystem so that still works better if you want to move tilesets around to desktop applications and mobile devices. The situation Paul is addressing is one unique to OpenStreetMap itself, which is minute-level updates of a global scale tileset. This is a use case pmtiles is designed explicitly not to address, where using a database is a better fit.
    - somishere 3y ago
      Appreciate you weighing in directly Brandon! To be clear though, I didn't mean to say that protomaps was potentially a better solution (of course an uncompressed DB makes much more sense), simply that the developing nature of the protocol meant that it's inclusion within the "ecosystem" shouldn't be a closed case :) Which is to say, distribution of OSM data feels like a large part of the process. Of course there are various bottlenecks / considerations around edits / writes, but in practice surely reads are the bigger vector. I wonder how many OSM-external use cases rely on "minutely" updates or need the full fidelity of the raw data source. Feels like there is a solid case for providing less frequent (hourly, daily?) official updates via official "single-file" formats that could be widely distributed to the benefit of all, and e.g. allow OSM to loosen up their hot-linking policies and ensure continued investment in the chosen protocols. But mainly I was questioning how a somewhat proven format like SQLite, with its many benefits (interoperability, distribution, etc.), would be so easily dropped from consideration without even a test having been run. Just my thoughts of course!
penorman 3y ago
Mapbox Vector Tiles are, for better or worse, the industry standard. They have good library support and are good enough that no one is rushing to reinvent them. If you want to distribute sets of tiles to people, PMTiles is the best container format as you can just throw the file on hosting that supports HTTP Range requests (e.g. S3) and add the appropriate plugins to your client library. If you have to support clients which can't do the range requests from pmtiles, you can throw a thin layer in front of it to turn HTTP z/x/y requests into the pmtiles range requests. There are no good file formats for distributing minutely updates of tiles but there's no need for this. If you are keeping your tiles up to date up to the minute you have a reasonable internet connection and you can just download individual tiles instead.
- timmg 3y ago
  Thanks! I just browsed the spec again (https://github.com/mapbox/vector-tile-spec/tree/master/2.1 https://github.com/mapbox/vector-tile-spec/tree/master/2.1) -- I didn't have as bad of a reaction as I did the first time I looked at it. I guess the idea of using (Google) protobufs for the "packaging" and then a super tight, custom, byte-encoded format for drawing commands seems... weird to me. But, honestly, it probably makes sense.

ajsnigrutin 3y ago

Slightly offtopic I have OsmAnd ( https://github.com/osmandapp/OsmAnd https://github.com/osmandapp/OsmAnd ) on my phone, download the basemaps, download my (small) country data (both sourced from open street maps), and with an app + ~1GB of data, I get the maps and full navigation within my country, POIs, etc., and can add other countries when needed. Is there something similar for a PC? I can download data from open street maps, but then I need postgres, postgis, a tile server and styles and apache running just to generate the tiles. Is there anything portable (short of running osmand in an android virtualbox) for offline navigation on a linux pc? QGIS can display vectors, but I wasn't able to easily style the data... navigation is a no-go there too. anything else?

dividedbyzero 3y ago
Not Linux, so it's probably not going to help you much, but the OsmAnd iPad app seems to work just fine on macOS and afaik those run as native apps on M1/2/3 machines, and this particular one feels very similar to the macOS-native Maps app. I guess that's pretty close for those on a Macbook, though turn-by-turn navigation is going to suck with no onboard GPS. I just got it from the App Store to test this and I think I'm going to keep it, it's actually pretty neat.
lol768 3y ago
You can download an extract of your country from Geofabrik, run it through Tilemaker (https://github.com/systemed/tilemaker https://github.com/systemed/tilemaker) to get a nice mbtiles file, and then use the built-in Ruby server to give you something you can load in your web-browser locally.
- stereo 3y ago
  Post author here. Geofabrik has even started providing extracts in the Shortbread vector schema, the same schema we'll be using for our vector tiles.
maxerickson 3y ago
There's a port of OSMAnd, looks like it has stagnated. https://sourceforge.net/projects/offroadosm/files/ https://sourceforge.net/projects/offroadosm/files/

clementmas 3y ago

Which tech stack are they planning to use? It seems like Planetiler and tilemaker are popular options but do they scale for OSM's needs? Protomaps PMtiles are also very interesting

maxerickson 3y ago
Probably something else that is straightforward to incrementally update (applies to all 3 cases from your comment). https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/ https://www.paulnorman.ca/blog/2022/10/tilekiln-storage/
- clementmas 3y ago
  Thanks. I found this article on Paul's blog which answers my question: https://www.paulnorman.ca/blog/2024/01/minutely-updated-tiles https://www.paulnorman.ca/blog/2024/01/minutely-updated-tile...
stereo 3y ago
Post author here! We’re going to reuse a lot of existing bricks like osm2pgsql, postgis’s ST_AsMVT, and the shortbread schema to start with. Tilekiln, the project Paul is working on, is python to orchestrate it all, and regenerate tiles on database updates.

nzeid 3y ago

As a religious OsmAnd user I welcome this. I'll be donating directly to the effort.

sorenjan 3y ago
But OsmAnd already uses their own vector tiles.

neomantra 3y ago

I happened to be using OpenStreetMaps for the last couple weeks and released a Golang OpenAPI library for OpenChargeMap [1] In that, I released an IPython notebook and used that to visualize the data on Vision Pro [2]. And I did get it working! But it was not near as cool as it could be because of two dramatic events: OSM Foundation violated the trust (nice way to put it) of a participant, OSMBuildings.org, in the search for sponsorship [3] (four years ago, such lost progress :( ) The other is that LeafletJS’s developer is Ukrainian and can’t advance his project [4]. Please consider those two issues when contemplating donating to OSM due to this press release. While I applaud OSM abstractly, it exists because of a massive amounts of public contributions of IP, not because of $$ from a commercial sponsor. [1] https://github.com/neomantra/go-openchargemap https://github.com/neomantra/go-openchargemap [2] https://github.com/neomantra/go-openchargemap/blob/main/examples/ocm_3d.ipynb https://github.com/neomantra/go-openchargemap/blob/main/exam... [3] https://medium.com/@osmbuildings/why-were-not-going-to-support-a-multimillion-dollar-company-bde6116a954d https://medium.com/@osmbuildings/why-were-not-going-to-suppo... [4] https://leafletjs.com/ https://leafletjs.com/

cldellow 3y ago
It seems unlikely to me that the OSMF did something shady in exchange for the promise of 6,000 Euros from Cesium. The OSM Buildings folks called themselves "OSM Buildings", using the OSM trademark. They offered a commercial service. The OSMF was OK with this. It seems the OSMF is OK with a lot of things -- including permitting Cesium to use the OSM mark in their own commercial project. Live and let live, may a thousand flowers bloom, etc. Rather than reflect on the fact that their name was not really theirs, but relied on the goodwill of the OSM mark, the OSM Buildings folks decided to ragequit and shut down their project, dramatically claiming that their name was "seized by a multimillion dollar company"? The whole thing is bizarre to me.
- neomantra 3y ago
  I was trying to understand it from both sides and not getting much. Some of your color helps too (like the euro amounts). It all was bizarre and sadly the whole thing blocked some progress. I had come back to temper my response because for sure this is great work by the tech team.

Humphrey 3y ago

I'm excited for this! I have noticed that most apps that use OSM data provide offline vector downloads for each region. Are most of these apps building their own vector databases? I assume that OSM themselves providing vector tiles will be game changing, allowing app developers to skip creating their own vector downloads as they could just hook into the live vector tiles. No doubt this will make providing OSM maps a much simpler process.

cldellow 3y ago
You often want to build your own tiles for specialized applications. Any map schema is a tradeoff -- if you include every feature in every tile at every zoom level so that the renderer can pick and choose what to show, you bloat the tile file size. The blog post is very light on details. My intuition is that this is primarily focused on powering the map on osm.org in service of people who edit OSM getting to see faster updates. This would be in keeping with their current raster tiles, which are not meant for use in "real" applications. There are enough technical challenges in rolling out vector tiles with minutely updates for editors on OSM -- in my opinion, they should solve it for OSM first before thinking about committing to a specific schema for public use by other apps. Still, anything that incentivizes innovation on vector tiles is pretty exciting. One option that could be interesting: provide "fat" builds of tiles that have every feature, and create tooling to do something akin to treeshaking so people can simplify the resulting tiles to only contain the features they need for their use case.
orblivion 3y ago
I recently made a self-hostable web app* that works the same way; the user downloads regions with a point and click, I have the data on S3. I used Protomaps, which is vector library for the browser. I did look into seeing if I could just download the maps straight from Protomaps. I didn't end up doing so, I think it's because they didn't seem to have an API that I felt like I could scrape wholesale. But also, it turns out that there's a question of how that data is generated. There's a certain schema of what sorts of things show up in the browser, perhaps with styles giving options for how to display them. But which raw OSM data you decide to include, and which visual element it corresponds to, is up to the producer of that data. So thinking of Organic Maps and OsmAnd, I bet they choose slighly different things and probably change it over time. * https://sandstorm.org/news/2023-12-05-osm-on-sandstorm https://sandstorm.org/news/2023-12-05-osm-on-sandstorm

deleted 3y ago

[deleted]

dewey 3y ago

Slightly related to OSM, can anyone recommend a good and polished iOS app? I recently tried to use OSM more but the web view on mobile was impossible to use. I was not able to navigate from point A to B with it.

tsunagatta 3y ago
I’ve had good luck with Organic Maps [1] on iOS. [1] https://apps.apple.com/us/app/organic-maps-offline-hike-bike/id1567437057 https://apps.apple.com/us/app/organic-maps-offline-hike-bike...
- dewey 3y ago
  Looks good, thank you!
Tagbert 3y ago
I’m curious why you wouldn’t use the Apple Maps app or something else? Is it to promote open software?
- dewey 3y ago
  There’s some hiking trails in the city that are only on OSM as they are not “official”, that’s why sometimes it’s nice to have a choice. I’m back on Google Maps as the POI data in Apple Maps is unfortunately very incomplete (Opening times, reviews, new restaurants etc.) even in bigger cities like Berlin.
  - Tagbert 3y ago
    Makes sense. POI data coverage varies from place to place.
class3shock 3y ago
Having tried multiple times to move away from Gmaps on Android, all the OSM based apps I've tried are barely usable at best (sadly).

simonw 3y ago

If anyone wants to experiment with vector OSM data today I can very firmly recommend Protomaps - I wrote about that here https://til.simonwillison.net/gis/pmtiles https://til.simonwillison.net/gis/pmtiles Here's a vector map I built with it of Half Moon Bay California https://simonw.github.io/hmb-map/ https://simonw.github.io/hmb-map/

jakelazaroff 3y ago
Oh this is awesome! Bookmarking for later, thank you for the link!

agumonkey 3y ago

OSM is a great project

jillesvangurp 3y ago

The maps on openstreetmap are nice of course. But most websites using openstreetmap don't get their maps from there but from companies like mapbox, maptiler, and others that typically already provided vector maps. Openstreetmap itself has been lagging behind a bit. Protomaps, mentioned elsewhere in this thread, is awesome for self hosting vector maps. The way it works is that you upload a single large pbf (protobuf) file with all map content and then use lambda functions to extract vector tiles from there. Put a CDN in front of it and you have cheap map hosting. Amazing stuff. I met Brandon Liu, the person that built this, some time ago at a meetup. Well worth checking out his stuff. There's also https://demo.f4map.com/#lat=47.5933049&lon=12.1748491&zoom=18&camera.phi=119.462 https://demo.f4map.com/#lat=47.5933049&lon=12.1748491&zoom=1..., which is a 3D rendering engine filled with openstreetmap data. Pretty nice stuff and be sure to fiddle with the options to change the time of day, turn on the moving ships, etc.

hokkos 3y ago
you don't need a lambda function if it can use the range header to get what it needs, also it is an aggregated pbf on a pyramid of space filling hilbert curve.
- globular-toast 3y ago
  Indeed, you can use pmtiles files on a completely static site which is the main benefit for me. I wrote it up on my blog: https://blog.gpkb.org/posts/hugo-static-site-maps/ https://blog.gpkb.org/posts/hugo-static-site-maps/ I have since started making my own pmtiles from pbf extracts using planetiler, though. I should add that as an addendum...
  - jillesvangurp 3y ago
    We're actually looking into using hugo for our main website currently. However, running pm-tiles at build/deploy time for the whole world is a bit expensive and time consuming. The point of using lambda functions is to run it on demand for just the tiles that are needed and then cache the results via a CDN.

tobilg 3y ago

Have a look at https://www.serverlessmaps.com/ https://www.serverlessmaps.com/ which uses PMTiles to generate the tiles from OSM data. It's also possible to host the PMTiles directly on S3 and use HTTP range requests to retrieve the desired bounding boxes. Under the hood, it uses CloudFront as CDN, Lambda@Edge functions and S3 (see architecture at https://github.com/serverlessmaps/serverlessmaps/#architecture https://github.com/serverlessmaps/serverlessmaps/#architectu...) It's a very cost-effective solution as CloudFront has a 1TB/Month free tier, compared to hosted solutions.

Freak_NL 3y ago

Way overdue. OpenStreetMap's website at openstreetmap.org is its calling card, and for the past few years the default style shown (called Carto) has all but stagnated in development. Accepted features like highway=busway (introduced three years ago) are not rendered there because the maintainers can no longer be bothered, or dislike the tag personally despite broad community backing. What worries me for this new effort is that Paul Norman is one of the two remaining Carto sometimes-active maintainers who refuse to merge contributed PRs or even provide alternative minimal support for features like highway=busway, leading to awkward gaps on the baseline map shown on openstreetmap.org. I would love to be surprised in a positive way about this new effort, but I'm not holding my hopes up. Thankfully OpenStreetMap can be thoroughly useful in apps like OsmAnd and OrganicMaps, and the tile-based Tracestrack Topo layer on openstreetmap.org is getting quite decent: https://www.openstreetmap.org/#layers=P https://www.openstreetmap.org/#layers=P

RicoElectrico 3y ago
Apparently even Paul Norman is tired of contributing at openstreetmap-carto. Honestly of the maintainers imagico is the one I'd say is impossible to reason with.
- Freak_NL 3y ago
  Oh absolutely. I have given up on engaging with him, it is pointless (I am jdhoek on GitHub, and have contributed various commits to Carto in the past). While I can understand that being a maintainer for a project like Carto can be taxing due to the high amount of feature requests which are often not suitable for the general purpose map, lack usage, or are otherwise unclear in their definition, I find that the way Carto has been maintained for the past few years has been harmful to the OpenStreetMap project in general. I think Christopher Hormann (imagico), despite his contributions in the past, is actively harming OpenStreetMap with his maintainership and (lack of) actions. Although to be fair the OpenStreetMap Foundation shares some of the blame here by shrugging 'not our problem' when the state of the default layer on openstreetmap.org is, repeatedly, pointed out, and letting this state of affairs persist. The typical response of 'Oh, OpenStreetMap is a do-ocracy, haha, if you want to change this, just create your own layer!' does not befit a project which has achieved so much as OpenStreetMap.

7e 3y ago

I don't use OpenStreeMap because their data quality is poor.

habi 3y ago
That really depends what you plan to use where. Roads and foot navigation is generally quite good, POIs is good to use in large cities. Where do you see OSM data lacking?
- 7e 3y ago
  Nope. It took three years to update a tunnel in my local area. Even now, other streets I've sampled which less than ten years old are wrong. What you're saying is that government data is good, which OpenStreetMap has taken. But that data doesn't update very frequently.
  - Doctor_Fegg 3y ago
    > It took three years to update a tunnel in my local area. The idea is that you can update it yourself?