6 ms·

How come these map projects still use raster tiles? Are there no open source map projects that render the OSM data to vector data, and renders that vector data

How come these map projects still use raster tiles? Are there no open source map projects that render the OSM data to vector data, and renders that vector data on the clients GPU? Maybe raster tiles are better at something I'm missing, but vector maps are easier to style[0], localize, they're sharper, easier to rotate and zoom smoothly. Maybe it's harder than I think to render it on all sorts of clients including mobile? When writing this I found out that MapTiler[1] is maintaining MapLibre GL JS[2], a fork of a Mapbox project to do just that. It would be interesting to see the difference between self hosting raster and vector maps and compare pros and cons. You can even render raster tiles from vector tiles on the server if the client needs it[3]. [0] https://openmaptiles.org/docs/style/mapbox-gl-style-spec/ https://openmaptiles.org/docs/style/mapbox-gl-style-spec/ [1] https://www.maptiler.com/open-source/ https://www.maptiler.com/open-source/ [2] https://github.com/MapLibre/maplibre-gl-js https://github.com/MapLibre/maplibre-gl-js [3] https://openmaptiles.org/docs/host/tileserver-gl/ https://openmaptiles.org/docs/host/tileserver-gl/

chrisdalke 4y ago
You’re absolutely right and here’s an open source self-hosted project for MBTiles: https://github.com/akhenakh/kvtiles https://github.com/akhenakh/kvtiles More info from the author here: https://blog.nobugware.com/post/2019/self_hosted_world_maps/ https://blog.nobugware.com/post/2019/self_hosted_world_maps/ The vector tiles are much smaller in filesize than pixel data and the rendering technology is much better. Almost all the styling is done on the client side so the server is basically just querying a spatial database.
deleted 4y ago
[deleted]
bdon 4y ago
A few reasons: 1) Some of the most popular map library frontends like Leaflet support only raster tile layers out of the box. 2) Quality vector generalization (feature-dropping) is much more difficult than raster generalization. 3) A large fraction of use cases just need a "good enough" basemap without those extra features.
- sorenjan 4y ago
  Seems like MapTiler is maintaining an open source full stack vector alternative, and OpenLayers[0] looks good as well, so maybe it's time for legacy libraries to add vector support, or for users to switch libraries? There's even bindings from Maplibre GL to Leaflet [1]. I at least would find it interesting to see the two compared by someone other than me ;). [0] https://openlayers.org/ https://openlayers.org/ [1] https://github.com/maplibre/maplibre-gl-leaflet https://github.com/maplibre/maplibre-gl-leaflet
  - bdon 4y ago
    Well, I wrote one already for leaflet ;) https://github.com/protomaps/protomaps.js https://github.com/protomaps/protomaps.js TBH, the blocker is not demand for vector rendering but the technical difficulty and performance constraints of doing rasterization in the browser.
    - sorenjan 4y ago
      Nice! Only on Hacker news can you make a comment about some kind of hypothetical programming project and have the reply be "Yes, I've made one of those already". Do you think it's still too early to switch to vector map data for all clients then? Maybe render raster tiles on demand on the server to those clients that can't handle it?
      - bdon 4y ago
        The barriers to adopting vector-everywhere are social and commercial, not technical. There are a couple great public raster services like osm.org's default style and http://maps.stamen.com http://maps.stamen.com. These are 100% free to use, so they get used everywhere, but incur significant expense to the organizations running (paying) for them. There aren't equivalent solutions in vector-land yet... I wrote a bit about this previously: https://protomaps.com/blog/free-tier-maps https://protomaps.com/blog/free-tier-maps
        4y ago
moffkalast 4y ago
The zoom is especially bad with raster OSM right now since they only support zoom up to level 19, which results in a lot of pixelation if you want to go closer than seeing an entire street.
- spiekerooger 4y ago
  But with vector tiles you (normally) start overzooming at level 15, resulting in either much less detail or really large file sizes for a zoom 14 vector tile that would have to carry all the data to be used in overzooming. So raster tiles load much faster and OpenStreetMap retina tiles help against pixelation. And you can - though not advisable for large areas - render raster tiles at much higher zoom levels, e.g. SomeOneElse renders his raster style up to level 28 for the UK.
  - moffkalast 4y ago
    Well that is true, the vector tiles do have the opposite problem of having too much data when zoomed out, but using raster beyond level 21 is just completely wasteful in terms of storage required. Most of the tiles will just be one colour. I'm not sure how level 28 is even possible without taking up multiple HDDs. I still think that the more proper solution would be to instead cap out the vector tiles on the other end, not going lower than say level 5.
    - spiekerooger 4y ago
      Both arguments of yours are quite true, I also don't think that going beyond zoom level 20 or 21 is advisible within a raster tile stack (and I just checked that the UK map I mentioned earlier goes up to zoom 24 and not 28 as I stated, while the software stack (in this case mod_tile&renderd) is prepared for up to zoom 28). I also don't intend to argue against a vector tile stack - both vector and raster have their use cases and pro's and con's. Vector tiles bring much more flexibility but less support in some use cases, raster normally loads/displays much faster (if served pre-rendered) and has universal support on the client side.
lukeqsee 4y ago
Small note of clarification about MapLibre. It's a vendor-neutral, community-led project supported by a lot of different industry participants (including AWS & Meta). https://maplibre.org https://maplibre.org
NikolaNovak 4y ago
Ignorant question - is this why osmnand always looked and performed worse than expected?
- sorenjan 4y ago
  OsmAnd uses offline vector tiles as standard, they support multiple different styles including custom ones. It performs fine on my phone, and it shows off many of the benefits with vector data. It's easy to choose which kind of point of interest you want displayed, hike and bike routes can be colored in, I think it uses much less storage space on the phone. You can add raster maps to OsmAnd if you want to though. https://osmand.net/docs/user/map/vector-maps/ https://osmand.net/docs/user/map/vector-maps/ https://osmand.net/docs/technical/osmand-file-formats/osmand-rendering-style/ https://osmand.net/docs/technical/osmand-file-formats/osmand... https://osmand.net/docs/user/map/raster-maps/ https://osmand.net/docs/user/map/raster-maps/ https://anygis.ru/Web/Html/Osmand_en https://anygis.ru/Web/Html/Osmand_en
  - NikolaNovak 4y ago
    Thx; appthe detail - I guess I'll try it again!
nyuriks 4y ago
Just to set the record straight: MapTiler is *not* the maintainer of MapLibre. MapTiler is *one of* many participating companies in the MapLibre organization, which has a board and a charter [1]. Amazon has donated $300,000 to MapLibre, Facebook - $80,000, TomTom - $10,000, and lots more companies have donated. Most of the code maintenance is done by various volunteers and companies, e.g. Amazon has hired a whole team of engineers to work on MapLibre native, etc. P.S. I came up with the name MapLibre, called people to unite around this effort, and am on the board of the MapLibre organization. [2] [1]: https://maplibre.org/charter/ https://maplibre.org/charter/ [2]: https://twitter.com/nyuriks/status/1336493514646052864 https://twitter.com/nyuriks/status/1336493514646052864
- lagrange77 4y ago
  I'm almost too afraid to ask, but is WebGL rendering really necessary for displaying an interactive 2D vector map as opposed to using SVG or Canvas? Is it, because sth. like a full street network layer would degrade performance too quickly?
  - bdon 4y ago
    If you want to display infinite fractional zoom levels, WebGL is mandatory. That's what most consumers will expect from a high fidelity map application, because that's the way Google and Apple Maps work.
    - basicplus2 4y ago
      High Fidelity.. Pity their road names don't zoom up to some degree.. it is so hard to read the road names, that as an old bugger i need a magnifying glass..
      - spiekerooger 4y ago
        Not vector but raster with leaflet: would https://www.osmap.uk/larger-text/ https://www.osmap.uk/larger-text/ (as an example) help in reading road names?
twelvechairs 4y ago
In addition to the links others have given I'd note that serving dynamically generated Mapbox Vector Tiles (MVT) can be done very easily from a very standard postgis server these days [0]. For me this is the no brainer way to go. Your main problem is likely to be converting from OSM's labyrinthine data structure to something easier to serve. [0] For the explanation and minmal version see https://www.youtube.com/watch?v=t8eVmNwqh7M https://www.youtube.com/watch?v=t8eVmNwqh7M source code here https://github.com/pramsey/minimal-mvt https://github.com/pramsey/minimal-mvt. For a full fledged package see https://github.com/maplibre/martin https://github.com/maplibre/martin or https://github.com/CrunchyData/pg_tileserv https://github.com/CrunchyData/pg_tileserv
gerogerke 4y ago
I would also like to throw in planetiler[0] and tileserver-gl[1] which I am happily running to serve vector tiles for some regions in Germany. [0]: https://github.com/onthegomap/planetiler https://github.com/onthegomap/planetiler [1]: https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl
KronisLV 4y ago
> MapLibre GL JS[2] Here's a nice example of the maps in action (though the other links also have showcases): https://openmaptiles.org/docs/website/maplibre-gl-js/ https://openmaptiles.org/docs/website/maplibre-gl-js/ The transitions seem really smooth and the user experience is pleasant... until I open them on an Android device that came out just 2 years ago (one of those rugged models), in Firefox. I'm sure that the slowness has something to do with rendering vector data, because all of the sudden those maps become unusable - slow zooming, slow loading (possibly the data is fetched reasonably quickly, just slow to display), slow scrolling. In a word, laggy. When looking at an area of my choice, using the vector maps (even after letting them load fully) I get probably less than 10 frames per second so panning feels choppy. On the other hand, the raster maps are basically what you'd get when panning across an image - the latter experience remains reasonably smooth. There, the only disadvantage is the typical swapping out of raster tiles as you zoom in or out, but that's a small nuisance at best, apart from the visual difference in quality. Have a look at OpenStreetMap, which the first link uses for its data: https://www.openstreetmap.org https://www.openstreetmap.org As it currently stands, if we all used only vector data, I feel like a lot of people on lower spec devices would be left out. I wonder whether any serious tests have been made on the battery usage or performance of both types, but even tests this simple show that there are certain problems with the current implementations of vector maps. I actually did a blog post about this just now, with a video example of the two types of maps running side by side: https://blog.kronis.dev/everything%20is%20broken/vector-maps-are-laggy https://blog.kronis.dev/everything%20is%20broken/vector-maps... Note: this isn't meant to be a super accurate comparison, but rather a quick look into some of the immediately apparent practical differences in what the user experience is like.
- sorenjan 4y ago
  That's strange, it works flawlessly on my 2018 phone, although that has a Snapdragon 845 which was considered the fastest SoC available for Android phones at the time. How is the Google Maps, OsmAnd, or Mapbox experience on your phone? They also use vector map data. If you need to support old underpowered devices you can serve raster tiles to them. I don't know how to automatically detect which type to send to a device, one way would be to just time how long it takes to render a frame of vector data and switch to raster maps if the device is too slow, or at least present the user with the option. [0] https://www.mapbox.com/maps/streets https://www.mapbox.com/maps/streets (Press "Preview Style")
birdstheword5 4y ago
Raster tiles are smaller in terms of amount data sent over the wire and work better for users with less powerful computers/ mobile Really if you aren't interacting with the data on the client I would stick with good old rasters - it's harder to get set up with them but they perform way better
qwertox 4y ago
My experience with it is that vector tiles can become very CPU-heavy on the client, specially when moving into 3D space like with MapLibre's terrain layers. It's less computationally expensive to just fetch a pre-rendered tile and drape it onto the map, which keeps rotating, panning an tilting very performant. It doesn't look that nice, so there's the tradeoff. It can also be easier to just cache tiles on the end device and resort to them instead of constantly reassemble the vector layers.

chrisdalke 4y ago

You’re absolutely right and here’s an open source self-hosted project for MBTiles: https://github.com/akhenakh/kvtiles https://github.com/akhenakh/kvtiles More info from the author here: https://blog.nobugware.com/post/2019/self_hosted_world_maps/ https://blog.nobugware.com/post/2019/self_hosted_world_maps/ The vector tiles are much smaller in filesize than pixel data and the rendering technology is much better. Almost all the styling is done on the client side so the server is basically just querying a spatial database.

deleted 4y ago

[deleted]

bdon 4y ago

A few reasons: 1) Some of the most popular map library frontends like Leaflet support only raster tile layers out of the box. 2) Quality vector generalization (feature-dropping) is much more difficult than raster generalization. 3) A large fraction of use cases just need a "good enough" basemap without those extra features.

sorenjan 4y ago
Seems like MapTiler is maintaining an open source full stack vector alternative, and OpenLayers[0] looks good as well, so maybe it's time for legacy libraries to add vector support, or for users to switch libraries? There's even bindings from Maplibre GL to Leaflet [1]. I at least would find it interesting to see the two compared by someone other than me ;). [0] https://openlayers.org/ https://openlayers.org/ [1] https://github.com/maplibre/maplibre-gl-leaflet https://github.com/maplibre/maplibre-gl-leaflet
- bdon 4y ago
  Well, I wrote one already for leaflet ;) https://github.com/protomaps/protomaps.js https://github.com/protomaps/protomaps.js TBH, the blocker is not demand for vector rendering but the technical difficulty and performance constraints of doing rasterization in the browser.
  - sorenjan 4y ago
    Nice! Only on Hacker news can you make a comment about some kind of hypothetical programming project and have the reply be "Yes, I've made one of those already". Do you think it's still too early to switch to vector map data for all clients then? Maybe render raster tiles on demand on the server to those clients that can't handle it?
    - bdon 4y ago
      The barriers to adopting vector-everywhere are social and commercial, not technical. There are a couple great public raster services like osm.org's default style and http://maps.stamen.com http://maps.stamen.com. These are 100% free to use, so they get used everywhere, but incur significant expense to the organizations running (paying) for them. There aren't equivalent solutions in vector-land yet... I wrote a bit about this previously: https://protomaps.com/blog/free-tier-maps https://protomaps.com/blog/free-tier-maps
      - 4y ago

moffkalast 4y ago

The zoom is especially bad with raster OSM right now since they only support zoom up to level 19, which results in a lot of pixelation if you want to go closer than seeing an entire street.

spiekerooger 4y ago
But with vector tiles you (normally) start overzooming at level 15, resulting in either much less detail or really large file sizes for a zoom 14 vector tile that would have to carry all the data to be used in overzooming. So raster tiles load much faster and OpenStreetMap retina tiles help against pixelation. And you can - though not advisable for large areas - render raster tiles at much higher zoom levels, e.g. SomeOneElse renders his raster style up to level 28 for the UK.
- moffkalast 4y ago
  Well that is true, the vector tiles do have the opposite problem of having too much data when zoomed out, but using raster beyond level 21 is just completely wasteful in terms of storage required. Most of the tiles will just be one colour. I'm not sure how level 28 is even possible without taking up multiple HDDs. I still think that the more proper solution would be to instead cap out the vector tiles on the other end, not going lower than say level 5.
  - spiekerooger 4y ago
    Both arguments of yours are quite true, I also don't think that going beyond zoom level 20 or 21 is advisible within a raster tile stack (and I just checked that the UK map I mentioned earlier goes up to zoom 24 and not 28 as I stated, while the software stack (in this case mod_tile&renderd) is prepared for up to zoom 28). I also don't intend to argue against a vector tile stack - both vector and raster have their use cases and pro's and con's. Vector tiles bring much more flexibility but less support in some use cases, raster normally loads/displays much faster (if served pre-rendered) and has universal support on the client side.

lukeqsee 4y ago

Small note of clarification about MapLibre. It's a vendor-neutral, community-led project supported by a lot of different industry participants (including AWS & Meta). https://maplibre.org https://maplibre.org

NikolaNovak 4y ago

Ignorant question - is this why osmnand always looked and performed worse than expected?

sorenjan 4y ago
OsmAnd uses offline vector tiles as standard, they support multiple different styles including custom ones. It performs fine on my phone, and it shows off many of the benefits with vector data. It's easy to choose which kind of point of interest you want displayed, hike and bike routes can be colored in, I think it uses much less storage space on the phone. You can add raster maps to OsmAnd if you want to though. https://osmand.net/docs/user/map/vector-maps/ https://osmand.net/docs/user/map/vector-maps/ https://osmand.net/docs/technical/osmand-file-formats/osmand-rendering-style/ https://osmand.net/docs/technical/osmand-file-formats/osmand... https://osmand.net/docs/user/map/raster-maps/ https://osmand.net/docs/user/map/raster-maps/ https://anygis.ru/Web/Html/Osmand_en https://anygis.ru/Web/Html/Osmand_en
- NikolaNovak 4y ago
  Thx; appthe detail - I guess I'll try it again!

nyuriks 4y ago

Just to set the record straight: MapTiler is *not* the maintainer of MapLibre. MapTiler is *one of* many participating companies in the MapLibre organization, which has a board and a charter [1]. Amazon has donated $300,000 to MapLibre, Facebook - $80,000, TomTom - $10,000, and lots more companies have donated. Most of the code maintenance is done by various volunteers and companies, e.g. Amazon has hired a whole team of engineers to work on MapLibre native, etc. P.S. I came up with the name MapLibre, called people to unite around this effort, and am on the board of the MapLibre organization. [2] [1]: https://maplibre.org/charter/ https://maplibre.org/charter/ [2]: https://twitter.com/nyuriks/status/1336493514646052864 https://twitter.com/nyuriks/status/1336493514646052864

lagrange77 4y ago
I'm almost too afraid to ask, but is WebGL rendering really necessary for displaying an interactive 2D vector map as opposed to using SVG or Canvas? Is it, because sth. like a full street network layer would degrade performance too quickly?
- bdon 4y ago
  If you want to display infinite fractional zoom levels, WebGL is mandatory. That's what most consumers will expect from a high fidelity map application, because that's the way Google and Apple Maps work.
  - basicplus2 4y ago
    High Fidelity.. Pity their road names don't zoom up to some degree.. it is so hard to read the road names, that as an old bugger i need a magnifying glass..
    - spiekerooger 4y ago
      Not vector but raster with leaflet: would https://www.osmap.uk/larger-text/ https://www.osmap.uk/larger-text/ (as an example) help in reading road names?

twelvechairs 4y ago

In addition to the links others have given I'd note that serving dynamically generated Mapbox Vector Tiles (MVT) can be done very easily from a very standard postgis server these days [0]. For me this is the no brainer way to go. Your main problem is likely to be converting from OSM's labyrinthine data structure to something easier to serve. [0] For the explanation and minmal version see https://www.youtube.com/watch?v=t8eVmNwqh7M https://www.youtube.com/watch?v=t8eVmNwqh7M source code here https://github.com/pramsey/minimal-mvt https://github.com/pramsey/minimal-mvt. For a full fledged package see https://github.com/maplibre/martin https://github.com/maplibre/martin or https://github.com/CrunchyData/pg_tileserv https://github.com/CrunchyData/pg_tileserv

gerogerke 4y ago

I would also like to throw in planetiler[0] and tileserver-gl[1] which I am happily running to serve vector tiles for some regions in Germany. [0]: https://github.com/onthegomap/planetiler https://github.com/onthegomap/planetiler [1]: https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl

KronisLV 4y ago

> MapLibre GL JS[2] Here's a nice example of the maps in action (though the other links also have showcases): https://openmaptiles.org/docs/website/maplibre-gl-js/ https://openmaptiles.org/docs/website/maplibre-gl-js/ The transitions seem really smooth and the user experience is pleasant... until I open them on an Android device that came out just 2 years ago (one of those rugged models), in Firefox. I'm sure that the slowness has something to do with rendering vector data, because all of the sudden those maps become unusable - slow zooming, slow loading (possibly the data is fetched reasonably quickly, just slow to display), slow scrolling. In a word, laggy. When looking at an area of my choice, using the vector maps (even after letting them load fully) I get probably less than 10 frames per second so panning feels choppy. On the other hand, the raster maps are basically what you'd get when panning across an image - the latter experience remains reasonably smooth. There, the only disadvantage is the typical swapping out of raster tiles as you zoom in or out, but that's a small nuisance at best, apart from the visual difference in quality. Have a look at OpenStreetMap, which the first link uses for its data: https://www.openstreetmap.org https://www.openstreetmap.org As it currently stands, if we all used only vector data, I feel like a lot of people on lower spec devices would be left out. I wonder whether any serious tests have been made on the battery usage or performance of both types, but even tests this simple show that there are certain problems with the current implementations of vector maps. I actually did a blog post about this just now, with a video example of the two types of maps running side by side: https://blog.kronis.dev/everything%20is%20broken/vector-maps-are-laggy https://blog.kronis.dev/everything%20is%20broken/vector-maps... Note: this isn't meant to be a super accurate comparison, but rather a quick look into some of the immediately apparent practical differences in what the user experience is like.

sorenjan 4y ago
That's strange, it works flawlessly on my 2018 phone, although that has a Snapdragon 845 which was considered the fastest SoC available for Android phones at the time. How is the Google Maps, OsmAnd, or Mapbox experience on your phone? They also use vector map data. If you need to support old underpowered devices you can serve raster tiles to them. I don't know how to automatically detect which type to send to a device, one way would be to just time how long it takes to render a frame of vector data and switch to raster maps if the device is too slow, or at least present the user with the option. [0] https://www.mapbox.com/maps/streets https://www.mapbox.com/maps/streets (Press "Preview Style")

birdstheword5 4y ago

Raster tiles are smaller in terms of amount data sent over the wire and work better for users with less powerful computers/ mobile Really if you aren't interacting with the data on the client I would stick with good old rasters - it's harder to get set up with them but they perform way better

qwertox 4y ago

My experience with it is that vector tiles can become very CPU-heavy on the client, specially when moving into 3D space like with MapLibre's terrain layers. It's less computationally expensive to just fetch a pre-rendered tile and drape it onto the map, which keeps rotating, panning an tilting very performant. It doesn't look that nice, so there's the tradeoff. It can also be easier to just cache tiles on the end device and resort to them instead of constantly reassemble the vector layers.