4 ms·

How can I generate raster tiles myself for a specific region, so I don't have to use planet.osm? I can find regions on geofabrik.

by jokoon 6y ago

bdon 6y ago
Most production raster tile systems use osm2pgsql + Mapnik. These are not easy to install. You may need to generate two or three sets of raster tiles for different pixel densities. The total number of tiles will explode past around zoom level 15. https://switch2osm.org https://switch2osm.org is a good intro. The newer way to do raster tiles is to actually generate vector tiles first, and then do the rasterization dynamically. Mapnik can do this, renderers like MapboxGL can be run in headless mode to do this as well. Yes, I wish this was easier too.
- berkes 6y ago
  In addition, modern browsers can display vector tiles directly. Mobile and apps have a somewhat harder time with this. https://www.maptiler.com/ https://www.maptiler.com/ has a good introduction for this wrt Open Street Map.
  - bdon 6y ago
    Vector tiles with OpenLayers doesn't feel as smooth to me compared to MapboxGL or Tangram https://openlayers.org/en/latest/examples/mapbox-style.html https://openlayers.org/en/latest/examples/mapbox-style.html but it seems to be improving rapidly.
    - for_xyz 6y ago
      Last time I checked (almost 2 years ago) OpenLayers did not support webgl backend for vector graphics. That's why it was slower.
  - jokoon 6y ago
    Aren't those vector tiles a little heavy, since they're likely json? Of course I can guess a lot of data can be removed and simplified, but still, PNG seem lighter...
    - durkie 6y ago
      It depends on the application, but I don't think they're necessarily larger in size. I believe the format is quite compressed (Mapbox Vector Tile/MVT, which I believe is a variant of protocol buffers). Many vector tile encoders have wide flexibility in geometry simplification/removal depending on zoom layer and this helps big time with tile size. I find them to be quite nice in lower bandwidth situations because you can overzoom them and still have them look good: if you zoom in and the tiles for the next zoom level haven't loaded, the vector geometry is still able to look decent, rather than pixellated like a raster tile.
      - jokoon 6y ago
        Compressed? Are you saying decompression is done with JS?
      - nbadg 6y ago
        You're mixing up the formats slightly: MVT is sqlite with a particular schema; OSM raw data is PBF with a particular schema.
        durkie 6y ago
        Isn't mbtiles sqlite?
        nbadg 6y ago
        It might be that pbfs are stored inside sqlite? The whole thing is really incredibly confusing to be honest. What I can tell you is that my vector slippymap XHR is requesting .pbf files, so that would definitely imply protobuffers as the wire format. Ultimately that's using t-rex and mapbox-gl.js, so... not sure if there's something intermediate. But the mapbox docs say sqlite? I'm just very confused now.
- kylebarron 6y ago
  A popular open source server-side rasterization program is tileserver-gl: https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl
- jokoon 6y ago
  > Yes, I wish this was easier too. I've used spatialite, and even though the doc is poor, I've managed to do things with it, and it works fine. I'm not sure I can generate tiles with QGIS and spatialite. It's true that open source GIS software is not easy to find, since GIS is not easy in the first place.
kristopolous 6y ago
You can map layers using different sources. You need to use fromlatlng to do the mapping. You can define your source so long as you use the tile server protocol (TMS). You don't need everything, you just provide the areas you want and layer accordingly I'm on my mobile typing this but hopefully that's enough to get you started. It should probably take less than a day. There's certainly nuanced GIS wonkiness, let's not pretend otherwise. But when I did set up a tile server I remember it being one of those things you think you simply cannot do without taking like 6 months of cartography night school, but then 45 minutes later it's done. The domain specific knowledge requirements for standard usage required lightly scanning a couple Wikipedia pages. nothing serious.
srrr 6y ago
I do a cycle map with my own style and these are my steps and tools: 1) Download a region from geofabrik. 2) Extract, transform and load the data into pgsql with osm2pgsql. I use the osm2pgsql flex mode. QGis can load osm extracts direcly, but the resulting layers have many errors in the details and using pgsql is much faster while viewing and rendering. Imposm is also a good alternative to osmpgsql. 3) Use QGis for styling the map. 4) Generate static tiles with https://docs.qgis.org/3.10/en/docs/user_manual/processing_algs/qgis/rastertools.html https://docs.qgis.org/3.10/en/docs/user_manual/processing_al... (With DPI 250 and a tilesize of 512x512 for use on high resolution screens.) 5) Upload the tiles to my own server. 6) Load the tiles in OSMAnd as a custom map. Qgis has a build in tile server https://www.qgis.org/de/site/about/features.html#qgis-server https://www.qgis.org/de/site/about/features.html#qgis-server and leaflet can load the tiles via https://leafletjs.com/reference-1.6.0.html#tilelayer-wms https://leafletjs.com/reference-1.6.0.html#tilelayer-wms . But this does not work for OSMAnd. QGis can also create beautiful print maps with the "Atlas" feature.
- jokoon 6y ago
  Can I use spatialite coupled with QGIS to generate those tiles? How well does it perform?
  - srrr 6y ago
    On my system spatialite is 10% the speed with the Germany extract compared to PostgreSQL. Bigger extracts don't work properly. But loading data directly from a osm.pbf or loading data via the QGis QuickOSM plugin into a spatialite file gives two problems: 1) You can not transform the data to fit your personal usage. Raw osm data is not nice to work with. 2) I had many problems with areas stored as relations (example: islands in lakes) and relation handling overall is a mess. For the first steps and to toy around osm data in spatialite works. But don't fear osm2pgsql. For standard use cases it's a oneliner.
OmarShehata 6y ago
(Disclaimer, I work at Cesium) we've been working on making this easier. If you have a bunch of GeoTIFFs of satellite imagery you can drag and drop them into our platform and get WMTS/TMS tiles you can stream into whatever client. See: https://cesium.com/docs/tutorials/uploading/ https://cesium.com/docs/tutorials/uploading/ Would love to hear your feedback.

How can I generate raster tiles myself for a specific region, so I don't have to use planet.osm? I can find regions on geofabrik.

by jokoon 6y ago

How can I generate raster tiles myself for a specific region, so I don't have to use planet.osm? I can find regions on geofabrik.

bdon 6y ago

Most production raster tile systems use osm2pgsql + Mapnik. These are not easy to install. You may need to generate two or three sets of raster tiles for different pixel densities. The total number of tiles will explode past around zoom level 15. https://switch2osm.org https://switch2osm.org is a good intro. The newer way to do raster tiles is to actually generate vector tiles first, and then do the rasterization dynamically. Mapnik can do this, renderers like MapboxGL can be run in headless mode to do this as well. Yes, I wish this was easier too.

berkes 6y ago
In addition, modern browsers can display vector tiles directly. Mobile and apps have a somewhat harder time with this. https://www.maptiler.com/ https://www.maptiler.com/ has a good introduction for this wrt Open Street Map.
- bdon 6y ago
  Vector tiles with OpenLayers doesn't feel as smooth to me compared to MapboxGL or Tangram https://openlayers.org/en/latest/examples/mapbox-style.html https://openlayers.org/en/latest/examples/mapbox-style.html but it seems to be improving rapidly.
  - for_xyz 6y ago
    Last time I checked (almost 2 years ago) OpenLayers did not support webgl backend for vector graphics. That's why it was slower.
- jokoon 6y ago
  Aren't those vector tiles a little heavy, since they're likely json? Of course I can guess a lot of data can be removed and simplified, but still, PNG seem lighter...
  - durkie 6y ago
    It depends on the application, but I don't think they're necessarily larger in size. I believe the format is quite compressed (Mapbox Vector Tile/MVT, which I believe is a variant of protocol buffers). Many vector tile encoders have wide flexibility in geometry simplification/removal depending on zoom layer and this helps big time with tile size. I find them to be quite nice in lower bandwidth situations because you can overzoom them and still have them look good: if you zoom in and the tiles for the next zoom level haven't loaded, the vector geometry is still able to look decent, rather than pixellated like a raster tile.
    - jokoon 6y ago
      Compressed? Are you saying decompression is done with JS?
    - nbadg 6y ago
      You're mixing up the formats slightly: MVT is sqlite with a particular schema; OSM raw data is PBF with a particular schema.
      - durkie 6y ago
        Isn't mbtiles sqlite?
        nbadg 6y ago
        It might be that pbfs are stored inside sqlite? The whole thing is really incredibly confusing to be honest. What I can tell you is that my vector slippymap XHR is requesting .pbf files, so that would definitely imply protobuffers as the wire format. Ultimately that's using t-rex and mapbox-gl.js, so... not sure if there's something intermediate. But the mapbox docs say sqlite? I'm just very confused now.
kylebarron 6y ago
A popular open source server-side rasterization program is tileserver-gl: https://github.com/maptiler/tileserver-gl https://github.com/maptiler/tileserver-gl
jokoon 6y ago
> Yes, I wish this was easier too. I've used spatialite, and even though the doc is poor, I've managed to do things with it, and it works fine. I'm not sure I can generate tiles with QGIS and spatialite. It's true that open source GIS software is not easy to find, since GIS is not easy in the first place.

kristopolous 6y ago

You can map layers using different sources. You need to use fromlatlng to do the mapping. You can define your source so long as you use the tile server protocol (TMS). You don't need everything, you just provide the areas you want and layer accordingly I'm on my mobile typing this but hopefully that's enough to get you started. It should probably take less than a day. There's certainly nuanced GIS wonkiness, let's not pretend otherwise. But when I did set up a tile server I remember it being one of those things you think you simply cannot do without taking like 6 months of cartography night school, but then 45 minutes later it's done. The domain specific knowledge requirements for standard usage required lightly scanning a couple Wikipedia pages. nothing serious.

srrr 6y ago

I do a cycle map with my own style and these are my steps and tools: 1) Download a region from geofabrik. 2) Extract, transform and load the data into pgsql with osm2pgsql. I use the osm2pgsql flex mode. QGis can load osm extracts direcly, but the resulting layers have many errors in the details and using pgsql is much faster while viewing and rendering. Imposm is also a good alternative to osmpgsql. 3) Use QGis for styling the map. 4) Generate static tiles with https://docs.qgis.org/3.10/en/docs/user_manual/processing_algs/qgis/rastertools.html https://docs.qgis.org/3.10/en/docs/user_manual/processing_al... (With DPI 250 and a tilesize of 512x512 for use on high resolution screens.) 5) Upload the tiles to my own server. 6) Load the tiles in OSMAnd as a custom map. Qgis has a build in tile server https://www.qgis.org/de/site/about/features.html#qgis-server https://www.qgis.org/de/site/about/features.html#qgis-server and leaflet can load the tiles via https://leafletjs.com/reference-1.6.0.html#tilelayer-wms https://leafletjs.com/reference-1.6.0.html#tilelayer-wms . But this does not work for OSMAnd. QGis can also create beautiful print maps with the "Atlas" feature.

jokoon 6y ago
Can I use spatialite coupled with QGIS to generate those tiles? How well does it perform?
- srrr 6y ago
  On my system spatialite is 10% the speed with the Germany extract compared to PostgreSQL. Bigger extracts don't work properly. But loading data directly from a osm.pbf or loading data via the QGis QuickOSM plugin into a spatialite file gives two problems: 1) You can not transform the data to fit your personal usage. Raw osm data is not nice to work with. 2) I had many problems with areas stored as relations (example: islands in lakes) and relation handling overall is a mess. For the first steps and to toy around osm data in spatialite works. But don't fear osm2pgsql. For standard use cases it's a oneliner.

OmarShehata 6y ago

(Disclaimer, I work at Cesium) we've been working on making this easier. If you have a bunch of GeoTIFFs of satellite imagery you can drag and drop them into our platform and get WMTS/TMS tiles you can stream into whatever client. See: https://cesium.com/docs/tutorials/uploading/ https://cesium.com/docs/tutorials/uploading/ Would love to hear your feedback.