5 ms·

While useful, this still uses a server to process tile requests. Why not https://github.com/protomaps/PMTiles https://github.com/protomaps/PMTiles ? For the re

by kh_hk 2y ago

While useful, this still uses a server to process tile requests. Why not https://github.com/protomaps/PMTiles https://github.com/protomaps/PMTiles ? For the record, I am currently using maptiler over at https://citybik.es https://citybik.es to serve tiles, and I am evaluating moving to serving static pmtiles on range requests to cut the middle man

wiredfool 2y ago
PMTiles is great, but there's a risk hosting multi-gb files on metered hosting -- a client might not respect range requests and you open yourself up to a denial of wallet attack. I can't really see doing it on S3, unless it's behind a lightweight tile server. I'd do it on hetzner or linode or one of the lower cost bandwidth providers. (edit -- citybik.es is pretty cool, are you backing up to GBFS or a different set of data feeds? e.g. https://github.com/MobilityData/gbfs/blob/master/systems.csv https://github.com/MobilityData/gbfs/blob/master/systems.csv is a list of ~ 800 gbfs feeds, some dockless, some docked I've done a bit of work in this space, feel free to ping me)
- kh_hk 2y ago
  Will keep that in mind if I go that route. GBFS is just one format of the many supported. This is where we keep all instance data https://github.com/eskerda/pybikes/tree/master/pybikes/data https://github.com/eskerda/pybikes/tree/master/pybikes/data . This is the list of GBFS systems we have https://github.com/eskerda/pybikes/blob/master/pybikes/data/gbfs.json https://github.com/eskerda/pybikes/blob/master/pybikes/data/...
- bdon 2y ago
  To mitigate the "denial of wallet" attack on S3 there are a few options: AWS Lambda: https://docs.protomaps.com/deploy/aws https://docs.protomaps.com/deploy/aws lightweight tile server: https://docs.protomaps.com/pmtiles/cli#serve https://docs.protomaps.com/pmtiles/cli#serve
sorenjan 2y ago
Hosting maps of the entire planet in a single file doesn't seem practical. How do you keep it updated? PMTiles is good for small local maps, but I don't think they're meant for this scale.
- kh_hk 2y ago
  I assumed you would keep it updated the same way you do with mbtiles, but I see that mbtiles are an sqlite so I guess you can do atomic updates with mbtiles. Note though that there are many cases on which served tile information rarely changes. A lot of us are serving a once-downloaded mbtile file and are using this just to _not_ use a provider (mapbox, ...)
  - sorenjan 2y ago
    > Note though that there are many cases on which served tile information rarely changes. It depends on what you're doing, but outdated maps can be worse than no maps. On low zoom values that probably works fine, but new buildings and streets are built everyday all over the world.
    - dagw 2y ago
      everyday all over the world If you need updates more often than once a week, you're going to have to take a different approach, since OSM only publish their database exports once a week. In that case you're going to have to use the OSM API to query their database for changes directly and it is only reasonable to track and manage continuous updates on a relatively small surface area. All of this is of course ignoring the fact that there can be a time lag of anything from hours to decades between a house being built and that house being added to OSM.
- dagw 2y ago
  For updates you're probably better off just regenerating and replacing the mbtiles file from scratch every time you want a new version, rather than trying to incrementally update it.
- 2y ago
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]

by kh_hk 2y ago

wiredfool 2y ago

PMTiles is great, but there's a risk hosting multi-gb files on metered hosting -- a client might not respect range requests and you open yourself up to a denial of wallet attack. I can't really see doing it on S3, unless it's behind a lightweight tile server. I'd do it on hetzner or linode or one of the lower cost bandwidth providers. (edit -- citybik.es is pretty cool, are you backing up to GBFS or a different set of data feeds? e.g. https://github.com/MobilityData/gbfs/blob/master/systems.csv https://github.com/MobilityData/gbfs/blob/master/systems.csv is a list of ~ 800 gbfs feeds, some dockless, some docked I've done a bit of work in this space, feel free to ping me)

kh_hk 2y ago
Will keep that in mind if I go that route. GBFS is just one format of the many supported. This is where we keep all instance data https://github.com/eskerda/pybikes/tree/master/pybikes/data https://github.com/eskerda/pybikes/tree/master/pybikes/data . This is the list of GBFS systems we have https://github.com/eskerda/pybikes/blob/master/pybikes/data/gbfs.json https://github.com/eskerda/pybikes/blob/master/pybikes/data/...
bdon 2y ago
To mitigate the "denial of wallet" attack on S3 there are a few options: AWS Lambda: https://docs.protomaps.com/deploy/aws https://docs.protomaps.com/deploy/aws lightweight tile server: https://docs.protomaps.com/pmtiles/cli#serve https://docs.protomaps.com/pmtiles/cli#serve

sorenjan 2y ago

Hosting maps of the entire planet in a single file doesn't seem practical. How do you keep it updated? PMTiles is good for small local maps, but I don't think they're meant for this scale.

kh_hk 2y ago
I assumed you would keep it updated the same way you do with mbtiles, but I see that mbtiles are an sqlite so I guess you can do atomic updates with mbtiles. Note though that there are many cases on which served tile information rarely changes. A lot of us are serving a once-downloaded mbtile file and are using this just to _not_ use a provider (mapbox, ...)
- sorenjan 2y ago
  > Note though that there are many cases on which served tile information rarely changes. It depends on what you're doing, but outdated maps can be worse than no maps. On low zoom values that probably works fine, but new buildings and streets are built everyday all over the world.
  - dagw 2y ago
    everyday all over the world If you need updates more often than once a week, you're going to have to take a different approach, since OSM only publish their database exports once a week. In that case you're going to have to use the OSM API to query their database for changes directly and it is only reasonable to track and manage continuous updates on a relatively small surface area. All of this is of course ignoring the fact that there can be a time lag of anything from hours to decades between a house being built and that house being added to OSM.
dagw 2y ago
For updates you're probably better off just regenerating and replacing the mbtiles file from scratch every time you want a new version, rather than trying to incrementally update it.
2y ago

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]