3 ms·

I'm the developer of Protomaps, to summarize: The latency you see on https://pmtiles.io/?url=https%3A%2F%2Fdata.source.coop%2Fprotomaps%2Fopenstreetmap%2Ftiles

by bdon 2y ago

I'm the developer of Protomaps, to summarize: The latency you see on https://pmtiles.io/?url=https%3A%2F%2Fdata.source.coop%2Fprotomaps%2Fopenstreetmap%2Ftiles%2Fv3.pmtiles#map=4.18/47.77/13.67 https://pmtiles.io/?url=https%3A%2F%2Fdata.source.coop%2Fpro... is representative of how PMTiles works on AWS S3, coming from the us-west-2 region. It will be reasonable to load for those in the western US and likely quite slow from Europe or Oceania. If you want to make a direct comparison of Protomaps to OpenFreeMap, you need to compare serving OpenFreemap with NGINX from btrfs on disk, to running `pmtiles serve` on a `.pmtiles` file on disk, as described here: https://docs.protomaps.com/deploy/server https://docs.protomaps.com/deploy/server The OpenFreeMap page for me (in Taiwan) takes 1-2 seconds per tile, which is more than double the load tile for the PMTiles in us-west-2 example linked above. The best solution to get latency competitive with commercial providers, for a global audience, is to cache tiles at CDN edge locations. I describe automated ways to do that with Cloudflare Workers and AWS Lambda + Cloudfront here: https://docs.protomaps.com/deploy/cloudflare https://docs.protomaps.com/deploy/cloudflare https://docs.protomaps.com/deploy/aws https://docs.protomaps.com/deploy/aws I'm also experimenting with http://tigrisdata.com http://tigrisdata.com which lets you geo-distribute a static storage bucket like in this example: https://pmtiles.io/?url=https%3A%2F%2Fdemo-bucket.protomaps.com%2Flatest.pmtiles#map=0.71/0/0 https://pmtiles.io/?url=https%3A%2F%2Fdemo-bucket.protomaps....

hyperknot 2y ago
No, the ping is 150 ms to us-west-2, and the tiles load in like 5 seconds on a cold start. Of course we cannot test cold start on HN comments because HN is the definition of hot :-) I can imagine workers to be fast, it's the range requests which are super slow. It's also outside of your control, it depends on how Cloudflare and S3 handles range requests to 90 GB files. I think if you could make PMTiles split into files <10 MB, it'd be perfect with range requests.
- bdon 2y ago
  I agree, there are tradeoffs to using static storage - the intended audience for PMTiles is those that prefer using static sites instead of administering a Linux server. I would be interested to see a comparison of Btrfs + nginx serving latency, vs `pmtiles serve` from https://github.com/protomaps/go-pmtiles https://github.com/protomaps/go-pmtiles on a PMTiles archive on disk. That would be a more direct comparison. I think there's potentially some interesting use case for tiles in Btrfs volumes and incremental updates, which I haven't tackled in PMTiles yet!
  - hyperknot 2y ago
    I think both solutions could easily saturate a 1 Gbps line. I benchmarked Btrfs + nginx and it could do 30 Gbps, which doesn't really make a difference if your server is 1 Gbps only. The fact that there is no service running was the more important for me. Mostly for security and bugs. I had so many problems with various tile servers in production, they needed daily restarting, they had memory leaks, etc. Basically I wanted to go nginx-only for security and to avoid tile server bugs.
    - bdon 2y ago
      I see, I think that's a good approach to enable serving with stock nginx as well as for companies that are built on Nginx or a plain HTTP serving stack already. For PMTiles the module is loadable directly as a Caddy plugin (https://docs.protomaps.com/deploy/server#caddyfile https://docs.protomaps.com/deploy/server#caddyfile) which I prefer to nginx for security and bugs (and automatic SSL), and also enables serving PMTiles from disk or a remote storage bucket without a separate service running.
      - hyperknot 2y ago
        Yes, PMTiles with the Caddy plugin is very similar to nginx + Btrfs. At that point, the difference between the two projects is mostly which schema is being used.
    - tecleandor 2y ago
      You both specify the filesystem to be Btrfs. Is there any advantage in this case against ZFS, ext4, XFS... or is it just a practical choice?
      - hyperknot 2y ago
        Yes, small files can fit in the metadata, which makes a super big difference when you have 300 million files of 405 bytes each. Also, inode handling is way better compared to ext4.

by bdon 2y ago

hyperknot 2y ago

No, the ping is 150 ms to us-west-2, and the tiles load in like 5 seconds on a cold start. Of course we cannot test cold start on HN comments because HN is the definition of hot :-) I can imagine workers to be fast, it's the range requests which are super slow. It's also outside of your control, it depends on how Cloudflare and S3 handles range requests to 90 GB files. I think if you could make PMTiles split into files <10 MB, it'd be perfect with range requests.

bdon 2y ago
I agree, there are tradeoffs to using static storage - the intended audience for PMTiles is those that prefer using static sites instead of administering a Linux server. I would be interested to see a comparison of Btrfs + nginx serving latency, vs `pmtiles serve` from https://github.com/protomaps/go-pmtiles https://github.com/protomaps/go-pmtiles on a PMTiles archive on disk. That would be a more direct comparison. I think there's potentially some interesting use case for tiles in Btrfs volumes and incremental updates, which I haven't tackled in PMTiles yet!
- hyperknot 2y ago
  I think both solutions could easily saturate a 1 Gbps line. I benchmarked Btrfs + nginx and it could do 30 Gbps, which doesn't really make a difference if your server is 1 Gbps only. The fact that there is no service running was the more important for me. Mostly for security and bugs. I had so many problems with various tile servers in production, they needed daily restarting, they had memory leaks, etc. Basically I wanted to go nginx-only for security and to avoid tile server bugs.
  - bdon 2y ago
    I see, I think that's a good approach to enable serving with stock nginx as well as for companies that are built on Nginx or a plain HTTP serving stack already. For PMTiles the module is loadable directly as a Caddy plugin (https://docs.protomaps.com/deploy/server#caddyfile https://docs.protomaps.com/deploy/server#caddyfile) which I prefer to nginx for security and bugs (and automatic SSL), and also enables serving PMTiles from disk or a remote storage bucket without a separate service running.
    - hyperknot 2y ago
      Yes, PMTiles with the Caddy plugin is very similar to nginx + Btrfs. At that point, the difference between the two projects is mostly which schema is being used.
  - tecleandor 2y ago
    You both specify the filesystem to be Btrfs. Is there any advantage in this case against ZFS, ext4, XFS... or is it just a practical choice?
    - hyperknot 2y ago
      Yes, small files can fit in the metadata, which makes a super big difference when you have 300 million files of 405 bytes each. Also, inode handling is way better compared to ext4.