4 ms·

The term "serving" is a bit misleading here. Most of the time, vector tile servers are serving pre generated tiles, which is extremely fast. This analysis is ab

The term "serving" is a bit misleading here. Most of the time, vector tile servers are serving pre generated tiles, which is extremely fast. This analysis is about generating tiles on the fly from PostGIS through a custom web server.

vlovich123 2y ago
Yup but super impressive just how much faster Martin was than all the other competition by significant margins with Bbox (Rust) and Tegola (Go) trailing at ~2-4x slower. That indicates the author(s) of Martin really optimized the data structures & algorithms to achieve a new Pareto frontier. Neat - would be nice if there were an accessible summary of the tricks employed to make it so fast that were missing in competitors.
- stevage 2y ago
  Yeah, that's definitely interesting - I'm surprised there is so much room for variation considering PostGIS is (if I'm not mistaken) doing most of the work. I couldn't find any description of what test 1, 2, 3 etc actually are though.
- GrayShade 2y ago
  Martin has an in-memory tile cache, which probably makes a difference: https://github.com/maplibre/martin/pull/1105 https://github.com/maplibre/martin/pull/1105. BBOX caches to a file instead.
  - darksaints 2y ago
    The benchmarking repository has config files used for the test, and they did not use the tile cache feature.
    - vlovich123 2y ago
      It defaults to 512MiB if not configured explicitly to 0 which it’s not in the repository.
      - darksaints 2y ago
        Oh that’s interesting, and I’m actually kinda peeved by any database-connected system that caches responses by default. Caching should be reserved as a performance optimization with serious correctness tradeoffs. Regardless, I don’t think caching came into play here, at least according to how I’m reading this repository. I would expect cold caches for everything.
- darksaints 2y ago
  The trick that makes Martin so fast is not doing any geospatial processing, and just being focused on making quick, non-blocking requests to Postgres. All geospatial processing is done by PostGIS, which is essentially just using the C++ geos library (which is by far the most comprehensive and well optimized geospatial processing library).
n4r9 2y ago
Not sure I agree; it sounds like the vector tiles are generated in advance of testing the servers. This description is from the linked GitHub: > six open-source vector tiles servers (BBOX, Ldproxy, Martin, pg_tileserv, Tegola, and TiPg) are set up and configured using Docker in a public cloud. Vector tiles are created for each server from the vector data of the PostGIS database. Various test scenarios with Apache JMeter are used to determine which server can deliver the vector tiles the fastest.
- GrayShade 2y ago
  They all seem to connect to PostGIS: https://github.com/FabianRechsteiner/vector-tiles-benchmark/blob/master/compose.yaml https://github.com/FabianRechsteiner/vector-tiles-benchmark/...
  - n4r9 2y ago
    Yes, true. I had the impression that the tiles themselves were being stored as geometric data in the postgres DB, then fetched and served. But I might have been confused by the article starting "Once you have created your vector tiles...". The GitHub page is a little ambiguous tbh.
    - stevage 2y ago
      >The GitHub page is a little ambiguous tbh. Agreed.

vlovich123 2y ago

Yup but super impressive just how much faster Martin was than all the other competition by significant margins with Bbox (Rust) and Tegola (Go) trailing at ~2-4x slower. That indicates the author(s) of Martin really optimized the data structures & algorithms to achieve a new Pareto frontier. Neat - would be nice if there were an accessible summary of the tricks employed to make it so fast that were missing in competitors.

stevage 2y ago
Yeah, that's definitely interesting - I'm surprised there is so much room for variation considering PostGIS is (if I'm not mistaken) doing most of the work. I couldn't find any description of what test 1, 2, 3 etc actually are though.
GrayShade 2y ago
Martin has an in-memory tile cache, which probably makes a difference: https://github.com/maplibre/martin/pull/1105 https://github.com/maplibre/martin/pull/1105. BBOX caches to a file instead.
- darksaints 2y ago
  The benchmarking repository has config files used for the test, and they did not use the tile cache feature.
  - vlovich123 2y ago
    It defaults to 512MiB if not configured explicitly to 0 which it’s not in the repository.
    - darksaints 2y ago
      Oh that’s interesting, and I’m actually kinda peeved by any database-connected system that caches responses by default. Caching should be reserved as a performance optimization with serious correctness tradeoffs. Regardless, I don’t think caching came into play here, at least according to how I’m reading this repository. I would expect cold caches for everything.
darksaints 2y ago
The trick that makes Martin so fast is not doing any geospatial processing, and just being focused on making quick, non-blocking requests to Postgres. All geospatial processing is done by PostGIS, which is essentially just using the C++ geos library (which is by far the most comprehensive and well optimized geospatial processing library).

n4r9 2y ago

Not sure I agree; it sounds like the vector tiles are generated in advance of testing the servers. This description is from the linked GitHub: > six open-source vector tiles servers (BBOX, Ldproxy, Martin, pg_tileserv, Tegola, and TiPg) are set up and configured using Docker in a public cloud. Vector tiles are created for each server from the vector data of the PostGIS database. Various test scenarios with Apache JMeter are used to determine which server can deliver the vector tiles the fastest.

GrayShade 2y ago
They all seem to connect to PostGIS: https://github.com/FabianRechsteiner/vector-tiles-benchmark/blob/master/compose.yaml https://github.com/FabianRechsteiner/vector-tiles-benchmark/...
- n4r9 2y ago
  Yes, true. I had the impression that the tiles themselves were being stored as geometric data in the postgres DB, then fetched and served. But I might have been confused by the article starting "Once you have created your vector tiles...". The GitHub page is a little ambiguous tbh.
  - stevage 2y ago
    >The GitHub page is a little ambiguous tbh. Agreed.