34 ms·

Getting to the bottom of web map performance

104 points by hampelm 5y ago

have_faith 5y ago
Interesting article. It's been a few years since I've had a play with this kind of stuff; what's the best option available for someone to host custom styled vector maps and display them in a performant way? preferably on an open source stack. I haven't used MapBox before as I was worried about costs and being locked in. I'm interested in experimenting with map interaction/UX.
- n8cpdx 5y ago
  Both ArcGIS Platform and Mapbox make this quite easy. Not fully open, but you get what you pay for. ArcGIS at least makes data import and export easy, and comes with a generous free tier
- Rebelgecko 5y ago
  There's some forks of pre-closed source Mapbox
  - TOMDM 5y ago
    One of these is called MapLibre for the curious. https://maplibre.org/ https://maplibre.org/ https://github.com/maplibre/maplibre-gl-js https://github.com/maplibre/maplibre-gl-js I've used it on a smaller project, it worked well for me.
- deleted 5y ago
  [deleted]
- bezossucks 5y ago
  Leaflet for simpler use cases or OpenLayers for more power Recent versions of both are semi-fast, enough for general use
  - KronisLV 5y ago
    Correct me if i'm wrong, but isn't OpenLayers (https://openlayers.org/ https://openlayers.org/) mostly just a client side library for the displaying of maps, much like Leaflet (https://leafletjs.com/ https://leafletjs.com/)? To have your own tile server, you'd probably want something like OpenMapTiles (https://openmaptiles.org/ https://openmaptiles.org/) or another alternative like Tilemaker (https://tilemaker.org/ https://tilemaker.org/).
    - VectorVoxels 5y ago
      You're not wrong. But not all maps are tile based. A layer can come from a tile server, yes, or local cached data served with the web page, or geojson, or bitmaps, or algorithmic shapes, or geo databases, etc. OL is much more than a rendering library, it understands different geospatial input formats (raster and vector, tiled or not) and dynamically reprojects them to the same map projection, and lets you overlay UI elements (points of interest, polygons, lines, animations, etc.) or blend layers together (backgrounds, base maps, hill shades, etc.) Check out the examples: https://openlayers.org/en/latest/examples/ https://openlayers.org/en/latest/examples/
      - KronisLV 5y ago
        That much is definitely true and it's indeed interesting to see the many approaches to maps! Now, personally, I'd still go for using a tile server in most cases in apps where the map functionality isn't the main part of the app, because of how simple that approach is, as well as how friendly to battery life those maps can be. Of course, if maps are front and center to your app, then things probably change somewhat! Though i do believe that OP was also particularly asking about how to host their own solution, presumably on a VPS or another server somewhere. In that regard, it's interesting to see the options out there, in part because there didn't used to be that many - sure, PostGIS is perfect for processing and storing geospatial data, yet when it comes to the display of maps, that niche of the industry is a little bit more new!
        VectorVoxels 5y ago
- hampelm 5y ago
  The author has been been building an open-source map rendering stack -- here's an intro blog post on that from April this year: https://protomaps.com/blog/new-way-to-make-maps https://protomaps.com/blog/new-way-to-make-maps
- pininja 5y ago
  deck.gl is another open source rendering option with a TileLayer, TerrainLayer, and MVTLayer. Other libraries mentioned have better text label and styling support out of the box compared to deck, so typically people do interleaved WebGL rendering with deck.gl and other basemap libraries to get a beautiful base and a super performant deck overlay. Tile hosting is still typically a paid service from someone, though COGS and S3 are a self-hosting option. I primarily work on libraries adjacent to deck.gl, happy to answer questions.
BbzzbB 5y ago
Offtopic, but while I may have nerdy GISers gathering here I'll shoot my shot. Someone knows what is a common/standard way of displaying vector information (points in this case) without having that information scrape-able? Is that even possible? I have a project where I want to sell some georeferenced data and be able to show said data (without all attributes, but showing all points) to potential customers in a map, but IIRC I've never failed to scrape vector web map data so I'm not sure it's even possible. I could imprint them in the tile rasters but then they wouldn't be interactable. Thanks (if offtopic ain't permitted on HN just delete this, sorry) Edit: So many insightful answers, thank you so much for the pointers HN, love y'all... and sorry OP for piggybacking.
- moritonal 5y ago
  Google Maps has the same issue. Their trick is to pass a ton of the data as protobuf, then decrypt it in WASM and load it into WebGL to render and interact with. Whilst all a massive pain, you can still scrape it with raw obj dumps from the GPU. So it's always a & game.
  - BbzzbB 5y ago
    Interesting solution, thanks for sharing.
- 3np 5y ago
  I’d just choose some small subset of your dataset as a sample dataset, expose just that, and accept that some users will scrape and make use of that. If this really isn’t satisfactory, the best you can do is non-interactive video, I think. For your actual paying customers, you control it via contract rather than DRM. (Worked with a data business, not specifically GIS)
  - BbzzbB 5y ago
    Thanks for your input! I'll be trying to implement hardly-scrapeable data as a learning opportunity (as I'm working towards coding + GIS career), but settling for a subset of data + demonstrative video would be a satisfactory compromise if the former turns out to be unachievable or overkill.
lmeyerov 5y ago
Rendering 100K-10M lines quickly is interesting! Now that WebGL2 is (almost) universal, any sense of which techniques make more sense now? Still no geometry shaders afaict, but maybe there is something else in there. We currently tessellate ahead of time - even streaming in from GPUs on our server - but that's not how we'd do it with say raw OpenGL. Maybe there's something else close enough now? (If you're into that kind of thing, plz email build@graphistry :D )

104 points by hampelm 5y ago

Interesting article. It's been a few years since I've had a play with this kind of stuff; what's the best option available for someone to host custom styled vector maps and display them in a performant way? preferably on an open source stack. I haven't used MapBox before as I was worried about costs and being locked in. I'm interested in experimenting with map interaction/UX.

n8cpdx 5y ago
Both ArcGIS Platform and Mapbox make this quite easy. Not fully open, but you get what you pay for. ArcGIS at least makes data import and export easy, and comes with a generous free tier
Rebelgecko 5y ago
There's some forks of pre-closed source Mapbox
- TOMDM 5y ago
  One of these is called MapLibre for the curious. https://maplibre.org/ https://maplibre.org/ https://github.com/maplibre/maplibre-gl-js https://github.com/maplibre/maplibre-gl-js I've used it on a smaller project, it worked well for me.
deleted 5y ago
[deleted]
bezossucks 5y ago
Leaflet for simpler use cases or OpenLayers for more power Recent versions of both are semi-fast, enough for general use
- KronisLV 5y ago
  Correct me if i'm wrong, but isn't OpenLayers (https://openlayers.org/ https://openlayers.org/) mostly just a client side library for the displaying of maps, much like Leaflet (https://leafletjs.com/ https://leafletjs.com/)? To have your own tile server, you'd probably want something like OpenMapTiles (https://openmaptiles.org/ https://openmaptiles.org/) or another alternative like Tilemaker (https://tilemaker.org/ https://tilemaker.org/).
  - VectorVoxels 5y ago
    You're not wrong. But not all maps are tile based. A layer can come from a tile server, yes, or local cached data served with the web page, or geojson, or bitmaps, or algorithmic shapes, or geo databases, etc. OL is much more than a rendering library, it understands different geospatial input formats (raster and vector, tiled or not) and dynamically reprojects them to the same map projection, and lets you overlay UI elements (points of interest, polygons, lines, animations, etc.) or blend layers together (backgrounds, base maps, hill shades, etc.) Check out the examples: https://openlayers.org/en/latest/examples/ https://openlayers.org/en/latest/examples/
    - KronisLV 5y ago
      That much is definitely true and it's indeed interesting to see the many approaches to maps! Now, personally, I'd still go for using a tile server in most cases in apps where the map functionality isn't the main part of the app, because of how simple that approach is, as well as how friendly to battery life those maps can be. Of course, if maps are front and center to your app, then things probably change somewhat! Though i do believe that OP was also particularly asking about how to host their own solution, presumably on a VPS or another server somewhere. In that regard, it's interesting to see the options out there, in part because there didn't used to be that many - sure, PostGIS is perfect for processing and storing geospatial data, yet when it comes to the display of maps, that niche of the industry is a little bit more new!
      - VectorVoxels 5y ago
hampelm 5y ago
The author has been been building an open-source map rendering stack -- here's an intro blog post on that from April this year: https://protomaps.com/blog/new-way-to-make-maps https://protomaps.com/blog/new-way-to-make-maps
pininja 5y ago
deck.gl is another open source rendering option with a TileLayer, TerrainLayer, and MVTLayer. Other libraries mentioned have better text label and styling support out of the box compared to deck, so typically people do interleaved WebGL rendering with deck.gl and other basemap libraries to get a beautiful base and a super performant deck overlay. Tile hosting is still typically a paid service from someone, though COGS and S3 are a self-hosting option. I primarily work on libraries adjacent to deck.gl, happy to answer questions.

BbzzbB 5y ago

Offtopic, but while I may have nerdy GISers gathering here I'll shoot my shot. Someone knows what is a common/standard way of displaying vector information (points in this case) without having that information scrape-able? Is that even possible? I have a project where I want to sell some georeferenced data and be able to show said data (without all attributes, but showing all points) to potential customers in a map, but IIRC I've never failed to scrape vector web map data so I'm not sure it's even possible. I could imprint them in the tile rasters but then they wouldn't be interactable. Thanks (if offtopic ain't permitted on HN just delete this, sorry) Edit: So many insightful answers, thank you so much for the pointers HN, love y'all... and sorry OP for piggybacking.

moritonal 5y ago
Google Maps has the same issue. Their trick is to pass a ton of the data as protobuf, then decrypt it in WASM and load it into WebGL to render and interact with. Whilst all a massive pain, you can still scrape it with raw obj dumps from the GPU. So it's always a & game.
- BbzzbB 5y ago
  Interesting solution, thanks for sharing.
3np 5y ago
I’d just choose some small subset of your dataset as a sample dataset, expose just that, and accept that some users will scrape and make use of that. If this really isn’t satisfactory, the best you can do is non-interactive video, I think. For your actual paying customers, you control it via contract rather than DRM. (Worked with a data business, not specifically GIS)
- BbzzbB 5y ago
  Thanks for your input! I'll be trying to implement hardly-scrapeable data as a learning opportunity (as I'm working towards coding + GIS career), but settling for a subset of data + demonstrative video would be a satisfactory compromise if the former turns out to be unachievable or overkill.

lmeyerov 5y ago

Rendering 100K-10M lines quickly is interesting! Now that WebGL2 is (almost) universal, any sense of which techniques make more sense now? Still no geometry shaders afaict, but maybe there is something else in there. We currently tessellate ahead of time - even streaming in from GPUs on our server - but that's not how we'd do it with say raw OpenGL. Maybe there's something else close enough now? (If you're into that kind of thing, plz email build@graphistry :D )