4 ms·

I mean, I realize hosting a service is expensive, but can't there be an open source set of service APIs that's self hosted? That way you can chose to get a full

I mean, I realize hosting a service is expensive, but can't there be an open source set of service APIs that's self hosted? That way you can chose to get a full copy of the data (and be responsible for pulling updates) and the API, or just pay someone else for access to their hosted version of the API?

michaelt 9y ago
can't there be an open source set of service APIs that's self hosted OSM does have open source self-hosted options. Unfortunately, the developer experience is much more painful than using Google Maps. With Google Maps, if you can add Javascript to a web page you're basically good to go. Data updates, software updates, server redundancy, server support and security, all taken care of. Even if you're exclusively a front-end dev, you're making a static site, and you've never heard of Linux. Compared to that, self-hosting OSM tile generation requires a bunch of extra knowledge, equipment and maintenance.
- chuckdries 9y ago
  Is it something that could be containerized? Maybe not as simple as a JS embed, but deploying a self hosted docker container is far easier than installing most things I've encountered.
  - petre 9y ago
    With vector tiles one doesn't have to containerize anything - just shard them onto a bunch of servers and server them over https. Put an accelerator/cache in front and proxy to the desired region backend. With image tiles it's a hurdle. You have to build 18 zoom levels of tiles. The most detailed layers have an astronomic number if tiles. You also need a huge amount if storage. http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=5.538062,47.236312,15.371071,54.954 http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox... Did I mention the data being huge? What's the point of having 15Tb container, which takes forever to deploy? You can certainly containerize the services, but there's no benefit containerizing the data, since you have to replace it all at once rather than synchronizing the differences.
    - voltagex_ 9y ago
      How intensive is it to rasterise on demand?
      - petre 9y ago
        Depends on the degree of complexity of the tiles. We tried it (using tilecache or tilestache and mapproxy) and it becomes unresponsive when you rasterize several tiles (9). It can take several secons for a busy tile. If the user gets impacient like I do, zooms in and out and pans the map, it freezes to a halt. It's doable over clear water and maybe on the highway, but not in busy cities, at least not with hundreds of users requesting on demand tiles and without an insane amount of rendering resources. My impression is that it works if you're the only user and a really patient one at that.
        6487698hhhh 9y ago
        Why not use vector tiles? I can generate a vector tile package of the whole planet in Esri .vtpk format and it takes up around 30Gb. A map service can be published from it easily. Takes me about 2 days for initial translation and about 1 week ti generate the vtpk. Neither of these have been optimised for speed. I have an old Galaxy S3 upstairs running an app that displays this vtpk - so I can have the entire world down to house level on my phone.
lucideer 9y ago
> you can chose to get a full copy of the data (and be responsible for pulling updates) Even if you could make the general setup and maintenance incredibly easy somehow, just syncing the map data from osm regularly would require decent hardware: the extra cost of each dev doing this themselves just doesn't compare to using a central tile host. On top of that, if every dev was regularly updating their own OSM data, OSM would have a lot of extra strain on their data download servers to deal with. Not sure if this could be comparable to the hardware overhead of providing a tile hosting service though.
- petre 9y ago
  > just syncing the map data from osm regularly would require decent hardware We need servers with at least 24 CPU threads and 256 GB RAM and moving the data into PostGIS, region by region takes a week or two. Plus you manually have to fix the process when it just spits out an error and aborts after 15 hours of churning.
  - aw3c2 9y ago
    Have you tried imposm? Your process sounds magnitudes too slow.
    - 6487698hhhh 9y ago
      Agreed. I can load with osm2pgsql on my home PC (AMD FX 8370, 32gb ram, 1tb SSD) in a couple of days, and that's generating 78 output layers. imposm is supposed to be faster still.
- rmc 9y ago
  > just syncing the map data from osm regularly would require decent hardware You can get country level extracts, which can be updated every day. Nearly any server which can import OSM data is able to keep it up to date.
rmc 9y ago
Yes, you can do just that.

michaelt 9y ago

can't there be an open source set of service APIs that's self hosted OSM does have open source self-hosted options. Unfortunately, the developer experience is much more painful than using Google Maps. With Google Maps, if you can add Javascript to a web page you're basically good to go. Data updates, software updates, server redundancy, server support and security, all taken care of. Even if you're exclusively a front-end dev, you're making a static site, and you've never heard of Linux. Compared to that, self-hosting OSM tile generation requires a bunch of extra knowledge, equipment and maintenance.

chuckdries 9y ago
Is it something that could be containerized? Maybe not as simple as a JS embed, but deploying a self hosted docker container is far easier than installing most things I've encountered.
- petre 9y ago
  With vector tiles one doesn't have to containerize anything - just shard them onto a bunch of servers and server them over https. Put an accelerator/cache in front and proxy to the desired region backend. With image tiles it's a hurdle. You have to build 18 zoom levels of tiles. The most detailed layers have an astronomic number if tiles. You also need a huge amount if storage. http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=5.538062,47.236312,15.371071,54.954 http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox... Did I mention the data being huge? What's the point of having 15Tb container, which takes forever to deploy? You can certainly containerize the services, but there's no benefit containerizing the data, since you have to replace it all at once rather than synchronizing the differences.
  - voltagex_ 9y ago
    How intensive is it to rasterise on demand?
    - petre 9y ago
      Depends on the degree of complexity of the tiles. We tried it (using tilecache or tilestache and mapproxy) and it becomes unresponsive when you rasterize several tiles (9). It can take several secons for a busy tile. If the user gets impacient like I do, zooms in and out and pans the map, it freezes to a halt. It's doable over clear water and maybe on the highway, but not in busy cities, at least not with hundreds of users requesting on demand tiles and without an insane amount of rendering resources. My impression is that it works if you're the only user and a really patient one at that.
      - 6487698hhhh 9y ago
        Why not use vector tiles? I can generate a vector tile package of the whole planet in Esri .vtpk format and it takes up around 30Gb. A map service can be published from it easily. Takes me about 2 days for initial translation and about 1 week ti generate the vtpk. Neither of these have been optimised for speed. I have an old Galaxy S3 upstairs running an app that displays this vtpk - so I can have the entire world down to house level on my phone.

lucideer 9y ago

> you can chose to get a full copy of the data (and be responsible for pulling updates) Even if you could make the general setup and maintenance incredibly easy somehow, just syncing the map data from osm regularly would require decent hardware: the extra cost of each dev doing this themselves just doesn't compare to using a central tile host. On top of that, if every dev was regularly updating their own OSM data, OSM would have a lot of extra strain on their data download servers to deal with. Not sure if this could be comparable to the hardware overhead of providing a tile hosting service though.

petre 9y ago
> just syncing the map data from osm regularly would require decent hardware We need servers with at least 24 CPU threads and 256 GB RAM and moving the data into PostGIS, region by region takes a week or two. Plus you manually have to fix the process when it just spits out an error and aborts after 15 hours of churning.
- aw3c2 9y ago
  Have you tried imposm? Your process sounds magnitudes too slow.
  - 6487698hhhh 9y ago
    Agreed. I can load with osm2pgsql on my home PC (AMD FX 8370, 32gb ram, 1tb SSD) in a couple of days, and that's generating 78 output layers. imposm is supposed to be faster still.
rmc 9y ago
> just syncing the map data from osm regularly would require decent hardware You can get country level extracts, which can be updated every day. Nearly any server which can import OSM data is able to keep it up to date.

rmc 9y ago

Yes, you can do just that.