6 ms·

How to set up an OpenStreetMap server

150 points by ashitlerferad 10y ago

spapas82 10y ago
Nice! Some questions: How much time is needed for the import ? How much disk space should the server have ? (120 GB as in the title?) What do you mean by the 1700 in the phrase "1700 is the GB of RAM" ? Thanks !
- westi 10y ago
  > What do you mean by the 1700 in the phrase "1700 is the GB of RAM" It looks like the 1700 is the Cache size for the command to use in MiB of ram - https://github.com/openstreetmap/osm2pgsql#usage https://github.com/openstreetmap/osm2pgsql#usage <cache size> is about 75% of memory in MiB, to a maximum of about 30000. Additional RAM will not be used.
- antome 10y ago
  OSM wiki says "XML variant over 666GB uncompressed, 49.1GB bz2 compressed and 31.1GB PBF at 2016-03-20". Not sure where the 120GB comes from.
  - spapas82 10y ago
    Maybe because he drops some info from the file (he keeps only the roads) ?
  - ungzd 10y ago
    He imports only Europe from geofabrik.de dumps, not whole planet.
  - brianshaler 10y ago
    A sizable chunk of that is probably database indexes.
ungzd 10y ago
But it's only Europe, without being able to update in real-time and only for serving bitmap tiles. Not much fun (although still useful for lots of use cases).
mynewtb 10y ago
a) This is only Europe b) This is only roads OSM is global and covers much much more than just roads. As it is this is a clickbait title. If you want more of its data displayed in client maps, check out http://www.osm2vectortiles.org http://www.osm2vectortiles.org which is preprocessed data at 50G global. This also has many omissions and as far as I know a limit to detail in terms of your maximum zoom but if roads is all you need this would probably be a much better choice.
- Tergmap 10y ago
  There is so much you can put in a title. Calling it as "clickbait" is too much. They are simple instructions to get the maps of a continent online.
  - privong 10y ago
    > There is so much you can put in a title. "Your OpenStreetMap server in 120GB" is only 35 characters, out of 80 allowed for the title, so there's plenty of room to be more specific. "Your OpenStreetMap server of Roads in Europe in 120GB" is still only 56/80 and is much more accurate.
    - tokenizerrr 10y ago
      HN wants us to use the title of the source. This is the title of the source, and it's not even clickbait.
      - privong 10y ago
        > HN wants us to use the title of the source. This is the title of the source, and it's not even clickbait. I never said it was clickbait, that was the GP comment. And if you're going to refer to the guidelines: "Otherwise please use the original title, unless it is misleading or linkbait." One could make a reasonable case that the title of the source is misleading because it is incomplete, so the additional specificity may be warranted.
madez 10y ago
I remember having asked once in the osm irc channel on how to setup a private mirror including web service of osm and nobody could answer, telling me that's the first request of that type. It's good to see that there are advances into that direction.
- maxerickson 10y ago
  This a tile rendering stack. Similar documentation has been here for years: https://switch2osm.org/serving-tiles/building-a-tile-server-from-packages/ https://switch2osm.org/serving-tiles/building-a-tile-server-... There certainly isn't much documentation about setting up a private editing API, but there is quite a bit of information published in this repo: https://git.openstreetmap.org/chef.git/ https://git.openstreetmap.org/chef.git/
- mynewtb 10y ago
  Huh? https://switch2osm.org https://switch2osm.org is many years old and provides exactly that.
  - madez 10y ago
    I was not referring to using osm. On the contrary, I wanted to be able to provide a working one-to-one copy of the service available at osm.org including all files, databasis and services. The reason was to make myself more independent of web services.
    - maxerickson 10y ago
      The switch2osm guides cover what I would call the interesting side of this, primarily the tile server and geocoder. A lot of the other stuff actually depends on external services (the navigation, some of the queries). If you aren't the nexus of worldwide OpenStreetMap editing (which parts of osm.org are that), there isn't so much reason to run a whole bunch of the stuff. You don't need to support thousands of people making edits, if you aren't capturing edits you don't need a system to replicate them out to data consumers. Etc, etc.
    - 10y ago
petepete 10y ago
I struggled with OSRM on CentOS 7 this morning. So easy on Debian but stxxl and boost woes aplenty.
- virtualwhys 10y ago
  Likewise, though when I tried a couple of weeks ago my first attempt was on a CentOS 6 VM; now that was painful, little in the way of distro supplied packages, having to workaround outdated dependencies, etc. CentOS 7 in comparison was much easier, and then, the easiest: upgrading my laptop to Fedora 24 -- tile server basically comes for free, sigh ;-)
ris 10y ago
Note that there are many different things that people tend to use the term "OpenStreetMap Server" for. What is being described here is what's known as a _tileserver_.
outside1234 10y ago
Sort of off topic but is there a walk through like this for installing and running a server that serves up points of interest within a bounding box? More generally - I find the OSM community super fragmented - am i missing a well organized inventory of the different open source efforts?
- maxerickson 10y ago
  It can be stale or contain weird opinions, but the OSM wiki certainly serves as an adequate inventory (maybe not perfectly organized). http://wiki.openstreetmap.org/wiki/Software http://wiki.openstreetmap.org/wiki/Software To your specific question: https://github.com/twain47/Nominatim#installation https://github.com/twain47/Nominatim#installation with a viewbox: http://wiki.openstreetmap.org/wiki/Nominatim#Search http://wiki.openstreetmap.org/wiki/Nominatim#Search

150 points by ashitlerferad 10y ago

Nice! Some questions: How much time is needed for the import ? How much disk space should the server have ? (120 GB as in the title?) What do you mean by the 1700 in the phrase "1700 is the GB of RAM" ? Thanks !

westi 10y ago
> What do you mean by the 1700 in the phrase "1700 is the GB of RAM" It looks like the 1700 is the Cache size for the command to use in MiB of ram - https://github.com/openstreetmap/osm2pgsql#usage https://github.com/openstreetmap/osm2pgsql#usage <cache size> is about 75% of memory in MiB, to a maximum of about 30000. Additional RAM will not be used.
antome 10y ago
OSM wiki says "XML variant over 666GB uncompressed, 49.1GB bz2 compressed and 31.1GB PBF at 2016-03-20". Not sure where the 120GB comes from.
- spapas82 10y ago
  Maybe because he drops some info from the file (he keeps only the roads) ?
- ungzd 10y ago
  He imports only Europe from geofabrik.de dumps, not whole planet.
- brianshaler 10y ago
  A sizable chunk of that is probably database indexes.

ungzd 10y ago

But it's only Europe, without being able to update in real-time and only for serving bitmap tiles. Not much fun (although still useful for lots of use cases).

mynewtb 10y ago

a) This is only Europe b) This is only roads OSM is global and covers much much more than just roads. As it is this is a clickbait title. If you want more of its data displayed in client maps, check out http://www.osm2vectortiles.org http://www.osm2vectortiles.org which is preprocessed data at 50G global. This also has many omissions and as far as I know a limit to detail in terms of your maximum zoom but if roads is all you need this would probably be a much better choice.

Tergmap 10y ago
There is so much you can put in a title. Calling it as "clickbait" is too much. They are simple instructions to get the maps of a continent online.
- privong 10y ago
  > There is so much you can put in a title. "Your OpenStreetMap server in 120GB" is only 35 characters, out of 80 allowed for the title, so there's plenty of room to be more specific. "Your OpenStreetMap server of Roads in Europe in 120GB" is still only 56/80 and is much more accurate.
  - tokenizerrr 10y ago
    HN wants us to use the title of the source. This is the title of the source, and it's not even clickbait.
    - privong 10y ago
      > HN wants us to use the title of the source. This is the title of the source, and it's not even clickbait. I never said it was clickbait, that was the GP comment. And if you're going to refer to the guidelines: "Otherwise please use the original title, unless it is misleading or linkbait." One could make a reasonable case that the title of the source is misleading because it is incomplete, so the additional specificity may be warranted.

madez 10y ago

I remember having asked once in the osm irc channel on how to setup a private mirror including web service of osm and nobody could answer, telling me that's the first request of that type. It's good to see that there are advances into that direction.

maxerickson 10y ago
This a tile rendering stack. Similar documentation has been here for years: https://switch2osm.org/serving-tiles/building-a-tile-server-from-packages/ https://switch2osm.org/serving-tiles/building-a-tile-server-... There certainly isn't much documentation about setting up a private editing API, but there is quite a bit of information published in this repo: https://git.openstreetmap.org/chef.git/ https://git.openstreetmap.org/chef.git/
mynewtb 10y ago
Huh? https://switch2osm.org https://switch2osm.org is many years old and provides exactly that.
- madez 10y ago
  I was not referring to using osm. On the contrary, I wanted to be able to provide a working one-to-one copy of the service available at osm.org including all files, databasis and services. The reason was to make myself more independent of web services.
  - maxerickson 10y ago
    The switch2osm guides cover what I would call the interesting side of this, primarily the tile server and geocoder. A lot of the other stuff actually depends on external services (the navigation, some of the queries). If you aren't the nexus of worldwide OpenStreetMap editing (which parts of osm.org are that), there isn't so much reason to run a whole bunch of the stuff. You don't need to support thousands of people making edits, if you aren't capturing edits you don't need a system to replicate them out to data consumers. Etc, etc.
  - 10y ago

petepete 10y ago

I struggled with OSRM on CentOS 7 this morning. So easy on Debian but stxxl and boost woes aplenty.

virtualwhys 10y ago
Likewise, though when I tried a couple of weeks ago my first attempt was on a CentOS 6 VM; now that was painful, little in the way of distro supplied packages, having to workaround outdated dependencies, etc. CentOS 7 in comparison was much easier, and then, the easiest: upgrading my laptop to Fedora 24 -- tile server basically comes for free, sigh ;-)

ris 10y ago

Note that there are many different things that people tend to use the term "OpenStreetMap Server" for. What is being described here is what's known as a _tileserver_.

outside1234 10y ago

Sort of off topic but is there a walk through like this for installing and running a server that serves up points of interest within a bounding box? More generally - I find the OSM community super fragmented - am i missing a well organized inventory of the different open source efforts?

maxerickson 10y ago
It can be stale or contain weird opinions, but the OSM wiki certainly serves as an adequate inventory (maybe not perfectly organized). http://wiki.openstreetmap.org/wiki/Software http://wiki.openstreetmap.org/wiki/Software To your specific question: https://github.com/twain47/Nominatim#installation https://github.com/twain47/Nominatim#installation with a viewbox: http://wiki.openstreetmap.org/wiki/Nominatim#Search http://wiki.openstreetmap.org/wiki/Nominatim#Search