3 ms·

Evolution of the OSM Data Model [pdf]

62 points by zigzag312 4y ago

deleted 4y ago
[deleted]
zigzag312 4y ago
Some proposed changes to the OSM file format / data model, are very much needed IMO. For discussion about these changes: https://github.com/osmlab/osm-data-model https://github.com/osmlab/osm-data-model
- mistrial9 4y ago
  last change to that repo are four years ago ? where are changes actually done for real working servers now?
  - zigzag312 4y ago
    Link to this study has been posted in a comment here: https://github.com/osmlab/osm-data-model/issues/7#issuecomment-1216316563 https://github.com/osmlab/osm-data-model/issues/7#issuecomme... This is only a study of what changes are needed, but nothing has been implemented yet. Now is a chance to discuss these proposals, before moving to the next phase.
  - maxerickson 4y ago
    https://github.com/openstreetmap/openstreetmap-website https://github.com/openstreetmap/openstreetmap-website is the website and database. In practice, a lot of the database code is in an accelerator https://github.com/openstreetmap/cgimap https://github.com/openstreetmap/cgimap . Then there is also a bunch of different code bases that consume the data in a format that is dumped out of the main database (but that format is more or less aligned with the database storage). Section 6.1, Core Software, in the pdf, lists some of the more important pieces of it.
dhx 4y ago
In section 3.8 Limited Coordinate Precision: The OSM data model uses coordinates with a precision of 7 digits after the decimal point. This allows coordinates to be stored in 32 bit integers, so 8 bytes are enough for a coordinate pair. Compared to the common IEEE double precision real numbers this needs only half the space which allows for significant savings in large databases. The 7 digits give a precision of 1 centimeter of better, so in practice this is usually not a problem with the kinds of things OSM is mapping. But with more and more micro-mapping in OSM (for instance for indoor maps) this could in the future become more of a problem. I'm failing to see much benefit from increasing the precision of a point data type which currently allows ~11mm[1] precision when there are bigger untreated problems of both precision and accuracy to deal with: * Skyscrapers can have lateral deflections exceeding 1500mm * Residential building slabs and timber frames can seasonally deflect by over 50mm * Australia is moving ~70mm per year * Jakarta is sinking by ~100-250mm per year * Sand dunes can migrate by 95000mm per year To improve quality of OpenStreetMap data, wouldn't it be more beneficial to first implement support for time-dependent transformation vectors on grids fine enough to distinguish Jakarta from North Jakarta (sinking rates of 100mm vs 250mm respectively) and adding a timestamp to each point data type? In this way, it is possible to automatically transform vector data added to OSM a year ago into more accurate vector data applicable today (based on estimated movement). And would <11mm precision ever be a reasonable possibility for OSM users? If higher precision was needed for mapping interiors of buildings, how would a system handle the width of a room that could reasonably be measured to be 10000mm or 10050mm depending on deflection of the building structure on the day? Or a bridge deck that could differ in elevation above ground by 1000mm depending on load and weather? Or the 100th floor of a skyscraper that has a lateral deflection of 1500mm versus the 10th floor of the building that only has a lateral deflection of 100mm? [1] https://en.wikipedia.org/wiki/Decimal_degrees#Precision https://en.wikipedia.org/wiki/Decimal_degrees#Precision
- zigzag312 4y ago
  > And would <11mm precision ever be a reasonable possibility for OSM users? If higher precision was needed for mapping interiors of buildings, how would a system handle the width of a room that could reasonably be measured to be 10000mm or 10050mm depending on deflection of the building structure on the day? Or a bridge deck that could differ in elevation above ground by 1000mm depending on load and weather? Or the 100th floor of a skyscraper that has a lateral deflection of 1500mm versus the 10th floor of the building that only has a lateral deflection of 100mm? I'm not associated with OSM project in any way, but one solution for micro-mapping would be to position a micro-map relative to 32 bit coordinates. This way size of map would remain smaller and micro map would have needed precision.

62 points by zigzag312 4y ago

deleted 4y ago

[deleted]

Some proposed changes to the OSM file format / data model, are very much needed IMO. For discussion about these changes: https://github.com/osmlab/osm-data-model https://github.com/osmlab/osm-data-model

mistrial9 4y ago
last change to that repo are four years ago ? where are changes actually done for real working servers now?
- zigzag312 4y ago
  Link to this study has been posted in a comment here: https://github.com/osmlab/osm-data-model/issues/7#issuecomment-1216316563 https://github.com/osmlab/osm-data-model/issues/7#issuecomme... This is only a study of what changes are needed, but nothing has been implemented yet. Now is a chance to discuss these proposals, before moving to the next phase.
- maxerickson 4y ago
  https://github.com/openstreetmap/openstreetmap-website https://github.com/openstreetmap/openstreetmap-website is the website and database. In practice, a lot of the database code is in an accelerator https://github.com/openstreetmap/cgimap https://github.com/openstreetmap/cgimap . Then there is also a bunch of different code bases that consume the data in a format that is dumped out of the main database (but that format is more or less aligned with the database storage). Section 6.1, Core Software, in the pdf, lists some of the more important pieces of it.

dhx 4y ago

In section 3.8 Limited Coordinate Precision: The OSM data model uses coordinates with a precision of 7 digits after the decimal point. This allows coordinates to be stored in 32 bit integers, so 8 bytes are enough for a coordinate pair. Compared to the common IEEE double precision real numbers this needs only half the space which allows for significant savings in large databases. The 7 digits give a precision of 1 centimeter of better, so in practice this is usually not a problem with the kinds of things OSM is mapping. But with more and more micro-mapping in OSM (for instance for indoor maps) this could in the future become more of a problem. I'm failing to see much benefit from increasing the precision of a point data type which currently allows ~11mm[1] precision when there are bigger untreated problems of both precision and accuracy to deal with: * Skyscrapers can have lateral deflections exceeding 1500mm * Residential building slabs and timber frames can seasonally deflect by over 50mm * Australia is moving ~70mm per year * Jakarta is sinking by ~100-250mm per year * Sand dunes can migrate by 95000mm per year To improve quality of OpenStreetMap data, wouldn't it be more beneficial to first implement support for time-dependent transformation vectors on grids fine enough to distinguish Jakarta from North Jakarta (sinking rates of 100mm vs 250mm respectively) and adding a timestamp to each point data type? In this way, it is possible to automatically transform vector data added to OSM a year ago into more accurate vector data applicable today (based on estimated movement). And would <11mm precision ever be a reasonable possibility for OSM users? If higher precision was needed for mapping interiors of buildings, how would a system handle the width of a room that could reasonably be measured to be 10000mm or 10050mm depending on deflection of the building structure on the day? Or a bridge deck that could differ in elevation above ground by 1000mm depending on load and weather? Or the 100th floor of a skyscraper that has a lateral deflection of 1500mm versus the 10th floor of the building that only has a lateral deflection of 100mm? [1] https://en.wikipedia.org/wiki/Decimal_degrees#Precision https://en.wikipedia.org/wiki/Decimal_degrees#Precision

zigzag312 4y ago
> And would <11mm precision ever be a reasonable possibility for OSM users? If higher precision was needed for mapping interiors of buildings, how would a system handle the width of a room that could reasonably be measured to be 10000mm or 10050mm depending on deflection of the building structure on the day? Or a bridge deck that could differ in elevation above ground by 1000mm depending on load and weather? Or the 100th floor of a skyscraper that has a lateral deflection of 1500mm versus the 10th floor of the building that only has a lateral deflection of 100mm? I'm not associated with OSM project in any way, but one solution for micro-mapping would be to position a micro-map relative to 32 bit coordinates. This way size of map would remain smaller and micro map would have needed precision.