10 ms·

Supercomputers will start building a 3D map of the world

141 points by jonbaer 7y ago

dzdt 7y ago
This public effort is still way behind private efforts like google earth, it seems.
- black_puppydog 7y ago
  You say that like it's written in the article but I don't see it. And even if (!) the quality of the map were slightly below google maps, having it freely available would be a huge plus nevertheless.
- 7952 7y ago
  Most of Google Earth is just based on publically available data from Nasa. They have suplemented it in places with commercial data.
- FillardMillmore 7y ago
  Google Earth was not strictly a private effort. Back in the early 2000s, Keyhole (then the company that owned the software that would become Google Earth after the acquisition) was bankrolled by a venture capital firm run by the CIA called In-Q-Tel. I'm sure there are other relations Google Earth (in particular) has had with government as well.
- dagw 7y ago
  Google earth uses older publicly available NASA data as a base and then improves it with better, often commercial, data in some interesting points. One problem with this approach is that the quality and accuracy differs wildly from place to place. For any place in at least the western world there almost certainly exists better (and sometimes open) data than this project will provide. The big win for this project isn't that the data will always be the best available, but that it will be a single source of open data of consistent quality and format for the entire earth.
jcfrei 7y ago
A very valuable effort. Global mapping data should eventually enter the public domain.
twelvechairs 7y ago
There's already multiple global DEM data sources. It would be good to understand how this is better and what the planned accuracy is. Its a very light article [edit] Their referenced Arctic DEM gives a 2m resolution however on accuracy notes 'Without ground control points absolute accuracy is approximately 4 meters in horizontal and vertical planes'. This is much better than most global data (SRTM and ASTER both at 30m resolution) However it is not as high resolution as many existing free models for individual parts of the globe. As an example here in Australia I can get free 1m resolution DEMs of cities with accuracy noted at "0.3m (95% Confidence Interval) vertical and 0.8m (95% Confidence Interval) horizontal".
- samirm 7y ago
  >As an example here in Australia I can get free 1m resolution DEMs of cities source please?
  - twelvechairs 7y ago
    https://elevation.fsdf.org.au/ https://elevation.fsdf.org.au/
    - samirm 7y ago
      Thank you! :)
- joshvm 7y ago
  MERIT is probably the best (free) LIDAR-derived global DEM right now: http://hydro.iis.u-tokyo.ac.jp/~yamadai/MERIT_DEM/ http://hydro.iis.u-tokyo.ac.jp/~yamadai/MERIT_DEM/ Basically fixes the problems with SRTM, but you're still left with 30m resolution. You can read more about ArcticDEM here: https://www.pgc.umn.edu/data/arcticdem/ https://www.pgc.umn.edu/data/arcticdem/ Seems like EarthDEM is the same. So all the advantages and disadvantages that come with stereo photogrammetry. Very high resolution, but potential for inaccuracy and quite difficult to check. Probably they used SRTM or something similar to validate their models. >IceSAT altimetry data points are used to improve the vertical accuracy of both the DEM strips and mosaic files. IceSAT data points are filtered to exclude points in areas of high relief and over hydrographic features. Additional filtering is applied to remove altimetry points collected outside the temporal window of the source imagery acquisition date. An xyz translation is calculated for each strip and the offset is added to the metadata file. The individual DEM strips are not translated before distribution. Users can apply their own corrections to the strip if they do not agree with the one originally provided. Where available, additional control information such as LiDAR or surveyed GPS points have been applied.
gourou 7y ago
Is there currently a way to get the elevation of a point? I know Google Earth has one but it's very imprecise
- namirez 7y ago
  Here is one option: https://www.usgs.gov/faqs/where-can-i-get-elevation-data?qt-news_science_products=7#qt-news_science_products https://www.usgs.gov/faqs/where-can-i-get-elevation-data?qt-...
- jubjubbird 7y ago
  USGS Elevation point query service with an API: https://nationalmap.gov/epqs/ https://nationalmap.gov/epqs/
theseadroid 7y ago
I wish every game depicting the real world contribute to a common 3D world repository. And maybe allow crowd sourcing that repository too. Imagine what we would get to play in flight simulators or driving simulators with such a repository.
- big_chungus 7y ago
  And imagine when some internet forum decides to put a phallic object on top of some significant building, and all go vote it as legitimate. I'm not claiming it's impossible, but there will certainly be obstacles (see wikipedia). This has to be one data-set we get right; with self-driving cars coming, it could cause injury or death if we don't.
  - melling 7y ago
    Like you said: See Wikipedia.
    - big_chungus 7y ago
      Right, but Wikipedia still has lots of issues with vandalism. They do a good job, but it's just not accurate to keep something perfect at that scale. Here's the difference: wikipedia gets vandalized and people laugh. Mapping data gets vandalized and people could die.
      - melling 7y ago
        The world is a dynamic place. People and AI need to be able to reconcile when what they see doesn’t match a map.
      - inscionent 7y ago
        Humans are vandals. Humans are murders. Humans are creators. Humans build things larger than themselves. Nothing is perfect. Choose what to focus on.
  - tjoff 7y ago
deleted 7y ago
[deleted]
andyljones 7y ago
Someone in the know - why do you need a supercomputer for this? From an amateur position it seems suitable for commodity compute and a big ol map-reduce. Getting chunk edges to align might be tricky, but the usual solution to that is to stagger the chunks and throw away the edges.
- 0xffff2 7y ago
  There's not really a huge difference between supercomputers and a large number of ordinary computers these days. The computational hardware is basically the same. Only the communication interconnects are different. I strongly suspect that they don't need a supercomputer, but they had one available and it sounds cool. I sit one building over from the ~13th largest supercomputer in the world and I've definitely used it when I needed a bit more computing power than my laptop could provide. I used the supercomputer because I could, not because I needed to.
  - dekhn 7y ago
    Did you have an embarassingly parallel code (like, "map"), or a code that was ported to supercomputers? I've worked on supercomputers for years and generally, one could not run code at scale unless it scaled- in terms of parallel performance- and also used the network in a non-trivial way. Supercomputers aren't just speedups for normal codes (I'm pretty sure you know this). The real issue with modern supercomputers is that basically none of them have decent disk IO. That's what differs between modern Internet/cloud clusters and supercomputers. Cloud clusters emphasize very high connectivity between durable storage and worker nodes. None of the supercomputers have a decent disk IO stack (mostly GPFS and Lustre) and this ends up being no end of pain for applicaiton developers. The only recent improvement in this area was "burst buffers", but that's really just accelerated data staging.
- dimatura 7y ago
  Seems like they are using photogrammetry, most likely with a large amount of high resolution images. Photogrammetry under these conditions can be fairly computationally expensive, specially if they use aim for high quality reconstructions (which they probably should, given this is a one-time or few-time cost). Of course, it depends on what algorithms they're using, of which I have no idea, but there's plenty of fairly fancy methods, many involving deep learning in the pipeline.
dekhn 7y ago
Supercomputers are exactly the wrong sort of tool to use for this- nearly every supercomputer has crappy disk IO and tons of fast CPU and RAM and network. Using a supercomputer for this would leave the expensive elements like the GPUs nearly idle, and the IO subsystem would be the bottleneck.
- HenryKissinger 7y ago
  Yes, I'm sure you saw something the world class experts of the National Geo-spatial Intelligence Agency missed. /s
  - dekhn 7y ago
    I'm a supercomputer expert - that's a part of my job - who works on data processing fulltime. I know these codes, I know the hardware architecture, and my statement above is technically correct. I'm not missing anything.
    - HenryKissinger 7y ago
      You don't know shit.
      - dekhn 7y ago
        I direct you to my literature. https://scholar.google.com/citations?user=TFgipkIAAAAJ&hl=en&oi=sra https://scholar.google.com/citations?user=TFgipkIAAAAJ&hl=en... You may find especially interesting an article I published that used an embarassingly parallel computing system that I built which ran on Google's internal infrastructure (not a supercomputer) in response to my codes not running well on supercomputers. If you wish to have a substantive discussion about my statements (rather than flinging insults), I'd be happy to.
    - lostgame 7y ago
      Wow, HenryKissinger - you're an asshole. dekhn has provided proof of their claims - what does telling them they're full of shit without any backup of your claims add to the conversation but unverified negativity?
blackbrokkoli 7y ago
The article misses the point a bit. There are 3D maps of the world, or rather, you can make your own on a weekend. Topographic data is out there for most things, the problem is just it's very inconsistent and of course, resolution. And I guess for the whole world you do need a decent computer. To prove my point, here is some art I made [1] from open data (I think NASA) with Blender and QGIS, meaning free software stack on very much a normal computer. My model was the state Schleswig Holstein, a german county, but as said, you can find data for pretty much everything. The resolution is not astonishing, but enough to spot Germany's only high sea island, the "Wattenmeer" where tide causes some spots to lie below water level on average, the mouth of the Elbe and more cool stuff when you know where to look. My point isn't to downplay the effort of EarthDEM, I just want to make more people aware of what is already very much out there :) [1]: https://imgur.com/a/rZUTajX https://imgur.com/a/rZUTajX
- rurban 7y ago
  > And I guess for the whole world you do need a decent computer. Nope, resolution is the problem, not scale. You need low orbit stereo photos, like from an airplane. Without clouds. With satellite you only get 30m, with airplanes you get under 1m. Cities usually rent a plane a year to adjust their maps. Countries don't, they usually just rely on the cheap satellite photos.
pbhjpbhj 7y ago
The Arctic 3D map mentioned in OP, https://livingatlas2.arcgis.com/arcticdemexplorer/ https://livingatlas2.arcgis.com/arcticdemexplorer/ -- covers the whole area from 60deg N.
frankharv 7y ago
When I was in port in Gitmo in 1987 there was a civilian boat in harbor with a large tethered balloon. They were mapping Cuba for the US Defense Mapping Agency(DMA). So I would guess that DMA had a 3D terrain map around 1989. I know it was used for cruise missile flight planning. No idea what the resolution was. Even with our very detailed underwater mapping the Navy still has areas that are not well covered. https://www.popularmechanics.com/military/navy-ships/a24158/uss-san-francisco-mountain-incident/ https://www.popularmechanics.com/military/navy-ships/a24158/...
crazygringo 7y ago
Serious question: what on earth have supercomputers got to do with this? My impression is supercomputers exist mainly for incredibly intensive large-scale simulation/calculation that can't be subdivided into parts -- e.g. weather or nuclear explosion simulation. 3D map processing feels like the literal opposite of that -- trivially parallelizable, with a much higher ratio of data/IO to calculation. Are they running out of tasks for Blue Waters to do, or trying to find a high-profile project to justify it politically or something? I really can't imagine for the life of me why you wouldn't just run this in any enterprise-grade cloud.
- eternalny1 7y ago
  It seems like they are using IMAGES of the area and then using visual processing to determine the 3D height map. That would be intense visual processing, stitching together various angles of photos to work out a terrain elevation. They are not dealing with a 3D point cloud of elevation data.
  - HellDunkel 7y ago
    Sounds as if they plan to use agisoft photoscan to process all satellite images they can find.
- jobigoud 7y ago
  > calculation that can't be subdivided into parts -- e.g. weather I'm not sure how weather simulation work but I always wondered why they aren't performed cellular automaton style. Bottom up rather than top-down. At each time step a given cell state is computed based on the state of its neighbors at t-1. This should be parallelizable. I feel it's also how it works in the real world anyway.
  - petschge 7y ago
    That is exactly what weather simulations do. Same most other kinds of hydrodynamic simulations (simulation of gas or fluid flows). The thing is: you need the state at t-1 of all your neighbors. Then you can do a small timestep to get from t-1 to time t. And then you need the NEW state of your neighbors. That requires a fast interconnect. Which HPC machines have, unlike most clouds or commodity clusters. In other words, yes it is parallelizable, but not trivially so, because the different grid points are coupled.
maxemitchell 7y ago
I found a much better source regarding this in the form of a November 2018 white paper[1]. Interesting is that they say it will take 3.2-3.5 years of Blue Waters computing time. [1]: https://www.exascale.org/bdec/sites/www.exascale.org.bdec/files/whitepapers/Kramer_BDEC2__WP_v2.pdf https://www.exascale.org/bdec/sites/www.exascale.org.bdec/fi...
sorenjan 7y ago
It's a bit strange that they use DigitalGlobe imagery, since DigitalGlobe owns 50% of Vricon who use aerial and satellite photos to make 3D maps. Their tagline is "The globe in 3D". https://www.vricon.com/news/saab-digitalglobe-announce-vricon-joint-venture-create-globe-3d/ https://www.vricon.com/news/saab-digitalglobe-announce-vrico...
ungzd 7y ago
Article title is misleading. It's not "3D map", like city map with 3D buildings or something like that, it's just a DEM, digital elevation model. It's not a map at all. It's useful for addition information to build a map, but it's not a map.
mnutsch23 7y ago
This has already been a thing for at least a year. For example: Go to Google Maps, satellite view. Then zoom in and rotate with Ctrl + click & drag. https://www.google.com/maps/@32.7644941,-96.8061872,1158a,35y,11.78h,55.61t/data=!3m1!1e3 https://www.google.com/maps/@32.7644941,-96.8061872,1158a,35... Microsoft and Mapbox also have an experimental project where computer vision on cars maps the location of recognized objects. This is probably true of every company researching self-driving cars.
mceoin 7y ago
This is very exciting. One step closer to a manual for spaceship earth.
llsf 7y ago
I am wondering what resolution they work with. Because earth crust is moving continuously, and sometimes suddenly like 6 feet at once: https://www.latimes.com/california/story/2019-07-22/ridgecrest-earthquake-images-broken-ground https://www.latimes.com/california/story/2019-07-22/ridgecre... I like the idea to survey the whole planet, I would be curious to see what resolution they end up with, and how often it is updated.
mlwhiz 7y ago
Don't they already have the map of the world in 3D?
csteubs 7y ago
Another step in the right direction, but satellites will continue to be the bottleneck to unlocking the full value of maps.
mjparrott 7y ago
How is this different than Google Earth / Maps?
paparush 7y ago
Paging Hiro Protagonist
gao8a 7y ago
The silent cartographer
rurban 7y ago
Marketing: "With EarthDEM, we will take this transformative new capability to where most of us live, to cities and coastlines, that will directly impact their lives." Tech: We need these maps for our cruise missiles.

141 points by jonbaer 7y ago

dzdt 7y ago

This public effort is still way behind private efforts like google earth, it seems.

black_puppydog 7y ago
You say that like it's written in the article but I don't see it. And even if (!) the quality of the map were slightly below google maps, having it freely available would be a huge plus nevertheless.
7952 7y ago
Most of Google Earth is just based on publically available data from Nasa. They have suplemented it in places with commercial data.
FillardMillmore 7y ago
Google Earth was not strictly a private effort. Back in the early 2000s, Keyhole (then the company that owned the software that would become Google Earth after the acquisition) was bankrolled by a venture capital firm run by the CIA called In-Q-Tel. I'm sure there are other relations Google Earth (in particular) has had with government as well.
dagw 7y ago
Google earth uses older publicly available NASA data as a base and then improves it with better, often commercial, data in some interesting points. One problem with this approach is that the quality and accuracy differs wildly from place to place. For any place in at least the western world there almost certainly exists better (and sometimes open) data than this project will provide. The big win for this project isn't that the data will always be the best available, but that it will be a single source of open data of consistent quality and format for the entire earth.

jcfrei 7y ago

A very valuable effort. Global mapping data should eventually enter the public domain.

twelvechairs 7y ago

There's already multiple global DEM data sources. It would be good to understand how this is better and what the planned accuracy is. Its a very light article [edit] Their referenced Arctic DEM gives a 2m resolution however on accuracy notes 'Without ground control points absolute accuracy is approximately 4 meters in horizontal and vertical planes'. This is much better than most global data (SRTM and ASTER both at 30m resolution) However it is not as high resolution as many existing free models for individual parts of the globe. As an example here in Australia I can get free 1m resolution DEMs of cities with accuracy noted at "0.3m (95% Confidence Interval) vertical and 0.8m (95% Confidence Interval) horizontal".

samirm 7y ago
>As an example here in Australia I can get free 1m resolution DEMs of cities source please?
- twelvechairs 7y ago
  https://elevation.fsdf.org.au/ https://elevation.fsdf.org.au/
  - samirm 7y ago
    Thank you! :)
joshvm 7y ago
MERIT is probably the best (free) LIDAR-derived global DEM right now: http://hydro.iis.u-tokyo.ac.jp/~yamadai/MERIT_DEM/ http://hydro.iis.u-tokyo.ac.jp/~yamadai/MERIT_DEM/ Basically fixes the problems with SRTM, but you're still left with 30m resolution. You can read more about ArcticDEM here: https://www.pgc.umn.edu/data/arcticdem/ https://www.pgc.umn.edu/data/arcticdem/ Seems like EarthDEM is the same. So all the advantages and disadvantages that come with stereo photogrammetry. Very high resolution, but potential for inaccuracy and quite difficult to check. Probably they used SRTM or something similar to validate their models. >IceSAT altimetry data points are used to improve the vertical accuracy of both the DEM strips and mosaic files. IceSAT data points are filtered to exclude points in areas of high relief and over hydrographic features. Additional filtering is applied to remove altimetry points collected outside the temporal window of the source imagery acquisition date. An xyz translation is calculated for each strip and the offset is added to the metadata file. The individual DEM strips are not translated before distribution. Users can apply their own corrections to the strip if they do not agree with the one originally provided. Where available, additional control information such as LiDAR or surveyed GPS points have been applied.

gourou 7y ago

Is there currently a way to get the elevation of a point? I know Google Earth has one but it's very imprecise

namirez 7y ago
Here is one option: https://www.usgs.gov/faqs/where-can-i-get-elevation-data?qt-news_science_products=7#qt-news_science_products https://www.usgs.gov/faqs/where-can-i-get-elevation-data?qt-...
jubjubbird 7y ago
USGS Elevation point query service with an API: https://nationalmap.gov/epqs/ https://nationalmap.gov/epqs/

theseadroid 7y ago

I wish every game depicting the real world contribute to a common 3D world repository. And maybe allow crowd sourcing that repository too. Imagine what we would get to play in flight simulators or driving simulators with such a repository.

big_chungus 7y ago
And imagine when some internet forum decides to put a phallic object on top of some significant building, and all go vote it as legitimate. I'm not claiming it's impossible, but there will certainly be obstacles (see wikipedia). This has to be one data-set we get right; with self-driving cars coming, it could cause injury or death if we don't.
- melling 7y ago
  Like you said: See Wikipedia.
  - big_chungus 7y ago
    Right, but Wikipedia still has lots of issues with vandalism. They do a good job, but it's just not accurate to keep something perfect at that scale. Here's the difference: wikipedia gets vandalized and people laugh. Mapping data gets vandalized and people could die.
    - melling 7y ago
      The world is a dynamic place. People and AI need to be able to reconcile when what they see doesn’t match a map.
    - inscionent 7y ago
      Humans are vandals. Humans are murders. Humans are creators. Humans build things larger than themselves. Nothing is perfect. Choose what to focus on.
- tjoff 7y ago

deleted 7y ago

[deleted]

andyljones 7y ago

Someone in the know - why do you need a supercomputer for this? From an amateur position it seems suitable for commodity compute and a big ol map-reduce. Getting chunk edges to align might be tricky, but the usual solution to that is to stagger the chunks and throw away the edges.

0xffff2 7y ago
There's not really a huge difference between supercomputers and a large number of ordinary computers these days. The computational hardware is basically the same. Only the communication interconnects are different. I strongly suspect that they don't need a supercomputer, but they had one available and it sounds cool. I sit one building over from the ~13th largest supercomputer in the world and I've definitely used it when I needed a bit more computing power than my laptop could provide. I used the supercomputer because I could, not because I needed to.
- dekhn 7y ago
  Did you have an embarassingly parallel code (like, "map"), or a code that was ported to supercomputers? I've worked on supercomputers for years and generally, one could not run code at scale unless it scaled- in terms of parallel performance- and also used the network in a non-trivial way. Supercomputers aren't just speedups for normal codes (I'm pretty sure you know this). The real issue with modern supercomputers is that basically none of them have decent disk IO. That's what differs between modern Internet/cloud clusters and supercomputers. Cloud clusters emphasize very high connectivity between durable storage and worker nodes. None of the supercomputers have a decent disk IO stack (mostly GPFS and Lustre) and this ends up being no end of pain for applicaiton developers. The only recent improvement in this area was "burst buffers", but that's really just accelerated data staging.
dimatura 7y ago
Seems like they are using photogrammetry, most likely with a large amount of high resolution images. Photogrammetry under these conditions can be fairly computationally expensive, specially if they use aim for high quality reconstructions (which they probably should, given this is a one-time or few-time cost). Of course, it depends on what algorithms they're using, of which I have no idea, but there's plenty of fairly fancy methods, many involving deep learning in the pipeline.

dekhn 7y ago

Supercomputers are exactly the wrong sort of tool to use for this- nearly every supercomputer has crappy disk IO and tons of fast CPU and RAM and network. Using a supercomputer for this would leave the expensive elements like the GPUs nearly idle, and the IO subsystem would be the bottleneck.

HenryKissinger 7y ago
Yes, I'm sure you saw something the world class experts of the National Geo-spatial Intelligence Agency missed. /s
- dekhn 7y ago
  I'm a supercomputer expert - that's a part of my job - who works on data processing fulltime. I know these codes, I know the hardware architecture, and my statement above is technically correct. I'm not missing anything.
  - HenryKissinger 7y ago
    You don't know shit.
    - dekhn 7y ago
      I direct you to my literature. https://scholar.google.com/citations?user=TFgipkIAAAAJ&hl=en&oi=sra https://scholar.google.com/citations?user=TFgipkIAAAAJ&hl=en... You may find especially interesting an article I published that used an embarassingly parallel computing system that I built which ran on Google's internal infrastructure (not a supercomputer) in response to my codes not running well on supercomputers. If you wish to have a substantive discussion about my statements (rather than flinging insults), I'd be happy to.
  - lostgame 7y ago
    Wow, HenryKissinger - you're an asshole. dekhn has provided proof of their claims - what does telling them they're full of shit without any backup of your claims add to the conversation but unverified negativity?

blackbrokkoli 7y ago

The article misses the point a bit. There are 3D maps of the world, or rather, you can make your own on a weekend. Topographic data is out there for most things, the problem is just it's very inconsistent and of course, resolution. And I guess for the whole world you do need a decent computer. To prove my point, here is some art I made [1] from open data (I think NASA) with Blender and QGIS, meaning free software stack on very much a normal computer. My model was the state Schleswig Holstein, a german county, but as said, you can find data for pretty much everything. The resolution is not astonishing, but enough to spot Germany's only high sea island, the "Wattenmeer" where tide causes some spots to lie below water level on average, the mouth of the Elbe and more cool stuff when you know where to look. My point isn't to downplay the effort of EarthDEM, I just want to make more people aware of what is already very much out there :) [1]: https://imgur.com/a/rZUTajX https://imgur.com/a/rZUTajX

rurban 7y ago
> And I guess for the whole world you do need a decent computer. Nope, resolution is the problem, not scale. You need low orbit stereo photos, like from an airplane. Without clouds. With satellite you only get 30m, with airplanes you get under 1m. Cities usually rent a plane a year to adjust their maps. Countries don't, they usually just rely on the cheap satellite photos.

pbhjpbhj 7y ago

The Arctic 3D map mentioned in OP, https://livingatlas2.arcgis.com/arcticdemexplorer/ https://livingatlas2.arcgis.com/arcticdemexplorer/ -- covers the whole area from 60deg N.

frankharv 7y ago

When I was in port in Gitmo in 1987 there was a civilian boat in harbor with a large tethered balloon. They were mapping Cuba for the US Defense Mapping Agency(DMA). So I would guess that DMA had a 3D terrain map around 1989. I know it was used for cruise missile flight planning. No idea what the resolution was. Even with our very detailed underwater mapping the Navy still has areas that are not well covered. https://www.popularmechanics.com/military/navy-ships/a24158/uss-san-francisco-mountain-incident/ https://www.popularmechanics.com/military/navy-ships/a24158/...

crazygringo 7y ago

Serious question: what on earth have supercomputers got to do with this? My impression is supercomputers exist mainly for incredibly intensive large-scale simulation/calculation that can't be subdivided into parts -- e.g. weather or nuclear explosion simulation. 3D map processing feels like the literal opposite of that -- trivially parallelizable, with a much higher ratio of data/IO to calculation. Are they running out of tasks for Blue Waters to do, or trying to find a high-profile project to justify it politically or something? I really can't imagine for the life of me why you wouldn't just run this in any enterprise-grade cloud.

eternalny1 7y ago
It seems like they are using IMAGES of the area and then using visual processing to determine the 3D height map. That would be intense visual processing, stitching together various angles of photos to work out a terrain elevation. They are not dealing with a 3D point cloud of elevation data.
- HellDunkel 7y ago
  Sounds as if they plan to use agisoft photoscan to process all satellite images they can find.
jobigoud 7y ago
> calculation that can't be subdivided into parts -- e.g. weather I'm not sure how weather simulation work but I always wondered why they aren't performed cellular automaton style. Bottom up rather than top-down. At each time step a given cell state is computed based on the state of its neighbors at t-1. This should be parallelizable. I feel it's also how it works in the real world anyway.
- petschge 7y ago
  That is exactly what weather simulations do. Same most other kinds of hydrodynamic simulations (simulation of gas or fluid flows). The thing is: you need the state at t-1 of all your neighbors. Then you can do a small timestep to get from t-1 to time t. And then you need the NEW state of your neighbors. That requires a fast interconnect. Which HPC machines have, unlike most clouds or commodity clusters. In other words, yes it is parallelizable, but not trivially so, because the different grid points are coupled.

maxemitchell 7y ago

I found a much better source regarding this in the form of a November 2018 white paper[1]. Interesting is that they say it will take 3.2-3.5 years of Blue Waters computing time. [1]: https://www.exascale.org/bdec/sites/www.exascale.org.bdec/files/whitepapers/Kramer_BDEC2__WP_v2.pdf https://www.exascale.org/bdec/sites/www.exascale.org.bdec/fi...

sorenjan 7y ago

It's a bit strange that they use DigitalGlobe imagery, since DigitalGlobe owns 50% of Vricon who use aerial and satellite photos to make 3D maps. Their tagline is "The globe in 3D". https://www.vricon.com/news/saab-digitalglobe-announce-vricon-joint-venture-create-globe-3d/ https://www.vricon.com/news/saab-digitalglobe-announce-vrico...

ungzd 7y ago

Article title is misleading. It's not "3D map", like city map with 3D buildings or something like that, it's just a DEM, digital elevation model. It's not a map at all. It's useful for addition information to build a map, but it's not a map.

mnutsch23 7y ago

This has already been a thing for at least a year. For example: Go to Google Maps, satellite view. Then zoom in and rotate with Ctrl + click & drag. https://www.google.com/maps/@32.7644941,-96.8061872,1158a,35y,11.78h,55.61t/data=!3m1!1e3 https://www.google.com/maps/@32.7644941,-96.8061872,1158a,35... Microsoft and Mapbox also have an experimental project where computer vision on cars maps the location of recognized objects. This is probably true of every company researching self-driving cars.

mceoin 7y ago

This is very exciting. One step closer to a manual for spaceship earth.

llsf 7y ago

I am wondering what resolution they work with. Because earth crust is moving continuously, and sometimes suddenly like 6 feet at once: https://www.latimes.com/california/story/2019-07-22/ridgecrest-earthquake-images-broken-ground https://www.latimes.com/california/story/2019-07-22/ridgecre... I like the idea to survey the whole planet, I would be curious to see what resolution they end up with, and how often it is updated.

mlwhiz 7y ago

Don't they already have the map of the world in 3D?

csteubs 7y ago

Another step in the right direction, but satellites will continue to be the bottleneck to unlocking the full value of maps.

mjparrott 7y ago

How is this different than Google Earth / Maps?

paparush 7y ago

Paging Hiro Protagonist

gao8a 7y ago

The silent cartographer

rurban 7y ago

Marketing: "With EarthDEM, we will take this transformative new capability to where most of us live, to cities and coastlines, that will directly impact their lives." Tech: We need these maps for our cruise missiles.