3 ms·

Someone in the know - why do you need a supercomputer for this? From an amateur position it seems suitable for commodity compute and a big ol map-reduce. Getti

Someone in the know - why do you need a supercomputer for this? From an amateur position it seems suitable for commodity compute and a big ol map-reduce. Getting chunk edges to align might be tricky, but the usual solution to that is to stagger the chunks and throw away the edges.

0xffff2 7y ago
There's not really a huge difference between supercomputers and a large number of ordinary computers these days. The computational hardware is basically the same. Only the communication interconnects are different. I strongly suspect that they don't need a supercomputer, but they had one available and it sounds cool. I sit one building over from the ~13th largest supercomputer in the world and I've definitely used it when I needed a bit more computing power than my laptop could provide. I used the supercomputer because I could, not because I needed to.
- dekhn 7y ago
  Did you have an embarassingly parallel code (like, "map"), or a code that was ported to supercomputers? I've worked on supercomputers for years and generally, one could not run code at scale unless it scaled- in terms of parallel performance- and also used the network in a non-trivial way. Supercomputers aren't just speedups for normal codes (I'm pretty sure you know this). The real issue with modern supercomputers is that basically none of them have decent disk IO. That's what differs between modern Internet/cloud clusters and supercomputers. Cloud clusters emphasize very high connectivity between durable storage and worker nodes. None of the supercomputers have a decent disk IO stack (mostly GPFS and Lustre) and this ends up being no end of pain for applicaiton developers. The only recent improvement in this area was "burst buffers", but that's really just accelerated data staging.
dimatura 7y ago
Seems like they are using photogrammetry, most likely with a large amount of high resolution images. Photogrammetry under these conditions can be fairly computationally expensive, specially if they use aim for high quality reconstructions (which they probably should, given this is a one-time or few-time cost). Of course, it depends on what algorithms they're using, of which I have no idea, but there's plenty of fairly fancy methods, many involving deep learning in the pipeline.

0xffff2 7y ago

There's not really a huge difference between supercomputers and a large number of ordinary computers these days. The computational hardware is basically the same. Only the communication interconnects are different. I strongly suspect that they don't need a supercomputer, but they had one available and it sounds cool. I sit one building over from the ~13th largest supercomputer in the world and I've definitely used it when I needed a bit more computing power than my laptop could provide. I used the supercomputer because I could, not because I needed to.

dekhn 7y ago
Did you have an embarassingly parallel code (like, "map"), or a code that was ported to supercomputers? I've worked on supercomputers for years and generally, one could not run code at scale unless it scaled- in terms of parallel performance- and also used the network in a non-trivial way. Supercomputers aren't just speedups for normal codes (I'm pretty sure you know this). The real issue with modern supercomputers is that basically none of them have decent disk IO. That's what differs between modern Internet/cloud clusters and supercomputers. Cloud clusters emphasize very high connectivity between durable storage and worker nodes. None of the supercomputers have a decent disk IO stack (mostly GPFS and Lustre) and this ends up being no end of pain for applicaiton developers. The only recent improvement in this area was "burst buffers", but that's really just accelerated data staging.

dimatura 7y ago

Seems like they are using photogrammetry, most likely with a large amount of high resolution images. Photogrammetry under these conditions can be fairly computationally expensive, specially if they use aim for high quality reconstructions (which they probably should, given this is a one-time or few-time cost). Of course, it depends on what algorithms they're using, of which I have no idea, but there's plenty of fairly fancy methods, many involving deep learning in the pipeline.