3 ms·

Exascale was an arbitrary target- it didn't unlock any magical capabilities. In fact the supercomputer folks are now saying the real barrier that will unlock th

by dekhn 2y ago

Exascale was an arbitrary target- it didn't unlock any magical capabilities. In fact the supercomputer folks are now saying the real barrier that will unlock their science is "zettaflops" (just saw a post from a luminary about it). (also please don't be rude or condescending, it detracts from your argument)

alephnerd 2y ago
Exascale is the primary target decided among everyone in the HPC community in the mid-late 2000s because FLOPS (floating point operations per second) is the unit used to benchmark, as there are various different variables like compiler, architecture, etc are very difficult to account for. It's functionally the same as arguing that GB or TB are arbitrary units to represent storage.
- dekhn 2y ago
  Yes, that's correct: GB and TB are arbitrary units. We use them because historically, successive multiples of 1000 have been used to represent significant changes in size, mass, and velocity. I used to work with/for NERSC and was around when they announced exascale as a target, and now they want to target zettascale. There is no magic threshold where science simulations suddenly work better as you scale up. It's mainly about setting goals that are 10-15 years away to stimulate spending and research.
  - alephnerd 2y ago
    > I used to work with/for NERSC and was around when they announced exascale as a target We most likely crossed paths. How close were you to faculty in AMPLab? > It's mainly about setting goals that are 10-15 years away to stimulate spending and research. And that's not a barrier to you? That feels very condescending about applied research or the amount of effort put into the entire Distributed Systems field.
    - dekhn 2y ago
      I am being condescending about the effort put into the distributed systems field- very specifically, about classical supercomputers. How close was I to faculty in AMPLab? Pretty close; I attended a retreat one year, helped steer funding their way, tried to hire Matei into Google Research, and have chatted with Patterson extensively around the time he wrote this (https://www.nytimes.com/2011/12/06/science/david-patterson-enlist-computer-scientists-in-cancer-fight.html https://www.nytimes.com/2011/12/06/science/david-patterson-e...) then later when he worked on TPUs at Google. (I'm not a stellar researcher or anything, really more of a functionary, but the one thing I do have is an absolutely realistic understanding of how academic and industrial HPC works)
    - kortilla 2y ago
      It’s not a barrier because there is nothing qualitatively different at 999 vs 1000. It’s just a goal. This is not condescending to the field at all. Crossing an arbitrary goal that is very difficult to get to is still impressive. Just stop using the “breaking the barrier” phrase.
    - digging 2y ago
      > > It's mainly about setting goals that are 10-15 years away to stimulate spending and research. > And that's not a barrier to you? In what way is that a barrier? Barrier and goal aren't synonymous, it's just marketing speak that confuses them. Running a marathon is not a barrier, it's a goal, even if many people can't reach it. A combat zone at the halfway point of the marathon is a barrier because it requires a completely different approach to solve.
  - bbatha 2y ago
    > There is no magic threshold where science simulations suddenly work better as you scale up. For certain problems there are absolutely magic thresholds. ML was famously abandoned for 2 decades because the computers were too slow, and the ML revolution has only been possible because of having a ~teraflop on a per-machine level. Weather and climate models are another where there have been concrete compute targets, a whole earth model requires about 10 teraflops (hence the earth simulator super computer). ~1 meter resolution is an exascale level target.

by dekhn 2y ago

alephnerd 2y ago

Exascale is the primary target decided among everyone in the HPC community in the mid-late 2000s because FLOPS (floating point operations per second) is the unit used to benchmark, as there are various different variables like compiler, architecture, etc are very difficult to account for. It's functionally the same as arguing that GB or TB are arbitrary units to represent storage.

dekhn 2y ago
Yes, that's correct: GB and TB are arbitrary units. We use them because historically, successive multiples of 1000 have been used to represent significant changes in size, mass, and velocity. I used to work with/for NERSC and was around when they announced exascale as a target, and now they want to target zettascale. There is no magic threshold where science simulations suddenly work better as you scale up. It's mainly about setting goals that are 10-15 years away to stimulate spending and research.
- alephnerd 2y ago
  > I used to work with/for NERSC and was around when they announced exascale as a target We most likely crossed paths. How close were you to faculty in AMPLab? > It's mainly about setting goals that are 10-15 years away to stimulate spending and research. And that's not a barrier to you? That feels very condescending about applied research or the amount of effort put into the entire Distributed Systems field.
  - dekhn 2y ago
    I am being condescending about the effort put into the distributed systems field- very specifically, about classical supercomputers. How close was I to faculty in AMPLab? Pretty close; I attended a retreat one year, helped steer funding their way, tried to hire Matei into Google Research, and have chatted with Patterson extensively around the time he wrote this (https://www.nytimes.com/2011/12/06/science/david-patterson-enlist-computer-scientists-in-cancer-fight.html https://www.nytimes.com/2011/12/06/science/david-patterson-e...) then later when he worked on TPUs at Google. (I'm not a stellar researcher or anything, really more of a functionary, but the one thing I do have is an absolutely realistic understanding of how academic and industrial HPC works)
  - kortilla 2y ago
    It’s not a barrier because there is nothing qualitatively different at 999 vs 1000. It’s just a goal. This is not condescending to the field at all. Crossing an arbitrary goal that is very difficult to get to is still impressive. Just stop using the “breaking the barrier” phrase.
  - digging 2y ago
    > > It's mainly about setting goals that are 10-15 years away to stimulate spending and research. > And that's not a barrier to you? In what way is that a barrier? Barrier and goal aren't synonymous, it's just marketing speak that confuses them. Running a marathon is not a barrier, it's a goal, even if many people can't reach it. A combat zone at the halfway point of the marathon is a barrier because it requires a completely different approach to solve.
- bbatha 2y ago
  > There is no magic threshold where science simulations suddenly work better as you scale up. For certain problems there are absolutely magic thresholds. ML was famously abandoned for 2 decades because the computers were too slow, and the ML revolution has only been possible because of having a ~teraflop on a per-machine level. Weather and climate models are another where there have been concrete compute targets, a whole earth model requires about 10 teraflops (hence the earth simulator super computer). ~1 meter resolution is an exascale level target.