6 ms·

If you understood anything about how literally every chemistry problem scales non-linearly, and requires exponential amount of sampling to yield any predictive

If you understood anything about how literally every chemistry problem scales non-linearly, and requires exponential amount of sampling to yield any predictive meaning you wouldn't talk about things you don't understand lol. Peta=>Exa equals 1000x so you can get 31x the accuracy in a monte carlo assuming perfect parallelism (at 1000x the cost), and that's one of the better scaling things it could be used for. It's fun to have big toys sure, it's fun to make big GPU clusters. But if they spent what they spend on this computer, on just funding students to solve problems and toss them a 3090 100000000x more scientific breakthroughs would happen. This machine is 60% paperweight, at best, with a hefty budget for good old fashioned contract pork.

YesWeWill 4y ago
anigbrowl 4y ago
I kind of assumed (from a largely ignorant perspective) that this was part vanity/pork project as yu say, but that the real underlying purpose was to either simulate nuclear explosions for the DoE or to synthesize stupid large AI models with a trillion variables. If the people in charge of this thing had an epiphany (or a blackout, take your pick) and left you with the keys for a year, what could you do with this inefficient but impressively large cluster, besides anchoring your paperwork?
- dekhn 4y ago
  at this point it seems like the folks building supercomputers for DoE and the folks building TPUs for Google should be talking. Both sides have interesting technology, but the communities are too far apart. Modern ML training is basically very similar to supercomputing that I think by working together, DeepMind could make better use of an exascale supercomputer than most scientists running simulations. Some of the network innovations in TPUv4 could be used in supercomputers.
  - alpineidyll3 4y ago
    100% agree with this. Although mini-batch SGD is much more parallel than most problems. the scientific codes which parallelize poorly often parallelize poorly because they are written in ancient languages with support for whatever supercomputer interconnect bolted on poorly, Whereas TPU's + JAX have beautiful functional abstractions for distributed tensor computations. Just funding re-writes of all the basic math/physics stack into a language with a PORTABLE parallel functional design and perhaps a compilation layer would definitely get more basic science done than this thing.
deleted 4y ago
[deleted]
closedloop129 4y ago
Is there a limit to the usability of that accuracy or will there be a huge increase in the demand for electricity soon that is only limited by the ability of NVIDIA and AMD to produce GPUs?
stonogo 4y ago
> Peta=>Exa equals 1000x so you can get 31x the accuracy in a monte carlo assuming perfect parallelism True. > (at 1000x the cost) Not as true. Summit, the previous machine, hit 148 petaflops (Rmax) at a cost of $325 million. Frontier has already hit 1102 petaflops (Rmax) at a cost of just under $600 million.
- alpineidyll3 4y ago
  You're right.

YesWeWill 4y ago

anigbrowl 4y ago

I kind of assumed (from a largely ignorant perspective) that this was part vanity/pork project as yu say, but that the real underlying purpose was to either simulate nuclear explosions for the DoE or to synthesize stupid large AI models with a trillion variables. If the people in charge of this thing had an epiphany (or a blackout, take your pick) and left you with the keys for a year, what could you do with this inefficient but impressively large cluster, besides anchoring your paperwork?

dekhn 4y ago
at this point it seems like the folks building supercomputers for DoE and the folks building TPUs for Google should be talking. Both sides have interesting technology, but the communities are too far apart. Modern ML training is basically very similar to supercomputing that I think by working together, DeepMind could make better use of an exascale supercomputer than most scientists running simulations. Some of the network innovations in TPUv4 could be used in supercomputers.
- alpineidyll3 4y ago
  100% agree with this. Although mini-batch SGD is much more parallel than most problems. the scientific codes which parallelize poorly often parallelize poorly because they are written in ancient languages with support for whatever supercomputer interconnect bolted on poorly, Whereas TPU's + JAX have beautiful functional abstractions for distributed tensor computations. Just funding re-writes of all the basic math/physics stack into a language with a PORTABLE parallel functional design and perhaps a compilation layer would definitely get more basic science done than this thing.

deleted 4y ago

[deleted]

closedloop129 4y ago

Is there a limit to the usability of that accuracy or will there be a huge increase in the demand for electricity soon that is only limited by the ability of NVIDIA and AMD to produce GPUs?

stonogo 4y ago

> Peta=>Exa equals 1000x so you can get 31x the accuracy in a monte carlo assuming perfect parallelism True. > (at 1000x the cost) Not as true. Summit, the previous machine, hit 148 petaflops (Rmax) at a cost of $325 million. Frontier has already hit 1102 petaflops (Rmax) at a cost of just under $600 million.

alpineidyll3 4y ago
You're right.