6 ms·

"When they were turbocharged with specialized graphical processing chips, they were between about 100,000 and 2 billion times faster than their simulations." N

"When they were turbocharged with specialized graphical processing chips, they were between about 100,000 and 2 billion times faster than their simulations." Now the critical question is: How much faster is it without AI, just because of the specialized dedicated processing chips? Otherwise, they might be comparing a single virtualized CPU core against a high-end GPU for things like matrix multiplication ... and then the result that GPU > slow CPU isn't really that impressive.

rrss 7y ago
An alternative question is: how much faster is it with the neural network-based emulation ("AI"), without the used of the specialized dedicated processing chips? I think the answer to this gives the information you are looking for. The paper answers this question: > While the simulations presented typically run in minutes to days, the DENSE emulators can process multiple sets of input parameters in milliseconds to a few seconds with one CPU core, or even faster when using a Titan X GPU card. For the GCM simulation which takes about 1150 CPU-hours to run, the emulator speedup is a factor of 110 million on a like-for-like basis, and over 2 billion with a GPU card. The speed up achieved by DENSE emulators for each test case is shown in Figure 2(h)
allovernow 7y ago
>Now the critical question is: How much faster is it without AI, just because of the specialized dedicated processing chips? Based on similar work we are doing at the startup I work for, this isn't just GPU magic. ML is a heuristic alternative to simulations which already operate on specialized GPUs and TPUs. This modeling acceleration is one of the many ways in which ML is poised to change everything. The same way that a human can, for instance, approximately draw iso-temperature lines around a candle flame, without having to perform simulations...except the neural net is some 99%+ as accurate and detailed as a full simulation. That's exactly why neural nets excel - they learn complex heuristics much like humans do, but with the added power of digitized computation and memory.
- nitrogen 7y ago
  What is the worst case loss of accuracy possible from the ML version?
  - petschge 7y ago
    Usually there is not error bound (or we don't know how to proof them), so the worst case is that it is infinitely wrong (or as wrong as floating point numbers can express).
    - allovernow 7y ago
      You're probably right, but from everything I've seen to date, the representations of functions that neural networks learn tend to be smooth, which is to say if your input is similar to the domain covered by your training data, you're quite unlikely to get totally crazy outputs...though I can't say I've come across anything that proves this. However, I think the degree to which neural networks can extrapolate is still an open question, so it is important to thoroughly sample your problem space during training. That's a bit of an artform given that these spaces are typically high-dimensional.
    - DonHopkins 7y ago
      Off by as much as two infinities, since floating point numbers have +infinity and -infinity. ;)
- tomp 7y ago
  Don't most really complex physical calculations / simulations (e.g. weather, planetary movements etc.) involve chaotic interactions? So an NN being 99% correct will still result in "catastrophic" differences down the line?
  - whatshisface 7y ago
    So will quantization error when you grid your conventional sim. If you can make your cells and timesteps smaller with the NN than without, the loss in accuracy in one place could be compensated by the gain elsewhere. The only issue is, good luck with developing a formal theory of error propagation through your trained network that is faster to compute than the conventional sim itself. Sometimes you care about strict formal guarantees about sim error and other times you don't.
lapink 7y ago
I made a benchmark a few months ago for a friend of mine working on computational physics. She needed to compute the determinant of a lot of matrices to perform MCMC sampling and wanted to know how much GPUs could speed up things. For this task, one V100 was equivalent to a 100 cpu cores, so a speed up of 100,000 is definitely not coming only from using GPUs vs CPUs.
dukoid 7y ago
https://twitter.com/ID_AA_Carmack/status/1228824996228800513 https://twitter.com/ID_AA_Carmack/status/1228824996228800513
cshenton 7y ago
The real question is how much the speed up is due to the low quality of the initial implementation. I’ve seen speed ups of 100-1000x on the same hardware. Academic simulation code is often bad.

rrss 7y ago

An alternative question is: how much faster is it with the neural network-based emulation ("AI"), without the used of the specialized dedicated processing chips? I think the answer to this gives the information you are looking for. The paper answers this question: > While the simulations presented typically run in minutes to days, the DENSE emulators can process multiple sets of input parameters in milliseconds to a few seconds with one CPU core, or even faster when using a Titan X GPU card. For the GCM simulation which takes about 1150 CPU-hours to run, the emulator speedup is a factor of 110 million on a like-for-like basis, and over 2 billion with a GPU card. The speed up achieved by DENSE emulators for each test case is shown in Figure 2(h)

allovernow 7y ago

>Now the critical question is: How much faster is it without AI, just because of the specialized dedicated processing chips? Based on similar work we are doing at the startup I work for, this isn't just GPU magic. ML is a heuristic alternative to simulations which already operate on specialized GPUs and TPUs. This modeling acceleration is one of the many ways in which ML is poised to change everything. The same way that a human can, for instance, approximately draw iso-temperature lines around a candle flame, without having to perform simulations...except the neural net is some 99%+ as accurate and detailed as a full simulation. That's exactly why neural nets excel - they learn complex heuristics much like humans do, but with the added power of digitized computation and memory.

nitrogen 7y ago
What is the worst case loss of accuracy possible from the ML version?
- petschge 7y ago
  Usually there is not error bound (or we don't know how to proof them), so the worst case is that it is infinitely wrong (or as wrong as floating point numbers can express).
  - allovernow 7y ago
    You're probably right, but from everything I've seen to date, the representations of functions that neural networks learn tend to be smooth, which is to say if your input is similar to the domain covered by your training data, you're quite unlikely to get totally crazy outputs...though I can't say I've come across anything that proves this. However, I think the degree to which neural networks can extrapolate is still an open question, so it is important to thoroughly sample your problem space during training. That's a bit of an artform given that these spaces are typically high-dimensional.
  - DonHopkins 7y ago
    Off by as much as two infinities, since floating point numbers have +infinity and -infinity. ;)
tomp 7y ago
Don't most really complex physical calculations / simulations (e.g. weather, planetary movements etc.) involve chaotic interactions? So an NN being 99% correct will still result in "catastrophic" differences down the line?
- whatshisface 7y ago
  So will quantization error when you grid your conventional sim. If you can make your cells and timesteps smaller with the NN than without, the loss in accuracy in one place could be compensated by the gain elsewhere. The only issue is, good luck with developing a formal theory of error propagation through your trained network that is faster to compute than the conventional sim itself. Sometimes you care about strict formal guarantees about sim error and other times you don't.

lapink 7y ago

I made a benchmark a few months ago for a friend of mine working on computational physics. She needed to compute the determinant of a lot of matrices to perform MCMC sampling and wanted to know how much GPUs could speed up things. For this task, one V100 was equivalent to a 100 cpu cores, so a speed up of 100,000 is definitely not coming only from using GPUs vs CPUs.

dukoid 7y ago

https://twitter.com/ID_AA_Carmack/status/1228824996228800513 https://twitter.com/ID_AA_Carmack/status/1228824996228800513

cshenton 7y ago

The real question is how much the speed up is due to the low quality of the initial implementation. I’ve seen speed ups of 100-1000x on the same hardware. Academic simulation code is often bad.