3 ms·

>Now the critical question is: How much faster is it without AI, just because of the specialized dedicated processing chips? Based on similar work we are doing

>Now the critical question is: How much faster is it without AI, just because of the specialized dedicated processing chips? Based on similar work we are doing at the startup I work for, this isn't just GPU magic. ML is a heuristic alternative to simulations which already operate on specialized GPUs and TPUs. This modeling acceleration is one of the many ways in which ML is poised to change everything. The same way that a human can, for instance, approximately draw iso-temperature lines around a candle flame, without having to perform simulations...except the neural net is some 99%+ as accurate and detailed as a full simulation. That's exactly why neural nets excel - they learn complex heuristics much like humans do, but with the added power of digitized computation and memory.

nitrogen 7y ago
What is the worst case loss of accuracy possible from the ML version?
- petschge 7y ago
  Usually there is not error bound (or we don't know how to proof them), so the worst case is that it is infinitely wrong (or as wrong as floating point numbers can express).
  - allovernow 7y ago
    You're probably right, but from everything I've seen to date, the representations of functions that neural networks learn tend to be smooth, which is to say if your input is similar to the domain covered by your training data, you're quite unlikely to get totally crazy outputs...though I can't say I've come across anything that proves this. However, I think the degree to which neural networks can extrapolate is still an open question, so it is important to thoroughly sample your problem space during training. That's a bit of an artform given that these spaces are typically high-dimensional.
  - DonHopkins 7y ago
    Off by as much as two infinities, since floating point numbers have +infinity and -infinity. ;)
tomp 7y ago
Don't most really complex physical calculations / simulations (e.g. weather, planetary movements etc.) involve chaotic interactions? So an NN being 99% correct will still result in "catastrophic" differences down the line?
- whatshisface 7y ago
  So will quantization error when you grid your conventional sim. If you can make your cells and timesteps smaller with the NN than without, the loss in accuracy in one place could be compensated by the gain elsewhere. The only issue is, good luck with developing a formal theory of error propagation through your trained network that is faster to compute than the conventional sim itself. Sometimes you care about strict formal guarantees about sim error and other times you don't.

nitrogen 7y ago

What is the worst case loss of accuracy possible from the ML version?

petschge 7y ago
Usually there is not error bound (or we don't know how to proof them), so the worst case is that it is infinitely wrong (or as wrong as floating point numbers can express).
- allovernow 7y ago
  You're probably right, but from everything I've seen to date, the representations of functions that neural networks learn tend to be smooth, which is to say if your input is similar to the domain covered by your training data, you're quite unlikely to get totally crazy outputs...though I can't say I've come across anything that proves this. However, I think the degree to which neural networks can extrapolate is still an open question, so it is important to thoroughly sample your problem space during training. That's a bit of an artform given that these spaces are typically high-dimensional.
- DonHopkins 7y ago
  Off by as much as two infinities, since floating point numbers have +infinity and -infinity. ;)

tomp 7y ago

Don't most really complex physical calculations / simulations (e.g. weather, planetary movements etc.) involve chaotic interactions? So an NN being 99% correct will still result in "catastrophic" differences down the line?

whatshisface 7y ago
So will quantization error when you grid your conventional sim. If you can make your cells and timesteps smaller with the NN than without, the loss in accuracy in one place could be compensated by the gain elsewhere. The only issue is, good luck with developing a formal theory of error propagation through your trained network that is faster to compute than the conventional sim itself. Sometimes you care about strict formal guarantees about sim error and other times you don't.