3 ms·

So the article basically says: 1) Some stuff is already written in Fortran so they don't want to rewrite that. I dig it. 2) It's fast (except C sometimes) but

by jxcole 9y ago

So the article basically says: 1) Some stuff is already written in Fortran so they don't want to rewrite that. I dig it. 2) It's fast (except C sometimes) but easier to write than c. Like 100x faster than python. I'm not sure about number two. With the gpu processing revolution wouldn't a python/TensorFlow stack be faster than Fortran? Am I missing something? I remember talking to someone who had worked heavily on atmospheric weather predictors recently and her description of the program was: we divide the space into tiny little cubes and then run some differential physics equations to predict what will happen next. My basic questions to her: 1) From a computational perspective this seems very GPU friendly. 2) Why not use a convolutional neural network? If you use the same data for training you will probably wind up with a more accurate prediction than a theoretically based physics system. Her reaction was basically that she hadn't heard of these things before so my impression is in fact that the physics community is just behind and they will catch up when they are ready. Hope I'm not missing something here.

openasocket 9y ago
TensorFlow is for machine learning, not general purpose computations. And no, you will not get better results from a neural network than state of the art computational fluid dynamics. As for general purpose GPU programming, some parts physics are GPU friendly put not all of them.
- jabl 9y ago
  IIRC there was some project doing more or less real-time weather forecasts for small areas (think airports and such) that used deep learning instead of traditional CFD style simulations. They were able to do it with several orders of magnitude less CPU usage than the CFD calculations.
  - sidlls 9y ago
    But was it more accurate? Was it for the same purpose? Getting within a few percent of a CFD model might be good enough for industrial use. For studying systems (from the perspective of scientific investigation) it's not even close.
    - noobermin 9y ago
      Of course it isn't which is why ML isn't used by us in the computational community...yet.
noobermin 9y ago
Dearest me. I hope you're not under the impression that the performant part of your ML code is written in python. Python is the interface to a C++ libary in tensorflow looking at its repo on github. I don't use tensorflow, but if it's anything like numpy and scipy, they are interfaces to C libraries which are extremely performant. If the question is why can't a CFD be in python...and the answer is well, I don't know if anyone has tried it due to your point 1).
- jxcole 9y ago
  I don't think you understand Tensorflow. Tensorflow compiles tensor mathematics into a shader that can be run on the gpu. This is usually faster than acpu computation.
  - noobermin 9y ago
    Alright. I imagine the shader compiler too isn't written in python, is it?
scottmmjackson 9y ago
Dude. https://developer.nvidia.com/cuda-fortran https://developer.nvidia.com/cuda-fortran And no, neural networks aren't suitable for discrete problems.
- blt 9y ago
  What do you mean by "discrete problems"? By my understanding, weather prediction is basically solving a large system of partial differential equations. Sure the grid methods are a discrete approximation of the true problem, but it is not what I would normally think of as a "discrete problem". Training a neural network to map "current state of atmosphere" to "state of atmosphere in the future" is definitely possible to do with a neural network, and sounds like a good idea to me.
deleted 9y ago
[deleted]
fantispug 9y ago
A lot of science is done with GPUs ... in FORTRAN and C++. I believe an n body simulation in GPUs is what started the idea of using GPU for general computation. I would be extremely surprised if you could get similar or better predictions for the same amount of computation from an off the shelf CNN than from a carefully tuned physical model. (I would love to see counterexamples). Even then I would question how well it extrapolates; you generally have a very good idea of where your physics simulation will break down. I think your friend is right; the physics community can't afford to throw away as much code as the tech community. I wouldn't spend years building a simulation in tensorflow because it will likely be obsolete before I finished (leading to the same problems as FORTRAN but increasing the amount of dependencies). In any case, if it works well enough it's not going to be rewritten. Eventually ML will become part of physics simulations, but I don't think there will be easy wins in these well developed areas.
The_suffocated 9y ago
While I can imagine that neural networks are useful in predicting rainfalls or precipitations, I don’t see how they are suitable for some tasks, such as the forecasting of typhoon paths. A neural network is basically a function interpolation device. To use it for prediction, the implicit assumption is that the function it approximates is somehow well behaved, but some weather systems are chaotic. Function interpolation doesn’t seem very useful in this regard. From what I see from googling, cyclone tracks forecasting by neural network is definitely an active area of research, but apparently they aren’t practical yet. The lady you talked to were probably working on some chaotic systems like this.

by jxcole 9y ago

openasocket 9y ago

TensorFlow is for machine learning, not general purpose computations. And no, you will not get better results from a neural network than state of the art computational fluid dynamics. As for general purpose GPU programming, some parts physics are GPU friendly put not all of them.

jabl 9y ago
IIRC there was some project doing more or less real-time weather forecasts for small areas (think airports and such) that used deep learning instead of traditional CFD style simulations. They were able to do it with several orders of magnitude less CPU usage than the CFD calculations.
- sidlls 9y ago
  But was it more accurate? Was it for the same purpose? Getting within a few percent of a CFD model might be good enough for industrial use. For studying systems (from the perspective of scientific investigation) it's not even close.
  - noobermin 9y ago
    Of course it isn't which is why ML isn't used by us in the computational community...yet.

noobermin 9y ago

Dearest me. I hope you're not under the impression that the performant part of your ML code is written in python. Python is the interface to a C++ libary in tensorflow looking at its repo on github. I don't use tensorflow, but if it's anything like numpy and scipy, they are interfaces to C libraries which are extremely performant. If the question is why can't a CFD be in python...and the answer is well, I don't know if anyone has tried it due to your point 1).

jxcole 9y ago
I don't think you understand Tensorflow. Tensorflow compiles tensor mathematics into a shader that can be run on the gpu. This is usually faster than acpu computation.
- noobermin 9y ago
  Alright. I imagine the shader compiler too isn't written in python, is it?

scottmmjackson 9y ago

Dude. https://developer.nvidia.com/cuda-fortran https://developer.nvidia.com/cuda-fortran And no, neural networks aren't suitable for discrete problems.

blt 9y ago
What do you mean by "discrete problems"? By my understanding, weather prediction is basically solving a large system of partial differential equations. Sure the grid methods are a discrete approximation of the true problem, but it is not what I would normally think of as a "discrete problem". Training a neural network to map "current state of atmosphere" to "state of atmosphere in the future" is definitely possible to do with a neural network, and sounds like a good idea to me.

deleted 9y ago

[deleted]

fantispug 9y ago

A lot of science is done with GPUs ... in FORTRAN and C++. I believe an n body simulation in GPUs is what started the idea of using GPU for general computation. I would be extremely surprised if you could get similar or better predictions for the same amount of computation from an off the shelf CNN than from a carefully tuned physical model. (I would love to see counterexamples). Even then I would question how well it extrapolates; you generally have a very good idea of where your physics simulation will break down. I think your friend is right; the physics community can't afford to throw away as much code as the tech community. I wouldn't spend years building a simulation in tensorflow because it will likely be obsolete before I finished (leading to the same problems as FORTRAN but increasing the amount of dependencies). In any case, if it works well enough it's not going to be rewritten. Eventually ML will become part of physics simulations, but I don't think there will be easy wins in these well developed areas.

The_suffocated 9y ago

While I can imagine that neural networks are useful in predicting rainfalls or precipitations, I don’t see how they are suitable for some tasks, such as the forecasting of typhoon paths. A neural network is basically a function interpolation device. To use it for prediction, the implicit assumption is that the function it approximates is somehow well behaved, but some weather systems are chaotic. Function interpolation doesn’t seem very useful in this regard. From what I see from googling, cyclone tracks forecasting by neural network is definitely an active area of research, but apparently they aren’t practical yet. The lady you talked to were probably working on some chaotic systems like this.