4 ms·

> Now, if there ever was a life-critical physical phenomenon, it is lift production by an airliner wing. But we don't actually have a "causal" explanation for i

> Now, if there ever was a life-critical physical phenomenon, it is lift production by an airliner wing. But we don't actually have a "causal" explanation for it, though we do have an accurate mathematical model and decades of experimental evidence This is argument is complete nonsense. Navier Stokes has a rigorous derivation based on extremely high-fidelity assumptions, such as conservation of mass, momentum, and energy. We understand these assumptions, and we understand regimes in which using N-S would result in catastrophe (such as rarefied gases, relativistic velocities, etc.) Neural networks require data. The Navier stokes does not need to be 'trained'. Deep networks have very little a-priori knowledge baked-in (from a baysian perspective there are priors such as translation invariance that are intrinsic). They are admittedly extremely useful, because they are high dimensional (and so are universal approximators) and can be trained efficiently. Furthermore, you can develop an intuitive approach to many fluid flows. I can provide a much better estimate of the drag profile for a given wing geometry than an untrained person. No such analog is possible with deep nets, which as significantly more opaque in terms of dynamics and non-linear response. The only way his comments make any sense is if you assume he isn't talking about physical models, like Navier stokes, and instead consider turbulence models, such as RANS or LES. These are parameterized models, and are used for turbulence modeling. They also have little physical intuition. However, this is not the same as saying we do not have high confidence in physical-based systems such as Navier stokes. Source: I have a PhD in CFD and several ML publications.

ericjang 7y ago
Yann was not making a statement about deep nets' ability (or lack thereof) to fly a plane in a way that matches expert design. He's making the point that the ML field's obsession with causal inference (and causal discovery) is overrated, precisely because our gold standards of interpretable, safety-critical systems (airplane flight) are based on Navier Stokes/CFD. Planes were made to fly based on empirical validation of these models, long before we gained a more detailed understanding of how causality (the equations and models themselves are time reversible, implying that they contain imperfect knowledge of causality)
- ssivark 7y ago
  > Planes were made to fly based on empirical validation of these models, long before we gained a more detailed understanding of how causality And their success is so repeatable that if they fail once we make a Really Big Deal out of it. I don’t think any ML model is close to that level of engineering rigor, let alone deep learning. Moving on to aerodynamics, we have a pretty good causal model and can simulate the system given a pattern of boundary conditions. Further, some people (who have studied aero/CFD) can even intuitively predict approximately what happens (otherwise we would have a hard time designing planes!). It just so happens that it’s not as simple as high school physics, and cannot be compressed in to perfect+simple rules (trade off between those two). Speaking in the context of time reversibility, you are being fast and loose, and using the word “causality” in a sense that is irrelevant to the rest of your comment.
allovernow 7y ago
>This is argument is complete nonsense. Navier Stokes has a rigorous derivation based on extremely high-fidelity assumptions, such as conservation of mass, momentum, and energy. We understand these assumptions, and we understand regimes in which using N-S would result in catastrophe (such as rarefied gases, relativistic velocities, etc.) I think you missed the point. Navier-Stokes does not explain lift, nor does Bernoulli's law or Netwon's third. Like the friction under an ice skate, it's a phenomenon we take for granted in science and engineering without a rigorous explanation >Neural networks require data. The Navier stokes does not need to be 'trained' But you can, and not just conceivably, but practically train a neural net to approximate navier Stokes to a high degree of accuracy with a fraction of the computation of a 3D finite model. Yeah, it will functionally be a black box, but LeCunn seems to be arguing that it's opacity doesn't make it any less valid than our opaque intuition of flight which does not stop us from flying thousands of jets every day.
- shoyer 7y ago
  > I think you missed the point. Navier-Stokes does not explain Lift, nor does Bernoulli's law or Netwon's third. Like the friction under an ice skate, it's a phenomenon we take for granted in science and engineering without a rigorous explanation In physics and engineering, the challenge is to make mathematical models of the world that accurately predict how it works. Once we've done that we can declare success. I agree with Yann that it is not useful to focus on "intuitive" explanations and a fixation on qualitative casual reasoning is largely useless, e.g., very few physicists care about the "why" of quantum theory. But there is a big difference between how we can explain how a plane flies and how we can explain how a neural network works: - The airplane flying doesn't have a "simple" explanation but it can be explained based on fundamental physical principles and mathematics. These models seem to have nearly unlimited ability to extrapolate (within their known bounds of validity). - The neural network giving accurate predictions can only (for the most part) be explained empirically. The models fail to extrapolate and worse, most of the time we can't even trust the model to know when it's extrapolating. > But you can, and not just conceivably, but practically train a neural net to approximate navier Stokes to a high degree of accuracy with a fraction of the computation of a 3D finite model. This is a lovely idea, but I as someone currently doing research in this space (deep learning for CFD) I don't think its true. There are loads of neural nets that can accurately approximate Navier-Stokes within some limited domain of applicability (e.g., for flow over a parameterized set of airfoils), but so far nothing is close to matching the generalization power of Navier-Stokes.

ericjang 7y ago

Yann was not making a statement about deep nets' ability (or lack thereof) to fly a plane in a way that matches expert design. He's making the point that the ML field's obsession with causal inference (and causal discovery) is overrated, precisely because our gold standards of interpretable, safety-critical systems (airplane flight) are based on Navier Stokes/CFD. Planes were made to fly based on empirical validation of these models, long before we gained a more detailed understanding of how causality (the equations and models themselves are time reversible, implying that they contain imperfect knowledge of causality)

ssivark 7y ago
> Planes were made to fly based on empirical validation of these models, long before we gained a more detailed understanding of how causality And their success is so repeatable that if they fail once we make a Really Big Deal out of it. I don’t think any ML model is close to that level of engineering rigor, let alone deep learning. Moving on to aerodynamics, we have a pretty good causal model and can simulate the system given a pattern of boundary conditions. Further, some people (who have studied aero/CFD) can even intuitively predict approximately what happens (otherwise we would have a hard time designing planes!). It just so happens that it’s not as simple as high school physics, and cannot be compressed in to perfect+simple rules (trade off between those two). Speaking in the context of time reversibility, you are being fast and loose, and using the word “causality” in a sense that is irrelevant to the rest of your comment.

allovernow 7y ago

>This is argument is complete nonsense. Navier Stokes has a rigorous derivation based on extremely high-fidelity assumptions, such as conservation of mass, momentum, and energy. We understand these assumptions, and we understand regimes in which using N-S would result in catastrophe (such as rarefied gases, relativistic velocities, etc.) I think you missed the point. Navier-Stokes does not explain lift, nor does Bernoulli's law or Netwon's third. Like the friction under an ice skate, it's a phenomenon we take for granted in science and engineering without a rigorous explanation >Neural networks require data. The Navier stokes does not need to be 'trained' But you can, and not just conceivably, but practically train a neural net to approximate navier Stokes to a high degree of accuracy with a fraction of the computation of a 3D finite model. Yeah, it will functionally be a black box, but LeCunn seems to be arguing that it's opacity doesn't make it any less valid than our opaque intuition of flight which does not stop us from flying thousands of jets every day.

shoyer 7y ago
> I think you missed the point. Navier-Stokes does not explain Lift, nor does Bernoulli's law or Netwon's third. Like the friction under an ice skate, it's a phenomenon we take for granted in science and engineering without a rigorous explanation In physics and engineering, the challenge is to make mathematical models of the world that accurately predict how it works. Once we've done that we can declare success. I agree with Yann that it is not useful to focus on "intuitive" explanations and a fixation on qualitative casual reasoning is largely useless, e.g., very few physicists care about the "why" of quantum theory. But there is a big difference between how we can explain how a plane flies and how we can explain how a neural network works: - The airplane flying doesn't have a "simple" explanation but it can be explained based on fundamental physical principles and mathematics. These models seem to have nearly unlimited ability to extrapolate (within their known bounds of validity). - The neural network giving accurate predictions can only (for the most part) be explained empirically. The models fail to extrapolate and worse, most of the time we can't even trust the model to know when it's extrapolating. > But you can, and not just conceivably, but practically train a neural net to approximate navier Stokes to a high degree of accuracy with a fraction of the computation of a 3D finite model. This is a lovely idea, but I as someone currently doing research in this space (deep learning for CFD) I don't think its true. There are loads of neural nets that can accurately approximate Navier-Stokes within some limited domain of applicability (e.g., for flow over a parameterized set of airfoils), but so far nothing is close to matching the generalization power of Navier-Stokes.