4 ms·

If I understood correctly, on page 2 they say they generated their training and test cases using random initial positions and velocities for the 3-body problem.

If I understood correctly, on page 2 they say they generated their training and test cases using random initial positions and velocities for the 3-body problem. But this is exactly the sort of case that a non-sophisticated solver should be able to handle easily as well. Brutus is special in the sense that it tries hard to give some sort of a guarantee about the accuracy of its solution to a chaotic problem. But the neural network solver provides no such guarantee, and has not been trained or evaluated on "hard" examples. I'm not sure the comparison is fair or particularly meaningful. I wonder if exactly the same sort of "positive" result could be achieved with a standard symplectic integrator, it's not at all obvious from the paper that they have ruled that out.

pixelpoet 7y ago
Argh, just looked up Brutus and it's pretty much exactly what I wanted to do on my own after quite a lot of reading on n-body simulations and finding nothing like it out there. I was (very naively, I realise) hoping to do some original research. Oh well, time to study this intensively. Here's a link for the curious: https://arxiv.org/abs/1411.6671 https://arxiv.org/abs/1411.6671
- aaronblohowiak 7y ago
  That’s total validation of your thinking!!!
  - jszymborski 7y ago
    You know, as a PhD student, I sometimes feel like the moment I come up with a good idea, it's guaranteed to show up as a result on Google Scholar the next day, which can be rather discouraging. I've never really thought about it in terms of your comment, however, which is actually very motivating. Thanks for the perspective random internet person. (Apologies for the OT comment)
    - mehrdadn 7y ago
      Reminds me of when I asked a professor once "why don't they try <alternative approach> instead?". He replied, "I tried that when I was a grad student. It turns out not to work that well because <reasons>." I never figured out if that validated or invalidated my thinking, but I found it funny and kind of eye-opening either way.
      - dwaltrip 7y ago
        It is validation that you had an interesting idea that others thought was worth checking out.
      - madhadron 7y ago
        A lot of the value of being in a department is being able to have these shortcuts. I remember as an undergrad in physics asking why we didn't use imaginary time to formulate special relativity? The grad students around me by and large looked puzzled and said they had never thought of it. A very, very experienced nuclear/mathematical physicist said, "Oh, yeah, folks did try that back in the early 20th century. It works fine, but you have to go to a full metric for general relativity, so why carry around two formalisms?" I contemplated that for about thirty seconds, said, "That makes sense," and happily put aside the question. It vindicates your thinking in that it indicates that you are picking up the techniques that professionals use to generate ideas. You just have to accept that as you're learning, most of the ideas will have been generated and addressed. What gets awkward is when you're doing interdisciplinary research and you're using the generative mechanisms from one discipline in the other. By and large both sides stare at you in confusion, or it requires multiple hour conversations to lead the other discipline's practitioners into a mental space where they can say, "Oh, yeah, that wouldn't be useful because of X." It makes the process of tuning your idea generation much slower. On the flip side, you hit the unknown way faster, sometimes great vistas of unknown where everyone goes, "Umm...wow. Yeah. Never heard of that continent. Let us know what you find."
    - orbifold 7y ago
      I think even a PhD is more about execution than good ideas. There is generally speaking no shortage of good ideas, but many turn out to be too labour intensive or have other subtle problems. In any case I’ve also gotten discouraged by this, but what eventually matters is really execution on one or two good ideas during your PhD
- YeGoblynQueenne 7y ago
  The 3-body problem is far from solved from what I understand. There's plenty of scope for original research.
knzhou 7y ago
Yeah, this is classic neural network hype. I wish there were a way to automatically copy-paste my usual comment on trusting the MIT Tech Review, and especially its Emerging Technologies column, whenever such a link is submitted.
- parineum 7y ago
  You could train a neural net to do it.
  - grepthisab 7y ago
    No, that would be classic neural network hype. I wish there were a way to automatically copy-paste knzhou's usual comment on trusting the MIT Tech Review, and especially its Emerging Technologies column, whenever such a link is submitted
    - knzhou 7y ago
      You could train a knzhou to do it.
      - snak 7y ago
        Yeah, this is classic neural network hype. I wish there were a way to automatically copy-paste my usual comment on trusting the MIT Tech Review, and especially its Emerging Technologies column, whenever such a link is submitted.
      - deleted 7y ago
        [deleted]
shoyer 7y ago
Agreed, this paper did not show any meaningful generalization and is sorely short on baselines. If you wanted to train a neural network model that would generalize, you would need build in physical constraints into the model architecture. For example, dynamics are governed by a Hamiltonian based on pairwise interactions, which should be integrated as an ODE. A nice recent example of this from folks at DeepMind is this paper “Hamiltonian Graph Networks with ODE Integrators”: https://arxiv.org/abs/1909.12790 https://arxiv.org/abs/1909.12790. That said, if you go down this road too far you may find that you are just learning an approximation to Newton’s law of gravity, which we already know exactly! The interesting work in this space is finding appropriate interpolation problems inside the context of state of the art physics based models. In any complex simulation, there are tunable parameters and approximations that are suitable to replace with ML. If I were working on this problem, I would start by making a differentiable version of Brutus, and try to deeply understand its strengths and weaknesses. Neural nets are just high dimensional interpolation. The only reason why they are more powerful than “classical” ML is that you can fit them when embedded in an arbitrary computational graph, in which it is easy to embed prior knowledge (“differentiable programming” style). If you’re not doing that, you might as well just stick with the blackbox algorithms of Scikit-Learn.
YeGoblynQueenne 7y ago
Also, they split their data to 9900 training and 100 validation simulations. They didn't do cross-validation which makes sense given the cost but ultimately their setup means that they tested their models on 100 examples out of 1000 they generated and out of who knows how many in nature. That's not very promising. In terms of practical applications I don't see what's the point of training a network on simulations produced by deterministic systems. But I guess they didn't really mean their technique as a replacement for deterministic systems because they say they envision it as a sort of crutch were Brutus falters.
deleted 7y ago
[deleted]
jedbrown 7y ago
The initial conditions are all stationary, and it's in the plane exploiting symmetry, so they are just 2 real numbers. They don't evolve the simulation very long and the errors rapidly increase with time. See Figure 3 for typical errors of about 20% on the validation set with clear signs of overfitting. https://arxiv.org/pdf/1910.07291.pdf https://arxiv.org/pdf/1910.07291.pdf

pixelpoet 7y ago

Argh, just looked up Brutus and it's pretty much exactly what I wanted to do on my own after quite a lot of reading on n-body simulations and finding nothing like it out there. I was (very naively, I realise) hoping to do some original research. Oh well, time to study this intensively. Here's a link for the curious: https://arxiv.org/abs/1411.6671 https://arxiv.org/abs/1411.6671

aaronblohowiak 7y ago
That’s total validation of your thinking!!!
- jszymborski 7y ago
  You know, as a PhD student, I sometimes feel like the moment I come up with a good idea, it's guaranteed to show up as a result on Google Scholar the next day, which can be rather discouraging. I've never really thought about it in terms of your comment, however, which is actually very motivating. Thanks for the perspective random internet person. (Apologies for the OT comment)
  - mehrdadn 7y ago
    Reminds me of when I asked a professor once "why don't they try <alternative approach> instead?". He replied, "I tried that when I was a grad student. It turns out not to work that well because <reasons>." I never figured out if that validated or invalidated my thinking, but I found it funny and kind of eye-opening either way.
    - dwaltrip 7y ago
      It is validation that you had an interesting idea that others thought was worth checking out.
    - madhadron 7y ago
      A lot of the value of being in a department is being able to have these shortcuts. I remember as an undergrad in physics asking why we didn't use imaginary time to formulate special relativity? The grad students around me by and large looked puzzled and said they had never thought of it. A very, very experienced nuclear/mathematical physicist said, "Oh, yeah, folks did try that back in the early 20th century. It works fine, but you have to go to a full metric for general relativity, so why carry around two formalisms?" I contemplated that for about thirty seconds, said, "That makes sense," and happily put aside the question. It vindicates your thinking in that it indicates that you are picking up the techniques that professionals use to generate ideas. You just have to accept that as you're learning, most of the ideas will have been generated and addressed. What gets awkward is when you're doing interdisciplinary research and you're using the generative mechanisms from one discipline in the other. By and large both sides stare at you in confusion, or it requires multiple hour conversations to lead the other discipline's practitioners into a mental space where they can say, "Oh, yeah, that wouldn't be useful because of X." It makes the process of tuning your idea generation much slower. On the flip side, you hit the unknown way faster, sometimes great vistas of unknown where everyone goes, "Umm...wow. Yeah. Never heard of that continent. Let us know what you find."
  - orbifold 7y ago
    I think even a PhD is more about execution than good ideas. There is generally speaking no shortage of good ideas, but many turn out to be too labour intensive or have other subtle problems. In any case I’ve also gotten discouraged by this, but what eventually matters is really execution on one or two good ideas during your PhD
YeGoblynQueenne 7y ago
The 3-body problem is far from solved from what I understand. There's plenty of scope for original research.

knzhou 7y ago

Yeah, this is classic neural network hype. I wish there were a way to automatically copy-paste my usual comment on trusting the MIT Tech Review, and especially its Emerging Technologies column, whenever such a link is submitted.

parineum 7y ago
You could train a neural net to do it.
- grepthisab 7y ago
  No, that would be classic neural network hype. I wish there were a way to automatically copy-paste knzhou's usual comment on trusting the MIT Tech Review, and especially its Emerging Technologies column, whenever such a link is submitted
  - knzhou 7y ago
    You could train a knzhou to do it.
    - snak 7y ago
      Yeah, this is classic neural network hype. I wish there were a way to automatically copy-paste my usual comment on trusting the MIT Tech Review, and especially its Emerging Technologies column, whenever such a link is submitted.
    - deleted 7y ago
      [deleted]

shoyer 7y ago

Agreed, this paper did not show any meaningful generalization and is sorely short on baselines. If you wanted to train a neural network model that would generalize, you would need build in physical constraints into the model architecture. For example, dynamics are governed by a Hamiltonian based on pairwise interactions, which should be integrated as an ODE. A nice recent example of this from folks at DeepMind is this paper “Hamiltonian Graph Networks with ODE Integrators”: https://arxiv.org/abs/1909.12790 https://arxiv.org/abs/1909.12790. That said, if you go down this road too far you may find that you are just learning an approximation to Newton’s law of gravity, which we already know exactly! The interesting work in this space is finding appropriate interpolation problems inside the context of state of the art physics based models. In any complex simulation, there are tunable parameters and approximations that are suitable to replace with ML. If I were working on this problem, I would start by making a differentiable version of Brutus, and try to deeply understand its strengths and weaknesses. Neural nets are just high dimensional interpolation. The only reason why they are more powerful than “classical” ML is that you can fit them when embedded in an arbitrary computational graph, in which it is easy to embed prior knowledge (“differentiable programming” style). If you’re not doing that, you might as well just stick with the blackbox algorithms of Scikit-Learn.

YeGoblynQueenne 7y ago

Also, they split their data to 9900 training and 100 validation simulations. They didn't do cross-validation which makes sense given the cost but ultimately their setup means that they tested their models on 100 examples out of 1000 they generated and out of who knows how many in nature. That's not very promising. In terms of practical applications I don't see what's the point of training a network on simulations produced by deterministic systems. But I guess they didn't really mean their technique as a replacement for deterministic systems because they say they envision it as a sort of crutch were Brutus falters.

deleted 7y ago

[deleted]

jedbrown 7y ago

The initial conditions are all stationary, and it's in the plane exploiting symmetry, so they are just 2 real numbers. They don't evolve the simulation very long and the errors rapidly increase with time. See Figure 3 for typical errors of about 20% on the validation set with clear signs of overfitting. https://arxiv.org/pdf/1910.07291.pdf https://arxiv.org/pdf/1910.07291.pdf