5 ms·

A Twisted Path to Equation-Free Prediction

34 points by bpolania 11y ago

mockery 11y ago
This extended version of the associated video provides much more information / better intuition about how delay embedding works: https://www.youtube.com/watch?v=6i57udsPKms https://www.youtube.com/watch?v=6i57udsPKms
- contravariant 11y ago
  Thanks, the video in the article cut off before it made any interesting points.
tansey 11y ago
Actual paper: http://www.pnas.org/content/112/13/E1569.full http://www.pnas.org/content/112/13/E1569.full The idea (from 5 minutes of skimming plus watching the little 3min tutorial) seems to be that any multi-dimensional time series implicitly contains all the information necessary to reconstruct it entirely within a single dimension. The authors show an example of a 3-dimensional time series with a butterfly-type pattern that you can reconstruct just by looking at the values of X(t) and then using X(t-a) and X(t-2a) as the other dimensions. Not sure how that leads to prediction-- can someone explain that part?
- powera 11y ago
  In complex analysis, this is generally true; in particular for any function on the real numbers, there is exactly one differentiable function on the entire complex plane that has the same value. https://en.wikipedia.org/wiki/Holomorphic_function https://en.wikipedia.org/wiki/Holomorphic_function . You can also go forward/backward in time with the functions, and knowing the value ahead of time is predicting it. On the other hand, there are non-differentiable events all the time in nature. It's madness to claim that salmon populations contain enough information to predict earthquakes (that could dam a river and cause a massive population change).
  - kragen 11y ago
    It's probably more accurate to say that predicting earthquakes from salmon populations is far outside our current modeling capabilities. Undoubtedly the seismic stresses and crustal movements leading up to earthquakes have numerous weak causal links to the salmon population: most obviously, they alter the slopes and flow rates of streams leading into the ocean, altering the nutrient balance available to the algae there, but also they alter the distances flown by migratory birds by centimeters per year, and those birds will consequently eat very slightly different fish, etc. There might not even be a way to analyze the salmon population in such a way as to tell you how far the continents have drifted — but declaring that postulating the existence of such a thing is "madness" seems like being far too sure of yourself. There's a science-fiction story that may be useful in gaining perspective on the limits of prediction at http://lesswrong.com/lw/qk/that_alien_message/ http://lesswrong.com/lw/qk/that_alien_message/. The problem with earthquakes, as I see it, is not so much that they are non-differentiable as that they (like a bird eating or not eating a particular fish) have very large local derivatives with respect to their precipitating conditions.
    - powera 11y ago
      If you have a theory that invalidates the Heisenberg Uncertainty Principle, go for it. Otherwise, I'm going to consider all of that pure science fiction.
      - kragen 11y ago
        Maybe you're not very familiar with the uncertainty principle, but continental plates and even individual birds have enough mass that the uncertainty in their position, for any reasonably large uncertainty in their momentum, is insignificant.
        powera 11y ago
        I'm claiming that to measure the salmon population well enough to predict earthquakes, you would have to violate the uncertainty principle.
- cafebeen 11y ago
  It's worth noting that this doesn't work for any multi-dimensional time series, but only some. For example, the lat-long-elevation plot of an airplane's flight path could not be recovered from just the latitude plot, or one channel could simply be noise. Although, it's incredibly interesting to find cases where this does work, as it indicates there's some special underlying manifold structure.
- contravariant 11y ago
  A good explanation can be found in the video mockery linked below.[1] Short summary: There's a theorem that says that (under certain conditions) if you have some model that evolves over time, then you can reconstruct the geometric properties of this model by only looking at how a single 'observable' (e.g. the number of salmon) evolves over time. Of course if you have multiple observables then you get multiple reconstructions and with enough data you can figure out how to go from one to the other. Now how this leads to prediction is that you can use any one of the observables you can figure out the current state of the model, which tells you both the value of the observables you didn't know and how it's going to evolve. Of course the accuracy depends on the accuracy of the reconstruction. [1]:https://news.ycombinator.com/item?id=10405320 https://news.ycombinator.com/item?id=10405320
scottfr 11y ago
When reading about any predictive technique the first thing to focus on before trying to understand the predictive mechanism is to understand the predictive accuracy of the model. A paper like this should have extensive discussions of overfitting, underfitting and overall predictive accuracy. The entirety of this paper's attempt to address this is: "Last, to avoid arbitrary fitting and to obtain a robust measure of forecast skill, we apply a fourfold cross-validation scheme for each model: the model is fit to three-fourths of the data to predict the remaining one-fourth out-of-sample, and the procedure is repeated for each one-fourth segment of the time series." They are dealing with time series data, which naturally has temporally correlated observations so a cross-validation like this will give a biased error estimate that will favor the more complex model (it encourages overfitting). I haven't delved into the details, but compared to the Ricker model their model is likely more complex so I have very little confidence in their claims of accuracy. Of course, the choice of the Ricker model (developed in the 1950's) is really a strawman in any case. There are much better competitive techniques they could have chosen to benchmark their work.
jostmey 11y ago
The name of the method seems a little sensational. Many scientists have become hung up on specific models when actually all they care about are making predictions from their data. This paper is reaction against that trend, but I have to wonder if a boiler plate machine learning algorithm wouldn't perform at least as well.
- IndianAstronaut 11y ago
  > This paper is reaction against that trend, but I have to wonder if a boiler plate machine learning algorithm wouldn't perform at least as well. The difference is simulations. Equation modeling lets us simulate and ask the what if questions.
j2kun 11y ago
There's a fine line between writing down a single equation that you claim governs an entire system and using data to come up with some equations which (based on the data) you claim describes the system. Actually I take it back. There is no line. They are the same thing. It's just that one is based on very small amounts of data, and the other has more complicated-looking equations. It's all still math, folks. > Complex natural systems defy standard mathematical analysis So the punchline is that ecologists and the author of this article have a very narrow view of what "standard" mathematical analysis entails. It's great that ecology is making strides forward, but this isn't a groundbreaking departure, it's just them catching up to modern mathematical analysis.
- cafebeen 11y ago
  Yeah, I think it's a parametric vs. non-parametric modeling comparison they're making, although they don't explicitly make that point. It does seem strange to call what they're doing "equation-free", but maybe within their field this kind of thing makes sense

34 points by bpolania 11y ago

This extended version of the associated video provides much more information / better intuition about how delay embedding works: https://www.youtube.com/watch?v=6i57udsPKms https://www.youtube.com/watch?v=6i57udsPKms

contravariant 11y ago
Thanks, the video in the article cut off before it made any interesting points.

tansey 11y ago

Actual paper: http://www.pnas.org/content/112/13/E1569.full http://www.pnas.org/content/112/13/E1569.full The idea (from 5 minutes of skimming plus watching the little 3min tutorial) seems to be that any multi-dimensional time series implicitly contains all the information necessary to reconstruct it entirely within a single dimension. The authors show an example of a 3-dimensional time series with a butterfly-type pattern that you can reconstruct just by looking at the values of X(t) and then using X(t-a) and X(t-2a) as the other dimensions. Not sure how that leads to prediction-- can someone explain that part?

powera 11y ago
In complex analysis, this is generally true; in particular for any function on the real numbers, there is exactly one differentiable function on the entire complex plane that has the same value. https://en.wikipedia.org/wiki/Holomorphic_function https://en.wikipedia.org/wiki/Holomorphic_function . You can also go forward/backward in time with the functions, and knowing the value ahead of time is predicting it. On the other hand, there are non-differentiable events all the time in nature. It's madness to claim that salmon populations contain enough information to predict earthquakes (that could dam a river and cause a massive population change).
- kragen 11y ago
  It's probably more accurate to say that predicting earthquakes from salmon populations is far outside our current modeling capabilities. Undoubtedly the seismic stresses and crustal movements leading up to earthquakes have numerous weak causal links to the salmon population: most obviously, they alter the slopes and flow rates of streams leading into the ocean, altering the nutrient balance available to the algae there, but also they alter the distances flown by migratory birds by centimeters per year, and those birds will consequently eat very slightly different fish, etc. There might not even be a way to analyze the salmon population in such a way as to tell you how far the continents have drifted — but declaring that postulating the existence of such a thing is "madness" seems like being far too sure of yourself. There's a science-fiction story that may be useful in gaining perspective on the limits of prediction at http://lesswrong.com/lw/qk/that_alien_message/ http://lesswrong.com/lw/qk/that_alien_message/. The problem with earthquakes, as I see it, is not so much that they are non-differentiable as that they (like a bird eating or not eating a particular fish) have very large local derivatives with respect to their precipitating conditions.
  - powera 11y ago
    If you have a theory that invalidates the Heisenberg Uncertainty Principle, go for it. Otherwise, I'm going to consider all of that pure science fiction.
    - kragen 11y ago
      Maybe you're not very familiar with the uncertainty principle, but continental plates and even individual birds have enough mass that the uncertainty in their position, for any reasonably large uncertainty in their momentum, is insignificant.
      - powera 11y ago
        I'm claiming that to measure the salmon population well enough to predict earthquakes, you would have to violate the uncertainty principle.
cafebeen 11y ago
It's worth noting that this doesn't work for any multi-dimensional time series, but only some. For example, the lat-long-elevation plot of an airplane's flight path could not be recovered from just the latitude plot, or one channel could simply be noise. Although, it's incredibly interesting to find cases where this does work, as it indicates there's some special underlying manifold structure.
contravariant 11y ago
A good explanation can be found in the video mockery linked below.[1] Short summary: There's a theorem that says that (under certain conditions) if you have some model that evolves over time, then you can reconstruct the geometric properties of this model by only looking at how a single 'observable' (e.g. the number of salmon) evolves over time. Of course if you have multiple observables then you get multiple reconstructions and with enough data you can figure out how to go from one to the other. Now how this leads to prediction is that you can use any one of the observables you can figure out the current state of the model, which tells you both the value of the observables you didn't know and how it's going to evolve. Of course the accuracy depends on the accuracy of the reconstruction. [1]:https://news.ycombinator.com/item?id=10405320 https://news.ycombinator.com/item?id=10405320

scottfr 11y ago

When reading about any predictive technique the first thing to focus on before trying to understand the predictive mechanism is to understand the predictive accuracy of the model. A paper like this should have extensive discussions of overfitting, underfitting and overall predictive accuracy. The entirety of this paper's attempt to address this is: "Last, to avoid arbitrary fitting and to obtain a robust measure of forecast skill, we apply a fourfold cross-validation scheme for each model: the model is fit to three-fourths of the data to predict the remaining one-fourth out-of-sample, and the procedure is repeated for each one-fourth segment of the time series." They are dealing with time series data, which naturally has temporally correlated observations so a cross-validation like this will give a biased error estimate that will favor the more complex model (it encourages overfitting). I haven't delved into the details, but compared to the Ricker model their model is likely more complex so I have very little confidence in their claims of accuracy. Of course, the choice of the Ricker model (developed in the 1950's) is really a strawman in any case. There are much better competitive techniques they could have chosen to benchmark their work.

jostmey 11y ago

The name of the method seems a little sensational. Many scientists have become hung up on specific models when actually all they care about are making predictions from their data. This paper is reaction against that trend, but I have to wonder if a boiler plate machine learning algorithm wouldn't perform at least as well.

IndianAstronaut 11y ago
> This paper is reaction against that trend, but I have to wonder if a boiler plate machine learning algorithm wouldn't perform at least as well. The difference is simulations. Equation modeling lets us simulate and ask the what if questions.

j2kun 11y ago

There's a fine line between writing down a single equation that you claim governs an entire system and using data to come up with some equations which (based on the data) you claim describes the system. Actually I take it back. There is no line. They are the same thing. It's just that one is based on very small amounts of data, and the other has more complicated-looking equations. It's all still math, folks. > Complex natural systems defy standard mathematical analysis So the punchline is that ecologists and the author of this article have a very narrow view of what "standard" mathematical analysis entails. It's great that ecology is making strides forward, but this isn't a groundbreaking departure, it's just them catching up to modern mathematical analysis.

cafebeen 11y ago
Yeah, I think it's a parametric vs. non-parametric modeling comparison they're making, although they don't explicitly make that point. It does seem strange to call what they're doing "equation-free", but maybe within their field this kind of thing makes sense