3 ms·

Everything you say is true, and that only covers the forward problem: given a multi-level model with stochastic dependencies at each stage (like winds, fragment

by mturmon 12y ago

Everything you say is true, and that only covers the forward problem: given a multi-level model with stochastic dependencies at each stage (like winds, fragmentation, etc., that you list), calculate the overall distribution of the output. At its simplest, the forward problem is like a functional composition: y = F( G( H(x) ) ) where F, G, and H are all noisy, and you want to characterize y. @gliese1337 nearby, in an excellent comment, addresses the more complex inverse problem: when you have outputs (y's above) from such a process, how do you determine the most likely model (F, G, H) that produced it? So for example, if you have a bunch of crater ejecta, what can you tell me about the mass of the incoming objects?

araes 12y ago
Thank you for the clarification, and yes, most of what I gave as an example could be characterized as forward probabilistic methods. - Start with a value + divergence and then propagate that state through your transforms to get a mean + divergent final state. I should note that in the given example the base divergences need to be found through some method, which is usually running a Monte-carlo around some baseline state, and then making the significant assumption that new calculations (with small deltas) follow roughly the same statistical behavior. For the backwards case, that can be a significantly more challenging case, as it basically comes down to performing approximate eigenvalue decomposition. However, there are a lot of SOTA tools like pretty much anything from this list: - http://en.wikipedia.org/wiki/Pattern_recognition#Algorithms http://en.wikipedia.org/wiki/Pattern_recognition#Algorithms IE, the crater example is fairly close to the setup for a Hidden Markov Model, where we get final data, and assume that it is based on a set of hidden states. Or for lighter reading, almost any of the major tools used in the Netflix recommendation contest, which is effectively an enormous version of this problem.
- mturmon 12y ago
  "...the backwards case ... basically comes down to performing approximate eigenvalue decomposition..." Hmm, not in general. (I've been working on various problem-specific versions of the backwards case for most of my career.)
  - psb217 12y ago
    FYI, some folks (see: https://spectrallearning.github.io/icml2014/ https://spectrallearning.github.io/icml2014/) are certainly working hard to apply eigenvalue decompositions (well, more like generalized SVDs) to any inference problems they can get their convexity-loving hands on. Disclaimer: my lab mate has been organizing these workshops for the past couple of years and I'm currently procrastinating from preparing a presentation on this material for a reading group later this week, so I've presently got mixed feelings towards it.

by mturmon 12y ago

araes 12y ago

Thank you for the clarification, and yes, most of what I gave as an example could be characterized as forward probabilistic methods. - Start with a value + divergence and then propagate that state through your transforms to get a mean + divergent final state. I should note that in the given example the base divergences need to be found through some method, which is usually running a Monte-carlo around some baseline state, and then making the significant assumption that new calculations (with small deltas) follow roughly the same statistical behavior. For the backwards case, that can be a significantly more challenging case, as it basically comes down to performing approximate eigenvalue decomposition. However, there are a lot of SOTA tools like pretty much anything from this list: - http://en.wikipedia.org/wiki/Pattern_recognition#Algorithms http://en.wikipedia.org/wiki/Pattern_recognition#Algorithms IE, the crater example is fairly close to the setup for a Hidden Markov Model, where we get final data, and assume that it is based on a set of hidden states. Or for lighter reading, almost any of the major tools used in the Netflix recommendation contest, which is effectively an enormous version of this problem.

mturmon 12y ago
"...the backwards case ... basically comes down to performing approximate eigenvalue decomposition..." Hmm, not in general. (I've been working on various problem-specific versions of the backwards case for most of my career.)
- psb217 12y ago
  FYI, some folks (see: https://spectrallearning.github.io/icml2014/ https://spectrallearning.github.io/icml2014/) are certainly working hard to apply eigenvalue decompositions (well, more like generalized SVDs) to any inference problems they can get their convexity-loving hands on. Disclaimer: my lab mate has been organizing these workshops for the past couple of years and I'm currently procrastinating from preparing a presentation on this material for a reading group later this week, so I've presently got mixed feelings towards it.