4 ms·

The reason Stan, Nimble, PyMC fit via simulation is because joint distributions generally don’t have an analytical form and require global fitting methods to id

The reason Stan, Nimble, PyMC fit via simulation is because joint distributions generally don’t have an analytical form and require global fitting methods to identify. I’d hazard to guess that if this package fits incrementally without simulation that either it doesn’t aim for a global optimum or that it places substantial limits on the form models can take.

mjhay 3y ago
Belief propagagation (i.e. message passing) has a long history and has been pretty successful. It's not always applicable, but it can have a lot of advantages when it is.
ckrapu 3y ago
I would use one of those for scenarios when I need proper measures of uncertainty like credible regions. The methods listed in this project (EM algorithm, VI) tend not to provide this as well but usually produce something that looks like a good posterior mean (global optimum under mean square loss) in my experience.
bvdmitri 3y ago
There inevitable drawbacks of both approaches. Nothing particularly wrong with the simulation, but it is really really slow and is not applicable in real-time or close to real-time applications, where situation may change rapidly. Also it usually requires high-end CPUs, which are not available on edge devices like drones. RxInfer tries to be as efficient as possible in running inference, but it does indeed places limits on the form of the models it can run inference for. The team has long-term plans and solutions under research, which should expand the set of available models substantially. Message passing has a long history, it is not only belief propagation on simple models, but also nowadays the variational inference is possible with message passing on sophisticated hierarchical models.
- tnecniv 3y ago
  Moreover, a lot of those methods are hard to parallelize. I’m not up on state of the art HMC for example, but the standard formulation isn’t easy to parallelize beyond the basic linear algebra operations.

mjhay 3y ago

Belief propagagation (i.e. message passing) has a long history and has been pretty successful. It's not always applicable, but it can have a lot of advantages when it is.

ckrapu 3y ago

I would use one of those for scenarios when I need proper measures of uncertainty like credible regions. The methods listed in this project (EM algorithm, VI) tend not to provide this as well but usually produce something that looks like a good posterior mean (global optimum under mean square loss) in my experience.

bvdmitri 3y ago

There inevitable drawbacks of both approaches. Nothing particularly wrong with the simulation, but it is really really slow and is not applicable in real-time or close to real-time applications, where situation may change rapidly. Also it usually requires high-end CPUs, which are not available on edge devices like drones. RxInfer tries to be as efficient as possible in running inference, but it does indeed places limits on the form of the models it can run inference for. The team has long-term plans and solutions under research, which should expand the set of available models substantially. Message passing has a long history, it is not only belief propagation on simple models, but also nowadays the variational inference is possible with message passing on sophisticated hierarchical models.

tnecniv 3y ago
Moreover, a lot of those methods are hard to parallelize. I’m not up on state of the art HMC for example, but the standard formulation isn’t easy to parallelize beyond the basic linear algebra operations.