6 ms·

I don't think this is right - small deviations from the true gradient will result in large deviations along the Hamiltonian path. The magic trick of HMC is that

I don't think this is right - small deviations from the true gradient will result in large deviations along the Hamiltonian path. The magic trick of HMC is that you can safely make large jumps in state space because you are following the right path - doesn't feel safe to break this assumption. But I'm not an expert on HMC. My intuition comes mainly from my past career as a physicist. So if someone knows why it would be safe I'd love to hear.

imurray 10y ago
Implementations of HMC are approximating ideal Hamiltonian dynamics by discretizing time. The "symplectic" integrators that are used have the property that they correspond to simulating continuous-time Hamiltonian dynamics exactly on some other energy surface, which is not too far from the one you had in mind. So, in some sense, an approximation like the grandparent had in mind is the usual case. You are right that a bad approximation of the surface could lead to proposals that get rejected a lot. But at least the dynamics are likely to be stable. It could be worth trying.
contravariant 10y ago
It will result in changes in total 'energy', but that's fine if you account for this in the acceptance probability.
betanalpha 10y ago
This intuition is correct -- symplectic integrators admit highly accurately energy conservation only when you have access to the exact gradients. When the gradients are inexact then you get bias in the integrator that pulls you away from the target energy level set and decreases the acceptance probabilities (or rather the transition probabilities away from the initial point if you using a more sophisticated scheme as advocated in the paper). Note that symplectic integrator trajectories stay near the target energy level set but they need not stay near the true trajectory itself. In general the two will diverge pretty quickly (a manifestation of Hamiltonian chaos) within the level sets, but that's not a problem as we don't really care how they explore the level sets, just that they're staying on them.

imurray 10y ago

Implementations of HMC are approximating ideal Hamiltonian dynamics by discretizing time. The "symplectic" integrators that are used have the property that they correspond to simulating continuous-time Hamiltonian dynamics exactly on some other energy surface, which is not too far from the one you had in mind. So, in some sense, an approximation like the grandparent had in mind is the usual case. You are right that a bad approximation of the surface could lead to proposals that get rejected a lot. But at least the dynamics are likely to be stable. It could be worth trying.

contravariant 10y ago

It will result in changes in total 'energy', but that's fine if you account for this in the acceptance probability.

betanalpha 10y ago

This intuition is correct -- symplectic integrators admit highly accurately energy conservation only when you have access to the exact gradients. When the gradients are inexact then you get bias in the integrator that pulls you away from the target energy level set and decreases the acceptance probabilities (or rather the transition probabilities away from the initial point if you using a more sophisticated scheme as advocated in the paper). Note that symplectic integrator trajectories stay near the target energy level set but they need not stay near the true trajectory itself. In general the two will diverge pretty quickly (a manifestation of Hamiltonian chaos) within the level sets, but that's not a problem as we don't really care how they explore the level sets, just that they're staying on them.