3 ms·
The point of those rhetorical questions is that HMC is _not_ just about adding augmenting the system with additional parameters and following the gradients of t
by betanalpha 10y ago
The point of those rhetorical questions is that HMC is _not_ just about adding augmenting the system with additional parameters and following the gradients of the larger system.
Even in your marble system, for example, the gradient of the plane does not affect the position of the marble directly. The gradient affects the velocity which then moves the marble. If the interactions aren't just right then you no longer have a physical system with energy conservation and all the other nice properties one might expect.
In both the physical system of the marble and the pseudo-physical system created by HMC there is a deeper mathematical structure that relates gradients to motion to probability in exactly the right way. If that structure is compromised, say in a naive implementation of HMC, then all of the magic is lost and you don't get the performance that you might expect.
It may sound like I'm being pedantic here, but it's only be understanding and respecting the underlying math far beneath the "hockey puck" description that we've been able to take HMC to the next level, especially in tools like Stan, with faster and more robust performance as well as sensitive diagnostics of failures.
- soVeryTired 10y agoYep, I can absolutely agree with that. It's not just the intuition that matters, the deeper details are important too.
- betanalpha 10y ago+1. As someone who has struggled to understand math for a long time, I have come to appreciate that it's not about finding any "intuitive" description of a system but rather the one that's compatible with the underlying mathematics.