betanalpha

3 ms·

1.
by betanalpha 10y ago
+1. As someone who has struggled to understand math for a long time, I have come to appreciate that it's not about finding any "intuitive" description of a system but rather the one that's compatible with the underlying
2.
by betanalpha 10y ago
May I suggest that you try sampling from a high-dimensional distribution and see how many samples end up near the mode? For example, try a 50-dimensional IID unit gaussian and check how often r = sqrt(x_1^2 + ... x_50^2) < 0.25 * sqrt(D
3.
by betanalpha 10y ago
"No, one evaluation near the mode is more efficient than one evaluation at the boundary (because the density is higher)." Incorrect in general. Firstly, one evaluation anywhere does not yield any reasonably accurate estimate of e
4.
by betanalpha 10y ago
Yes, "a whole bunch" is not a technical term. :-p You are correct that including the entire convex hull would not itself be absurdly costly. A naive search (say grid search) would still spend most of its time evaluating points at
5.
by betanalpha 10y ago
The point of those rhetorical questions is that HMC is _not_ just about adding augmenting the system with additional parameters and following the gradients of the larger system. Even in your marble system, for example, the gradient of the p
6.
by betanalpha 10y ago
Because probability has to sum to one. The higher you go in dimension that unit probability is distributed across more and more "boxes" until any single box, including the one around the mode becomes irrelevant. When you are co
7.
by betanalpha 10y ago
So V(x) = exp(-pi(x)) and you use HMC to explore those potential energy level sets? But what happens when you change coordinates and get a different density, pi(x), a different potential energy, V(x), and hence different level sets? And
8.
by betanalpha 10y ago
There are also talks focusing on HMC available online, the most recent of which are https://www.youtube.com/watch?v=VnNdhsm0rJQ (more introductory) and https://icerm.brown.edu/video_archive/#/play&
9.
by betanalpha 10y ago
This intuition is correct -- symplectic integrators admit highly accurately energy conservation only when you have access to the exact gradients. When the gradients are inexact then you get bias in the integrator that pulls you away from t
10.
by betanalpha 10y ago
"And despite what the autodiff people say, that might be hard and/or not very useful in practice, if your likelihood is based on running a lengthy astrophysical simulation." Only if you treat the simulation as an un-interroga