This intuition is correct -- symplectic integrators admit highly accurately energy conservation only when you have access to the exact gradients. When the grad

This intuition is correct -- symplectic integrators admit highly accurately energy conservation only when you have access to the exact gradients. When the gradients are inexact then you get bias in the integrator that pulls you away from the target energy level set and decreases the acceptance probabilities (or rather the transition probabilities away from the initial point if you using a more sophisticated scheme as advocated in the paper). Note that symplectic integrator trajectories stay near the target energy level set but they need not stay near the true trajectory itself. In general the two will diverge pretty quickly (a manifestation of Hamiltonian chaos) within the level sets, but that's not a problem as we don't really care how they explore the level sets, just that they're staying on them.