That's not because you need little background in information theory. That's because KL-divergences are such a universal info-theoretic quantity that if you dee

That's not because you need little background in information theory. That's because KL-divergences are such a universal info-theoretic quantity that if you deeply understand them, you understand much to most of information theory. This is like saying, "You don't need to really know calculus, just integrals."