> K-L divergence is the pretty much really dumb, obvious thing you would probably think up if someone asked you to compare two probability distributions. I don

> K-L divergence is the pretty much really dumb, obvious thing you would probably think up if someone asked you to compare two probability distributions. I don't think if anyone was asked simply to "compare two probability distributions", they'd come up with something asymmetric like the KL divergence. You'd need to at least add that one of the distributions is the "real" distribution.