Yeah, it's quite disturbing that the differential entropy (unlike the discrete entropy) depends on the units. Even worse, the differential entropy can be negati

Yeah, it's quite disturbing that the differential entropy (unlike the discrete entropy) depends on the units. Even worse, the differential entropy can be negative! Interestingly, the differential KL-divergence (differential cross-entropy - differential entropy) doesn't seem to have any of these problems.