5 ms·
Skilling wrote a nice critique of information geometry that's worth reading: http://djafari.free.fr/MaxEnt2014/papers/Tutorial4_paper.pdf http://djafari.free.f
by jwmerrill 10y ago
Skilling wrote a nice critique of information geometry that's worth reading:
http://djafari.free.fr/MaxEnt2014/papers/Tutorial4_paper.pdf http://djafari.free.fr/MaxEnt2014/papers/Tutorial4_paper.pdf
The main point is that KL-divergence is not a metric, so imagining it as a distance in a space may give you some wrong intuitions. Its matrix of 2nd derivatives, the Fischer Information, works as a local metric, but then many people want to draw global pictures that try to extend this back to a global metric, which doesn't actually work.
- otoburb 10y agoAs a complete layman reading that paper, Skilling's arguments seem quite damning of IG. In particular, Amari's fundamental assumption seems to be refuted: "For inference, the only acceptable value for the Rényi-Tsallis parameter is α = 1, which is the correct information. That negates the generalisation to α != 1 which underlies Amari’s “α-divergences” in information geometry." Have any IG proponents responded to or refuted Skilling's critiques? This is interesting because Shun'ichi Amari is credited amongst others with advancing the field in the 80's[1]. [1] https://en.m.wikipedia.org/wiki/Information_geometry https://en.m.wikipedia.org/wiki/Information_geometry
- deleted 10y ago[deleted]
- jqgatsby 10y agoI've been hoping to find a rebuttal to Skilling's paper from someone who understands information geometry. Anyone here able to provide one? Or are his critiques viewed as legitimate?
- deleted 10y ago[deleted]