4 ms·

Shannon info and KC are asymptotically equivalent.

• chriswarbo 6y ago
  That's why I used 'scare quotes'. It's very similar (perhaps provably equivalent, I'm not sure) to how Frequentist and Bayesian probabilities are asymptotically equivalent; yet offer their own perspective on few/one-shot cases. In my experience, general practice w.r.t. Shannon information is to manually pick a model with some numerical parameters (Huffman tree, deformation-axes-NN, etc.), then set those parameters to work well on an example population. General practice w.r.t. algorithmic information seems to involve searching program space for a good model. The adjustable part is the language, which can certainly be manually tailored to the application, but doesn't tend to be the bulk of the work (e.g. automatically searching for an efficient language tends to be redundant once we're doing program search; since those programs will contain their own abstractions in any case).