4 ms·

Is there a mathematical theory trying to quantify this value?

• deleted 8y ago
  [deleted]
• antidesitter 8y ago
  Bennett’s logical depth.
• sgentle 8y ago
  You might find relative entropy, aka information gain, aka Kullback-Liebler divergence interesting: https://en.m.wikipedia.org/wiki/Kullback–Leibler_divergence https://en.m.wikipedia.org/wiki/Kullback–Leibler_divergence Also this treatment of K-L divergence as a measure of "Bayesian surprise": http://ilab.usc.edu/surprise/ http://ilab.usc.edu/surprise/ That's all based on Shannon entropy (probabilistic), not Kolmogorov complexity (algorithmic), but there are a lot of connections between them. This paper is a pretty thorough summary: https://homepages.cwi.nl/~paulv/papers/info.pdf https://homepages.cwi.nl/~paulv/papers/info.pdf And here's a paper defining algorithmic relative complexity by analogy to relative entropy: http://www.mdpi.com/1099-4300/13/4/902/pdf-vor http://www.mdpi.com/1099-4300/13/4/902/pdf-vor "We define the cross-complexity of an object x with respect to another object y as the amount of computational resources needed to specify x in terms of y, and the complexity of x related to y as the compression power which is lost when adopting such a description for x, compared to the shortest representation of x"
• ppod 8y ago
  Might not be exactly what you're asking, but Landauer's principles that information is physical and necessarily requires some energy to compute is relevant. The 'cost' Bennett is talking about here might be the energy required to change the entropy.