amen brother, shalizi explains this problem well: Complexity, Entropy and the Physics of gzip: http://www.cscs.umich.edu/~crshalizi/notebooks/cep-gzip.html htt

amen brother, shalizi explains this problem well: Complexity, Entropy and the Physics of gzip: http://www.cscs.umich.edu/~crshalizi/notebooks/cep-gzip.html http://www.cscs.umich.edu/~crshalizi/notebooks/cep-gzip.html I think one of the limitations of the traditional formulation of information theory is that the entropy of a signal doesn't obviously tell you how much useful information is in the signal. This paper provides an interesting perspective on this issue. "The Information Bottleneck Method" Bialek, Tishby and Pereira http://www.princeton.edu/~wbialek/our_papers/tishby+al_99.pdf http://www.princeton.edu/~wbialek/our_papers/tishby+al_99.pd...