2 ms·
As cameldrv said, information theory. The late David MacKay has a good free book: http://www.inference.org.uk/itprnn/book.html http://www.inference.org.uk/itpr
by SilasX 9y ago
As cameldrv said, information theory. The late David MacKay has a good free book:
http://www.inference.org.uk/itprnn/book.html http://www.inference.org.uk/itprnn/book.html
The intro chapters cover the topic of measuring informational entropy.
Aside: One interesting application is to the problem of “use a balance scale only three times to determine which of twelve balls has the wrong weight (too heavy or light)”. You choose the weighings so as as to maximize the entropy of the outcome i.e. so there are many possible outcomes of equal probability.