Cryptography, error coding, and lossy compression are particularly concerned with what the author identifies as Jane Austen's concept of information but also ha

Cryptography, error coding, and lossy compression are particularly concerned with what the author identifies as Jane Austen's concept of information but also have a very close relationship to Shannon entropy. Basically in cryptography the problem is that plaintext messages reveal important information, and the probability distribution of plaintexts is roughly known. While "attack at dawn" and "attack at noon" may have a lot of letters those two messages may be the only expected messages, and they each may be close to 50% likely, reducing the Shannon entropy to nearly a single bit. Parties wishing to communicate secretly would like to obscure their communication by revealing no information about the actual probability distribution of the plaintexts, so Shannon entropy is maximized (with respect to tractability) when generating a ciphertext. Successful cryptography transforms a message channel into one where e.g. 2^N ciphertexts are all equally likely, hiding the semantic content of both the actual plaintext messages and their probabilities. On the other hand error codes strengthen the semantic meaning of the underlying message by significantly increasing the distribution while adjusting the probabilities toward favoring the important messages. Lossy compression (images, audio, video) is kind of the inverse of error coding. A reduction in the distribution by choosing which parts of the message are least semantically important and removing the distinction between them and making them indistinguishable. Roughly, Austen-information is limited by the Shannon entropy as an upper bound and also by a computable, tractable function of the available Shannon information (which we might as well call the semantic function which translates between bits and useful human representation). In cryptography's case P = D_K(C) is the particular function that turns an N-bit ciphertext from 0 bits of Austen information into N bits of Austin information with the proper key. Lossy compression turns an N-bit message into a semantically useful approximation of an N+M-bit message, excluding the production of a large class of semantically indistinguishable resulting messages in the process. There's another layer that maybe the author was getting at, and that's the human (or any agent) interpretation and response to sensory inputs. At that semantic layer it's again a function from inputs to experiences, responses, or actions. Information theory suggests that such functions are probably most useful if they can extract the most efficiency from all available channels, e.g. distinguish between important states of the world with the fewest bits possible. Going beyond that is a digression into intelligence itself where we have to determine what the most important states of the world are and how to respond.