3 ms·
Ok his fundamental theorems aren't strictly binary. He wrote of discerning possible 'symbols' from noisy measurement where a symbol could have N possible values
by JoeAltmaier 8y ago
Ok his fundamental theorems aren't strictly binary. He wrote of discerning possible 'symbols' from noisy measurement where a symbol could have N possible values. The optimum discernment was at 2 values (1 and 0) but his math handled any base.
If I remember right after 20 years.
- hervature 8y agoActually, if I recall correctly after only 5 years, your first sentence is correct. It doesn't matter how many symbols your alphabet has (i.e. base system used in application) because they can all be represented using binary. I don't recall anything about "optimum discernment" just that it is convenient to standardize proofs using a binary alphabet. For example, Huffman coding is proved using binary but it easily applies to ternary systems as well.
- JoeAltmaier 8y agoYeah it had to do with Signal-to-Noise Ratio (SNR) being best at 2 symbols I think.
- neurocline 8y agoApparently the optimum information-theoretic alphabet would have e (2.718...) bits, but it's a little awkward to work with a transcendental base. The pain in working with trinary circuits is why we use binary instead.
- DoctorOetker 8y ago>The optimum discernment was at 2 values (1 and 0) I think this "optimum discernment" at base 2 you remember is nonsense? could you provide a pointer?
- laretluval 8y agoIndeed, his results are neither strictly binary nor strictly discrete.
- pjc50 8y agoYes, it's still useful for multi-symbol channel coding systems, and there are quite a few in common use; QAM is the obvious example.
- srean 8y agoIn fact it need not even be restricted to discrete signals. I think he was uncomfortable with the fact that the continuous signal story was not as well fleshed out. He came to revisit that after about a decade in "coding theorems for a discrete source with a fidelity criterion". This did for lossy compression what his 1948 paper did for lossless. It rarely happens that a paper that creates a field also resolves its most important questions. That was Shannon's standard, I guess. That he revisited the question perhaps speaks to his inner discomfort that the story was not complete.