4 ms·

Claude Shannon 1948, "A Mathematical Theory of Communication" http://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Communication http://en.wikipedia.org/wiki/A

Claude Shannon 1948, "A Mathematical Theory of Communication" http://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Communication http://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Commun... This paper kickstarted the concept of information theory, and was hugely influential on many fields of research. Signal-to-noise ratio, the bit, information entropy, etc. are all theories and concepts presented by Shannon.

thehoneybadger 12y ago
Wow you beat me to it! This paper affected my college major, my career in my twenties, and my current career. Such an important paper, although, I must admit, it is not easy to decipher at first.
apricot 12y ago
This was the first academic paper I ever read, way back when. For a brief, wonderful time, I thought all academic papers were of a similar caliber, and I was in awe (and not a little bit intimidated).
- avinashv 12y ago
  > For a brief, wonderful time, I thought all academic papers were of a similar caliber I had the exact same thought in college when I started reading a handful of well-written papers like this myself. This really set a standard that for 60+ years now academia hasn't always maintained.
ivan_ah 12y ago
+1! I find the whole notion of "typical set" to be absolutely amazing. I would like to share with you a few pages from the intro to my thesis which cover Shannon's channel coding theorem. There are some nice TiKZ illustrations. http://minireference.com/static/excerpts/Shannon_channel_coding_thm.pdf http://minireference.com/static/excerpts/Shannon_channel_cod... (it's not super detailed, but the definitions of all the moving parts are given)
- swairshah 12y ago
  This is really cool. I'm working in systems research, but I'm fascinated by mathematical research in CS. Did you continue with similar work after your PhD, If I may ask?
  - ivan_ah 12y ago
    I've since switched my research focus to machine learning (look up latent Dirichlet allocation, very cool stuff). I find a lot of parallels between the two fields: prob. theory, matrices, uncertainty, ... I'm still following quantum information theory research, but more as a spectator from the sidelines. However, a couple of weeks ago I had to come back to quantum info. theory to "defend" my academic reputation. Our colleagues from TIFR found a bug in one of our papers (http://arxiv.org/abs/1111.3645v3 http://arxiv.org/abs/1111.3645v3) so my coauthor and I had to fix it. It was kind of cool to see I hadn't "lost my quantum skills" after two years of running a business. I guess, once you go quantum you never go back? :)
- sasvari 12y ago
  > http://minireference.com/static/excerpts/Shannon_channel_coding_thm.pdf http://minireference.com/static/excerpts/Shannon_channel_cod... in case somebody is interested in more than the 15 pages provided in the excerpt, here's ivan's complete thesis: http://arxiv.org/pdf/1208.4188v1.pdf http://arxiv.org/pdf/1208.4188v1.pdf
- avinashv 12y ago
  Very cool! Shannon's paper was, to be honest, a little out of my comfort zone when I first read it (this isn't really my field academically), and I got a little lost on the first read of this, but someone below posted your full thesis which I will definitely read when I have the time.

thehoneybadger 12y ago

Wow you beat me to it! This paper affected my college major, my career in my twenties, and my current career. Such an important paper, although, I must admit, it is not easy to decipher at first.

apricot 12y ago

This was the first academic paper I ever read, way back when. For a brief, wonderful time, I thought all academic papers were of a similar caliber, and I was in awe (and not a little bit intimidated).

avinashv 12y ago
> For a brief, wonderful time, I thought all academic papers were of a similar caliber I had the exact same thought in college when I started reading a handful of well-written papers like this myself. This really set a standard that for 60+ years now academia hasn't always maintained.

ivan_ah 12y ago

+1! I find the whole notion of "typical set" to be absolutely amazing. I would like to share with you a few pages from the intro to my thesis which cover Shannon's channel coding theorem. There are some nice TiKZ illustrations. http://minireference.com/static/excerpts/Shannon_channel_coding_thm.pdf http://minireference.com/static/excerpts/Shannon_channel_cod... (it's not super detailed, but the definitions of all the moving parts are given)

swairshah 12y ago
This is really cool. I'm working in systems research, but I'm fascinated by mathematical research in CS. Did you continue with similar work after your PhD, If I may ask?
- ivan_ah 12y ago
  I've since switched my research focus to machine learning (look up latent Dirichlet allocation, very cool stuff). I find a lot of parallels between the two fields: prob. theory, matrices, uncertainty, ... I'm still following quantum information theory research, but more as a spectator from the sidelines. However, a couple of weeks ago I had to come back to quantum info. theory to "defend" my academic reputation. Our colleagues from TIFR found a bug in one of our papers (http://arxiv.org/abs/1111.3645v3 http://arxiv.org/abs/1111.3645v3) so my coauthor and I had to fix it. It was kind of cool to see I hadn't "lost my quantum skills" after two years of running a business. I guess, once you go quantum you never go back? :)
sasvari 12y ago
> http://minireference.com/static/excerpts/Shannon_channel_coding_thm.pdf http://minireference.com/static/excerpts/Shannon_channel_cod... in case somebody is interested in more than the 15 pages provided in the excerpt, here's ivan's complete thesis: http://arxiv.org/pdf/1208.4188v1.pdf http://arxiv.org/pdf/1208.4188v1.pdf
avinashv 12y ago
Very cool! Shannon's paper was, to be honest, a little out of my comfort zone when I first read it (this isn't really my field academically), and I got a little lost on the first read of this, but someone below posted your full thesis which I will definitely read when I have the time.