4 ms·
Actually the limit predicted by Shannon can be significantly beaten, because Shannon assumes gaussian noise, but if we use photon counting receivers we need to
by cycomanic 2y ago
Actually the limit predicted by Shannon can be significantly beaten, because Shannon assumes gaussian noise, but if we use photon counting receivers we need to use a poisson distribution. This is the Gordon-Holevo limit.
To beat Shannon you need PPM formats and photon counters (single photon detectors).
One can do significantly better than the numbers from voyager in the article using optics even without photon cpunting. Our group has shown 1 photon/bit at 10 Gbit/s [1] but others have shown even higher sensitivity (albeit at much lower data rates).
[1] https://www.nature.com/articles/s41377-020-00389-2 https://www.nature.com/articles/s41377-020-00389-2
- nico 2y agoInteresting. Is that related to compressed sensing? I wonder if compress sensing could be used for something like the Voyager signals It seems there might be multiple ways to go beyond Shannon’s limit, depending on what you are trying to do
- cycomanic 2y agoI don't think compressed sensing is really extracting more information than Shannon, it simply exploits the fact that the signal we are interested in is sparse so we don't need to sample "everything". But this is somewhat outside my area of expertise so my understanding could be wrong.
- nico 2y agoMaybe I’m mixing Shannon’s limit with the sampling rate imposed by the Nyquist-Shannon Sampling theorem > Around 2004, Emmanuel Candès, Justin Romberg, Terence Tao, and David Donoho proved that given knowledge about a signal's sparsity, the signal may be reconstructed with even fewer samples than the sampling theorem requires.[4][5] This idea is the basis of compressed sensing … > However, if further restrictions are imposed on the signal, then the Nyquist criterion may no longer be a necessary condition. A non-trivial example of exploiting extra assumptions about the signal is given by the recent field of compressed sensing, which allows for full reconstruction with a sub-Nyquist sampling rate. Specifically, this applies to signals that are sparse (or compressible) in some domain From: https://en.m.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem https://en.m.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_samp...
- sunk1st 2y agoIn so many words, Shannon gave a proof showing that in general the sample rate of a digital sensor puts an upper bound on the frequency of any signal that sensor is able to detect. Unlike the Nyquist-Shannon theory, compressed sensing is not generally applicable: it requires a sparse signal. As with many other optimization techniques, it’s a trade off between soundness and completeness.
- nico 2y agoGreat way to explain it! Loved this: > As with many other optimization techniques, it’s a trade off between soundness and completeness
- kragen 2y agothat is not correct. digital sensors detect frequencies above the nyquist limit all the time, which is why they need an analog antialiasing filter in front of them. what they can't do is distinguish them from baseband aliases you could just as correctly say 'nyquist-shannon theory is not generally applicable; it requires a bandlimited signal' (which is why compressed sensing doesn't violate it)
- nico 2y agoThank you for the clarification, great point about the importance of distinguishing the acts of "detecting" and "making sense of" some signal/data/information
- immibis 2y agoConsider the simple example of frequency aliasing. If you sample a 3.2MHz sine wave at a 1MHz sample rate, it looks the same as a 0.2MHz wave. But if you know a priori that the signal only has frequency components between 3 and 3.5 MHz, then you know the 0.2MHz you are measuring is actually 3.2MHz - you can fully reconstruct the original signal even though you are not sampling it fast enough.
- 2y ago
- ramraj07 2y agoCan’t you calculate the CRLB for any given distribution if you wanted? That’s what my lab did for microscopy anyway. Saying you’re beating the Shannon limit is like saying you’re beating the second law of thermodynamics to me.. but I could be wrong.
- deleted 2y ago[deleted]
- Sanzig 2y agoYou are correct. People often say "Shannon limit" (the general case) when they are really referring to the "Shannon-Hartley Limit" (the simplified case of an additive white Gaussian noise channel). For example, MIMO appears to "break" the Shannon-Hartley limit because it does exceed the theoretical AWGN capacity for a simple channel. However, when you apply Shannon's theory to reformulate the problem for the case of a multipath channel with defined mutual coupling, you find that there is a higher limit you are still bounded by.
- s1dev 2y agoI often wondered why MIMO was such an investigated topic. It would make sense if the Shannon limit is higher for this channel. Is there a foundational paper or review that shows this?
- cycomanic 2y agoShannon theory assumes Gaussian noise, however in the very low power regime that's just not true. I agree it's unintuitive. Have a look at the Gordon paper I posted earlier.
- aptitude_moo 2y agoVery interesting, I studied telecommunications and I thought the Shannon limit was the absolute limit. I wonder now if this Gordon Holevo limit is applicable for "traditional" telecommunications (like 5G) as opposed to photon counting a deep space probe EDIT: This paper seems to answer my question [1] [1] https://opg.optica.org/directpdfaccess/8711ab35-bbc2-4d51-8e532b36ec497bcf_432047/jlt-38-10-2741.pdf https://opg.optica.org/directpdfaccess/8711ab35-bbc2-4d51-8e...
- stracer 2y agoCan you please post another link? This one does not work.
- HappyPanacea 2y agoI think it is https://arxiv.org/abs/2002.05766 https://arxiv.org/abs/2002.05766.
- richarme 2y agoI think it's this one: https://opg.optica.org/jlt/viewmedia.cfm?uri=jlt-38-10-2741&seq=0 https://opg.optica.org/jlt/viewmedia.cfm?uri=jlt-38-10-2741&... Quantum Limits in Optical Communications Konrad Banaszek, Ludwig Kunz, Michał Jachura, and Marcin Jarzyna
- aptitude_moo 2y agoWeird, it works for me. You can use any of the other published links in the sibling comments. The name of the paper is "Quantum Limits in Optical Communications" by Banaszek
- adrian_b 2y agoAs also explained in the conclusion of the paper linked by you, "photon-counting" detectors are possible only when the energy of one photon is high enough, which happens only for infrared light or for higher frequencies. "Photon-counting" methods cannot be implemented at frequencies so low as used in 5G networks or in any other traditional radio communications.
- stracer 2y agoIs there some fundamental limit to the number of bits per photon that can be communicated via EM radiation? I think it does not exist, because photons aren't all equal, we can use very high frequency and X-ray quantum can probably carry much more information than RF quantum.
- resters 2y agothis is called the Shannon limit. To discern signal from noise, a minimum sample rate of 2x the frequency of the signal is required. A signal is something that can be turned on or off to send a bit. Higher frequencies can carry more data as you infer but the engineering challenges of designing transmitters and receivers create tradeoffs in practical systems.
- ganzuul 2y agoIn addition to wavelength EM also has several polarization modes and near/far field characteristics that can carry information.
- resters 2y agoCan individual photons be measured for polarization and phase or is there a similar limit that requires more than one photon to do so? I suppose both are relative to some previous polarization or phase?
- stracer 2y agoPolarization can be measured using polarization filter and light detector, but it is destructive in the usual sense of quantum theory. That is, if the detector after polarization filter clicks, we know the EM field had non-zero component in the direction of the filter, but we do not find out the other components it had before entering the filter.
- ganzuul 2y agoGood induction to thinking about quantum gravity.
- sansseriff 2y agoThe Deep Space Optical Communication (Dsoc) between earth and the psyche spacecraft uses large-M PPM for this reason! This mission is currently ongoing. They send optical pulses in one of up to 128 possible time slots, thereby carrying 7 bits each. And each optical pulse on earth may only be received by 5-10 photons.
- codedokode 2y agoI am not sure if you can use 1 photon per bit because (as I understand) emitting and capturing photons is a probabilistic process and when you have 1 photon, there is a probability that it will not be captured by an antenna, but rather will be reflected or will turn into heat. Or am I wrong here?
- eru 2y agoIn principle, you can send more than one photon per bit on average. Photons have a lot of ways they can encode additional bits, eg frequency, polarisation, timing. You are right that you can randomly lose some photons. That's what error correcting codes are. See https://en.wikipedia.org/wiki/Error_correction_code https://en.wikipedia.org/wiki/Error_correction_code As an example, assume every photon can encode 10 bits without losses, but you lose 10% of your photons. Then with a clever error correcting code you can encode just shy of 9 bits per photon. You can think of the error correcting code 'smearing' 9 * n bits of information over 10 * n photos, and as long as you collect 0.9 * n photons, you can recover the 9 * n bits of information. It's the same reason your CD still plays, even if you scratch it. In fact, you can glue a paper strip of about 1 cm width on the bottom of your CD, and it'll still play just fine. Go wider, and it won't, because you'll be exceeding the error correcting capacity of the code they are using for CDs.
- lebed2045 2y ago*typo: you can send more than one bit per photon on average I'm very curious to learn more about 1cm, what is the math behind it? Do you speak about classical music CD with ±700mb of capacity? I was always fascinating by ability of old super scratched optical disks still functioning without problems.
- eru 2y ago> I'm very curious to learn more about 1cm, what is the math behind it? So I actually got that from a cool math talk I attended about 20 years ago. At the end the professor had a cool demonstration where he glued paper strips of various sizes radially on the CD, and exactly as the math he spend an hour explaining predicted, the CD player could cope with up to a 1cm width strip, but no more. Let me try to find some written material. https://en.wikipedia.org/wiki/Cross-interleaved_Reed%E2%80%93Solomon_coding https://en.wikipedia.org/wiki/Cross-interleaved_Reed%E2%80%9... is a good start, but doesn't go into the details. https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_cor... might also be worth a read. In a nutshell, you arrive at the 1cm like this: you can look up what proportion of 'wrong' bits the CD's coding can correct and other overhead. Then you look up the circumference of a CD (about 28 cm), then you do some multiplication, and figure out that you can lose about 1cm out of every 28cm, and still be able to correct. Most of the interesting math happens at the first step of 'what proportion of errors can the music CD correct?' and more interestingly 'how does the CD player do that?' > I was always fascinating by ability of old super scratched optical disks still functioning without problems. Keep in mind that CD-ROMs have one additional layer of coding on top of what music CDs have. That's because if a bit error slips through the error correction chances are it still won't be audible to the human ear for music, but software might still crash with a single wrong bit. > Do you speak about classical music CD with ±700mb of capacity? Yes, that's because that's what I heard the talk about. I am sure more modern formats also have interesting error correction, but I don't know what they use and how much you could cover up.