4 ms·

30 years ago? What took so long for broad adoption? Patents? edit Article doesn't elaborate on why that might be, but it does note: "an alternative that has bee

30 years ago? What took so long for broad adoption? Patents? edit Article doesn't elaborate on why that might be, but it does note: "an alternative that has been given a new lease on life is low-density parity check (LDPC) codes, invented in the early 1960s by Robert Gallager at MIT but largely forgotten since then...Now researchers have implemented LDPC codes so that they actually outperform turbo codes and get even closer to the Shannon limit...Another advantage, perhaps the biggest of all, is that the LDPC patents have expired, so companies can use them without having to pay for intellectual-property rights."

Yoric 4y ago
Unfortunately, one of the common scenarios when it comes to percolating results from academia to industry is the following: 1. Academics make discovery/invent something. 2. Academics attempt to convince industry to adopt discovery. 3. Discovery is laughed out of industry as "impractical", "academic", etc. 4. Academics teach discovery to their students. 5. Students get into industry. 6. Students get into position of tech leads, architects, etc. 7. Students demonstrate the use of discovery. (by then, 30 years have elapsed) e.g. garbage collectors, type systems, threads, distributed systems, message passing concurrency, actors, JITs, metacompilation, machine learning, functional programming...
- pipo234 4y ago
  > 3. Discovery is laughed out of industry as "impractical", "academic", etc. To be fair, sometimes academic inventions are impractical given the technical landscape du jour. They only become feasible as technology and society progresses, say 30 years. Also, your observation does not mean any invention is meaningful (and will eventually be recognized as such). I for one, learned a bunch of "impractical", "academic" nonsense too in university. But yes, I think as a whole, your 7 points observation as a whole is very much at play in the Turbo Codes case. Yes.
  - Yoric 4y ago
    Fair enough.
  - jiggawatts 4y ago
    The specific issue with LDPC and Turbo Codes is that they both introduce latency, and require a ton of compute to decode. If you don't care about latency and have an "unlimited" compute budget, then they're great and hence they were used very early for deep space communications. The speed-of-light delay means that there's minutes or even hours of latency anyway. And of course, the receiving station is a data centre with as much computer power available as you please. In hand-held devices running on battery with latency constraints measured in milliseconds, it's an entirely different ballgame. Suddenly, there's real engineering problems to solve, and the trade-offs aren't always worth it. Something I've told people when both 4G and 5G technologies rolled out is that these are enabled by the advances in the bleeding edge of silicon chip technology. Both rely on very heavyweight signal decoding algorithms that require dedicated "offload" circuitry almost as big as the main CPU! Some anti-5G crazies think that their hand is getting hot from the "radiation", when in fact it is the decoding chip that's getting warm from the power it has to draw from the battery! PS: This is why every new 4G/5G/6G/etc... tech always gets panned as impacting battery life. One or two silicon process shrinks later, this is not a problem. The radio power didn't change, but the decoding power did!
- HopenHeyHi 4y ago
  0. Science Advances One Funeral at a Time
- readthenotes1 4y ago
  1.5 More established academics deride, scoff, and do their best to extinguish this new work.
  - Yoric 4y ago
    I'm sure that this happens. When I was an academic, I never encountered (or heard of) such situations, though.
  - hinkley 4y ago
    4.5 some of the less arrogant academics internalize the complaints about their solutions and iterate on it to make it 20% easier to use and/or 20% better in practice, making the cost of change more attractive.
    - Yoric 3y ago
      Academics have different goals, metrics and incentives than industry, but I would seldom call academic work "arrogant". In particular, iterating to make things easier may or may not be sufficient to get the industry to take a second look at the project, and there is a chance that it may consume enough of the academic's time to ruin their career in academia. It's anecdata, but I have seen a few cases (e.g. Coccinelle) in which an academic essentially hides the fact that their contributions to e.g. an open-source project comes from academic research until after it has been battle-tested by industry.
  - Turing_Machine 4y ago
    Sometimes the academics aren't talking to each other, not out of spite, but just because they're in different fields. I recall from a computer vision class that some of the early CV researchers wasted a lot of time redeveloping math techniques that had already been developed by the photogrammetry people literally decades before.
- bheadmaster 4y ago
  Wait, what exactly is metacompilation, and in what form is it used in the industry? I've tried searching the internet, but only found theoretical explanations I'm too tired to parse right now, and something about yaks and bisons.
  - abecedarius 4y ago
    I don't think it's a really standard term, but I guess from context it's referring to partial-evaluation-style approaches to compiling. (In the Forth world it means a special Forth vocabulary the Forth system can use to bootstrap itself.)
  - Yoric 3y ago
    Sorry, I meant multi-staged compilation: https://en.wikipedia.org/wiki/Multi-stage_programming https://en.wikipedia.org/wiki/Multi-stage_programming, e.g. MetaOCaml or Template Haskell. And you're right, it may be a bit early to speak of it being used in the industry. Examples of subset of multi-staged compilation include `constexpr` in C++ and `comptime` in Zig. I also seem to remember seeing stuff in the GraalVM ecosystem but I can't find them right now.
- nunuvit 4y ago
  While that does sort of happen, that's not the reason. Article says it was used in satellite links and deep space networks before it was cool. Those applications had hit their technological limits and no one was making something new they could buy next year, so they had to do it themselves. Contrast that with phone networks being able to rely on the next G coming out. Now we have 5G, but it's 5G with an asterisk, so it's time to look for a new approach. Same with Moore's law. The semiconductor industry was really focused on smaller transistors for a long time until they started getting close to the physical limit. Then suddenly everyone is talking about chiplets and other ideas that had been around a long time but weren't mainstream. Changing approach requires a lot of coordination and carries a lot of risk. It's easier to finish milking one cow for as long as possible before moving on to the next. The laughing is just because your advisor knows how long it will probably be before the general market has exhausted its old approach.
  - taneq 4y ago
    Yep, as in all things, we do the easy thing until it stops working. Only then is it worth putting more work into harder things. I’m not deriding this approach, mind you. It’s efficient. Edit: Also the incremental improvement over the current system needs not only to be big enough that it outweighs the cost of the changeover (including hidden costs like risks and opportunity cost), but also to be big enough to bother with in absolute terms. Things like zip, mp3 and jpg are good enough that even though there are now alternatives that are better by any reasonable metric, for most users it's just not worth the bother of changing over.
giobox 4y ago
Not an expert, but I've encountered Reed Solomon error correction extensively in my career, and this seems to be replacing RS applications? I wouldn't be surprised if there was a fair amount of inertia to overcome given how widely used and how well RS worked for a very long time. RS was critical for things like ADSL data transmission over POTS and other similar large scale applications. The wikipedia page for RS suggests some RS implementations are now "being replaced by more powerful turbo codes". > https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_cor...
- akhayam 4y ago
  Just adding some extra color: Turbo codes did get instant traction in the industry once the academics actually "believed" them to be true. So the friction came from the academics and not the industry. Today, every communication system you are using are relying on Turbo codes in some shape or form.
topaz0 4y ago
Note that the article is almost 20 years old, so it is more like 10 years for industry adoption.
mturmon 4y ago
Turbo codes were used in the Mars Reconnaissance Orbiter (2005 launch) due to work by JPL telecommunication researchers. (https://en.wikipedia.org/wiki/Mars_Reconnaissance_Orbiter#Telecommunications_system https://en.wikipedia.org/wiki/Mars_Reconnaissance_Orbiter#Te...) Edited to add an early link to more granular technical work: https://ipnpr.jpl.nasa.gov/progress_report/42-120/120D.pdf https://ipnpr.jpl.nasa.gov/progress_report/42-120/120D.pdf
cryptonector 4y ago
Patent lifetimes are 20 years.

Yoric 4y ago

Unfortunately, one of the common scenarios when it comes to percolating results from academia to industry is the following: 1. Academics make discovery/invent something. 2. Academics attempt to convince industry to adopt discovery. 3. Discovery is laughed out of industry as "impractical", "academic", etc. 4. Academics teach discovery to their students. 5. Students get into industry. 6. Students get into position of tech leads, architects, etc. 7. Students demonstrate the use of discovery. (by then, 30 years have elapsed) e.g. garbage collectors, type systems, threads, distributed systems, message passing concurrency, actors, JITs, metacompilation, machine learning, functional programming...

pipo234 4y ago
> 3. Discovery is laughed out of industry as "impractical", "academic", etc. To be fair, sometimes academic inventions are impractical given the technical landscape du jour. They only become feasible as technology and society progresses, say 30 years. Also, your observation does not mean any invention is meaningful (and will eventually be recognized as such). I for one, learned a bunch of "impractical", "academic" nonsense too in university. But yes, I think as a whole, your 7 points observation as a whole is very much at play in the Turbo Codes case. Yes.
- Yoric 4y ago
  Fair enough.
- jiggawatts 4y ago
  The specific issue with LDPC and Turbo Codes is that they both introduce latency, and require a ton of compute to decode. If you don't care about latency and have an "unlimited" compute budget, then they're great and hence they were used very early for deep space communications. The speed-of-light delay means that there's minutes or even hours of latency anyway. And of course, the receiving station is a data centre with as much computer power available as you please. In hand-held devices running on battery with latency constraints measured in milliseconds, it's an entirely different ballgame. Suddenly, there's real engineering problems to solve, and the trade-offs aren't always worth it. Something I've told people when both 4G and 5G technologies rolled out is that these are enabled by the advances in the bleeding edge of silicon chip technology. Both rely on very heavyweight signal decoding algorithms that require dedicated "offload" circuitry almost as big as the main CPU! Some anti-5G crazies think that their hand is getting hot from the "radiation", when in fact it is the decoding chip that's getting warm from the power it has to draw from the battery! PS: This is why every new 4G/5G/6G/etc... tech always gets panned as impacting battery life. One or two silicon process shrinks later, this is not a problem. The radio power didn't change, but the decoding power did!
HopenHeyHi 4y ago
0. Science Advances One Funeral at a Time
readthenotes1 4y ago
1.5 More established academics deride, scoff, and do their best to extinguish this new work.
- Yoric 4y ago
  I'm sure that this happens. When I was an academic, I never encountered (or heard of) such situations, though.
- hinkley 4y ago
  4.5 some of the less arrogant academics internalize the complaints about their solutions and iterate on it to make it 20% easier to use and/or 20% better in practice, making the cost of change more attractive.
  - Yoric 3y ago
    Academics have different goals, metrics and incentives than industry, but I would seldom call academic work "arrogant". In particular, iterating to make things easier may or may not be sufficient to get the industry to take a second look at the project, and there is a chance that it may consume enough of the academic's time to ruin their career in academia. It's anecdata, but I have seen a few cases (e.g. Coccinelle) in which an academic essentially hides the fact that their contributions to e.g. an open-source project comes from academic research until after it has been battle-tested by industry.
- Turing_Machine 4y ago
  Sometimes the academics aren't talking to each other, not out of spite, but just because they're in different fields. I recall from a computer vision class that some of the early CV researchers wasted a lot of time redeveloping math techniques that had already been developed by the photogrammetry people literally decades before.
bheadmaster 4y ago
Wait, what exactly is metacompilation, and in what form is it used in the industry? I've tried searching the internet, but only found theoretical explanations I'm too tired to parse right now, and something about yaks and bisons.
- abecedarius 4y ago
  I don't think it's a really standard term, but I guess from context it's referring to partial-evaluation-style approaches to compiling. (In the Forth world it means a special Forth vocabulary the Forth system can use to bootstrap itself.)
- Yoric 3y ago
  Sorry, I meant multi-staged compilation: https://en.wikipedia.org/wiki/Multi-stage_programming https://en.wikipedia.org/wiki/Multi-stage_programming, e.g. MetaOCaml or Template Haskell. And you're right, it may be a bit early to speak of it being used in the industry. Examples of subset of multi-staged compilation include `constexpr` in C++ and `comptime` in Zig. I also seem to remember seeing stuff in the GraalVM ecosystem but I can't find them right now.
nunuvit 4y ago
While that does sort of happen, that's not the reason. Article says it was used in satellite links and deep space networks before it was cool. Those applications had hit their technological limits and no one was making something new they could buy next year, so they had to do it themselves. Contrast that with phone networks being able to rely on the next G coming out. Now we have 5G, but it's 5G with an asterisk, so it's time to look for a new approach. Same with Moore's law. The semiconductor industry was really focused on smaller transistors for a long time until they started getting close to the physical limit. Then suddenly everyone is talking about chiplets and other ideas that had been around a long time but weren't mainstream. Changing approach requires a lot of coordination and carries a lot of risk. It's easier to finish milking one cow for as long as possible before moving on to the next. The laughing is just because your advisor knows how long it will probably be before the general market has exhausted its old approach.
- taneq 4y ago
  Yep, as in all things, we do the easy thing until it stops working. Only then is it worth putting more work into harder things. I’m not deriding this approach, mind you. It’s efficient. Edit: Also the incremental improvement over the current system needs not only to be big enough that it outweighs the cost of the changeover (including hidden costs like risks and opportunity cost), but also to be big enough to bother with in absolute terms. Things like zip, mp3 and jpg are good enough that even though there are now alternatives that are better by any reasonable metric, for most users it's just not worth the bother of changing over.

giobox 4y ago

Not an expert, but I've encountered Reed Solomon error correction extensively in my career, and this seems to be replacing RS applications? I wouldn't be surprised if there was a fair amount of inertia to overcome given how widely used and how well RS worked for a very long time. RS was critical for things like ADSL data transmission over POTS and other similar large scale applications. The wikipedia page for RS suggests some RS implementations are now "being replaced by more powerful turbo codes". > https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_cor...

akhayam 4y ago
Just adding some extra color: Turbo codes did get instant traction in the industry once the academics actually "believed" them to be true. So the friction came from the academics and not the industry. Today, every communication system you are using are relying on Turbo codes in some shape or form.

topaz0 4y ago

Note that the article is almost 20 years old, so it is more like 10 years for industry adoption.

mturmon 4y ago

Turbo codes were used in the Mars Reconnaissance Orbiter (2005 launch) due to work by JPL telecommunication researchers. (https://en.wikipedia.org/wiki/Mars_Reconnaissance_Orbiter#Telecommunications_system https://en.wikipedia.org/wiki/Mars_Reconnaissance_Orbiter#Te...) Edited to add an early link to more granular technical work: https://ipnpr.jpl.nasa.gov/progress_report/42-120/120D.pdf https://ipnpr.jpl.nasa.gov/progress_report/42-120/120D.pdf

cryptonector 4y ago

Patent lifetimes are 20 years.