5 ms·

As someone with PhD studies in Information Theory, I always found the whole "compression breakthrough" very far-fetched, I wish they had chosen another technica

As someone with PhD studies in Information Theory, I always found the whole "compression breakthrough" very far-fetched, I wish they had chosen another technical basis for the show. Never mind that compression of any and all files is impossible (see pigeon hole principle) -- while it could theoretically be done for "normal" files (video etc), it's extremely unlikely that a black-box method such as theirs (as in, they don't have any prior knowledge of the files' structures) would beat out white-box methods (such as video compression) by such a large margin. It's a bit like writing a best selling novel in a language you don't speak: theoretically possible, but just so far from normal reality that it's hard to suspend disbelief.

technofiend 9y ago
That's the pain we all feel on more accessible topics and their treatment in mainstream entertainment. It's just more acute for you because you're an expert. Did you enjoy the Sneaker's "can decrypt anything" plot line? How about that stupid TV show with a magical button that turned on and off electricity? I couldn't be in the same room when my wife watched that crap. Happens to everyone. I'm not dismissing what you say, more like welcome to the club. :-)
- philbarr 9y ago
  > How about that stupid TV show with a magical button that turned on and off electricity? I think you're talking about "Revolution" which had JJ Abrams splattered all over the adverts. I watched one episode and thought, after Lost, "I'm not falling for this again."
  - Cyph0n 9y ago
    "Revolution" proved to me that Lindelof was the more interesting one from the "Lost" duo. Case in point: Lindelof's "The Leftovers" is one of the best shows airing right now.
    - prodmerc 9y ago
      The leftovers is still on? Damn, who the hell watches that...
      - Cyph0n 9y ago
        Oh man, "The Leftovers" is an amazing show in my opinion, but it's definitely not for the faint of heart. Another gem I discovered this year is Amazon's "Patriot". Coincidentally, it features one of the main actors from "Lost" :P
        technofiend 9y ago
        I classify horror movies and post-apocalyptic shows like The Walking Dead as "bad things happen to good people for no reason other than plotline." I can only watch so much of that before it wears me out because there are no surprises and nothing beyond temporary respite for the characters. As you say, it requires viewers be really invested to watch everyone die, but if anyone can do it George R.R. Martin can. The old joke is if people bother him enough about the next Game of Thrones book it'll just be 1,000 pages of snow blowing across graves. When he's done with the series I'm not sure how many characters, readers or viewers will be left. Even so he's the only author I can point to who can create compelling enough content that people may stay to the end. I'll never watch another Walking Dead or Leftovers because I just don't care enough about the characters, but George will probably manage to drag me to the end of GoT. Damn you, George.
  - technofiend 9y ago
    Yeah, I think I was blocking the show name like some childhood trauma. I was a little surprised JJ did such a poor job for backstory but then again don't get me started on his Star Trek reboots. :grumble:
    - tnecniv 9y ago
      I've seen probably a half dozen things JJ has worked on, but none of them end up being any good. He normally starts out with a pretty good premise, but it rapidly becomes apparent that he doesn't know what to do with it. Besides Star Trek (at least the first one, the second one is a different story) and Star Wars, Fringe and Super 8 jump to mind. Fortunately Stranger Things got to be what Super 8 could have been.
  - e12e 9y ago
    Interestingly enough, they sort of save the hard sci-fi plot of revolution ; they just take a number of seasons to pull the origin of the nano-tech scourge out of the hat (yes, it was well intentioned, yes self-replicating nano-tech turned out to be a bad idea, yes there was a command and control feature built in).
- lotsofpulp 9y ago
  Not just for topics that you're knowledgeable on, but also if you pick up on the patterns that writers use or read tvtropes it also ruins the watch-ability. I can't watch anything made by broadcast TV or TBS/TNT/USA or that Marvel stuff, it's just so boring and repetitive. Although, I do wonder what it says about people that the most popular movies involve creating a fantasy where the main character has some type of calamity during childhood then goes on to save the world with superpowers.
  - technofiend 9y ago
    >Although, I do wonder what it says about people that the most popular movies involve creating a fantasy where the main character has some type of calamity during childhood then goes on to save the world with superpowers. Lots of folks carrying childhood angst and wishing they could make things right, I guess.
  - tnecniv 9y ago
    > Not just for topics that you're knowledgeable on, but also if you pick up on the patterns that writers use or read tvtropes it also ruins the watch-ability. This is true of story telling in general, though. There's really only so many story elements. That doesn't mean you can't write a new, great story with them today (just like there's still great music being made despite us only having 12 notes in Western music).
modalduality 9y ago
>It's a bit like writing a best selling novel in a language you don't speak: theoretically possible, but just so far from normal reality that it's hard to suspend disbelief. Jhumpa Lahiri managed to do this! See http://www.newyorker.com/magazine/2015/12/07/teach-yourself-italian http://www.newyorker.com/magazine/2015/12/07/teach-yourself-....
hannob 9y ago
So my feeling was similar, but I had a slightly different issue with it: Even if you had a breakthrough compression algorithm, it's highly unlikely you'd build a startup with it and sell it as a proprietary tech. There are plenty of better compression techs out there for various use cases (zstd, brotli, opus, ...), and pretty much all of them today are thrown out by some major corp as open source. Even giving the better compression tech out for free it's incredibly hard to get people to adopt it. I mean github still uses gzip for their source downloads for whatever unclear reason.
- sanderjd 9y ago
  That's why they had to build the platform!
Udik 9y ago
Exactly my problem. Moreover, technical aspects aside, it seems a very poor choice for a satire on silicon valley: data compression is hard science, more likely to come from university maths departments than from garage startups; it doesn't require much marketing either, or hype, or social networking, or even fancy websites. Given that the real SV offers endless examples of silly products and enterprises (just the first that come to mind: the app to send "YO" to your friends; the completely useless $400 juice squeezer with wifi connection) putting such a no-nonsense innovation like a novel data compression algorithm at the center of the show is already completely off target.
rusk 9y ago
There's an old story-tellers saying: "Don't let the truth get in the way of a good yarn". Abandon disbelief my friend, and immerse yourself in the theatre. Focus on the message, not the lettering.
empath75 9y ago
> As someone with PhD studies in Information Theory In order for them to come up with a business plan that made 100% logical sense to anyone with expertise on the topic, they'd have had to come up with a legitimate business plan, and at that point, they could have just launched a startup instead of making a sitcom.
- tnecniv 9y ago
  That would make for a great interview "Well, we wanted to write this show that was a satire of the culture in Silicon Valley, but in the process we revolutionized the field of information theory."
eduren 9y ago
While I agree that the compression breakthrough doesn't make much sense realistically, I think it makes a great premise for the show. The plot needs Richard to actually invent something revolutionary (and thus fail to get it off the ground time and time again), that is also simple enough for the audience to understand. Any algorithmic breakthrough that is real-world feasible would either need too much vague handwaving from the writing ("A brand new way to compute data!"). Or be close enough to reality that neither the layman nor the expert is happy (imagine the show having to explain why the big-O of Richard's new magic sorting algorithm matters). Compression is the logical choice because it's: 1) easier to explain why fitting more data into less space is significant (laymen already have some rough visualization of compression), 2) lends itself to being "black boxed" in the plot ("Look at all the companies we can build with this one breakthrough"), 3) just barely feasible enough that if some SV nerd actually did make the discovery, then events would play out roughly the same as the show depicts (winning techcrunch, getting funding)
- monocasa 9y ago
  And I'd add that most compressors make a lot of assumptions about the general case that reduce that real world compression ratio when compared to theoretical maximums. That's why stuff like kkrunchy can't be beat in their domains (first and last rule of optimization, "know your data inside and out!"). If someone was to stumble upon a heuristic that gives you noticeable gains for general compression as well, it could be close to as game changing as Richard's stuff is. Obviously we have a good handle on "theoretical maximums" thanks to information theory and so that end of the plot is BS, but looking past that embellishment the show isn't outside the realm of possibility.
- thomastjeffery 9y ago
  It's perfect, because there is one impossible plot device in the real world. The attention to detail used to accomplish that is very appealing to me. It's my very favorite aspect of the show.
Nimitz14 9y ago
How do you not realize for any subject you pick there'll be another PhD student like you bitching about it?
e12e 9y ago
I always felt that was part of the joke; young programmers chasing the gold at the end of the rainbow.
ericd 9y ago
I once sat on a plane next to someone who was an expert in glass. It ruined every action movie for him, because the glass always shattered all wrong. And sometimes they would cut holes in glass that you couldn't do that with. I'd try to suspend the disbelief :-)

technofiend 9y ago

That's the pain we all feel on more accessible topics and their treatment in mainstream entertainment. It's just more acute for you because you're an expert. Did you enjoy the Sneaker's "can decrypt anything" plot line? How about that stupid TV show with a magical button that turned on and off electricity? I couldn't be in the same room when my wife watched that crap. Happens to everyone. I'm not dismissing what you say, more like welcome to the club. :-)

philbarr 9y ago
> How about that stupid TV show with a magical button that turned on and off electricity? I think you're talking about "Revolution" which had JJ Abrams splattered all over the adverts. I watched one episode and thought, after Lost, "I'm not falling for this again."
- Cyph0n 9y ago
  "Revolution" proved to me that Lindelof was the more interesting one from the "Lost" duo. Case in point: Lindelof's "The Leftovers" is one of the best shows airing right now.
  - prodmerc 9y ago
    The leftovers is still on? Damn, who the hell watches that...
    - Cyph0n 9y ago
      Oh man, "The Leftovers" is an amazing show in my opinion, but it's definitely not for the faint of heart. Another gem I discovered this year is Amazon's "Patriot". Coincidentally, it features one of the main actors from "Lost" :P
      - technofiend 9y ago
        I classify horror movies and post-apocalyptic shows like The Walking Dead as "bad things happen to good people for no reason other than plotline." I can only watch so much of that before it wears me out because there are no surprises and nothing beyond temporary respite for the characters. As you say, it requires viewers be really invested to watch everyone die, but if anyone can do it George R.R. Martin can. The old joke is if people bother him enough about the next Game of Thrones book it'll just be 1,000 pages of snow blowing across graves. When he's done with the series I'm not sure how many characters, readers or viewers will be left. Even so he's the only author I can point to who can create compelling enough content that people may stay to the end. I'll never watch another Walking Dead or Leftovers because I just don't care enough about the characters, but George will probably manage to drag me to the end of GoT. Damn you, George.
- technofiend 9y ago
  Yeah, I think I was blocking the show name like some childhood trauma. I was a little surprised JJ did such a poor job for backstory but then again don't get me started on his Star Trek reboots. :grumble:
  - tnecniv 9y ago
    I've seen probably a half dozen things JJ has worked on, but none of them end up being any good. He normally starts out with a pretty good premise, but it rapidly becomes apparent that he doesn't know what to do with it. Besides Star Trek (at least the first one, the second one is a different story) and Star Wars, Fringe and Super 8 jump to mind. Fortunately Stranger Things got to be what Super 8 could have been.
- e12e 9y ago
  Interestingly enough, they sort of save the hard sci-fi plot of revolution ; they just take a number of seasons to pull the origin of the nano-tech scourge out of the hat (yes, it was well intentioned, yes self-replicating nano-tech turned out to be a bad idea, yes there was a command and control feature built in).
lotsofpulp 9y ago
Not just for topics that you're knowledgeable on, but also if you pick up on the patterns that writers use or read tvtropes it also ruins the watch-ability. I can't watch anything made by broadcast TV or TBS/TNT/USA or that Marvel stuff, it's just so boring and repetitive. Although, I do wonder what it says about people that the most popular movies involve creating a fantasy where the main character has some type of calamity during childhood then goes on to save the world with superpowers.
- technofiend 9y ago
  >Although, I do wonder what it says about people that the most popular movies involve creating a fantasy where the main character has some type of calamity during childhood then goes on to save the world with superpowers. Lots of folks carrying childhood angst and wishing they could make things right, I guess.
- tnecniv 9y ago
  > Not just for topics that you're knowledgeable on, but also if you pick up on the patterns that writers use or read tvtropes it also ruins the watch-ability. This is true of story telling in general, though. There's really only so many story elements. That doesn't mean you can't write a new, great story with them today (just like there's still great music being made despite us only having 12 notes in Western music).

modalduality 9y ago

>It's a bit like writing a best selling novel in a language you don't speak: theoretically possible, but just so far from normal reality that it's hard to suspend disbelief. Jhumpa Lahiri managed to do this! See http://www.newyorker.com/magazine/2015/12/07/teach-yourself-italian http://www.newyorker.com/magazine/2015/12/07/teach-yourself-....

hannob 9y ago

So my feeling was similar, but I had a slightly different issue with it: Even if you had a breakthrough compression algorithm, it's highly unlikely you'd build a startup with it and sell it as a proprietary tech. There are plenty of better compression techs out there for various use cases (zstd, brotli, opus, ...), and pretty much all of them today are thrown out by some major corp as open source. Even giving the better compression tech out for free it's incredibly hard to get people to adopt it. I mean github still uses gzip for their source downloads for whatever unclear reason.

sanderjd 9y ago
That's why they had to build the platform!

Udik 9y ago

Exactly my problem. Moreover, technical aspects aside, it seems a very poor choice for a satire on silicon valley: data compression is hard science, more likely to come from university maths departments than from garage startups; it doesn't require much marketing either, or hype, or social networking, or even fancy websites. Given that the real SV offers endless examples of silly products and enterprises (just the first that come to mind: the app to send "YO" to your friends; the completely useless $400 juice squeezer with wifi connection) putting such a no-nonsense innovation like a novel data compression algorithm at the center of the show is already completely off target.

rusk 9y ago

There's an old story-tellers saying: "Don't let the truth get in the way of a good yarn". Abandon disbelief my friend, and immerse yourself in the theatre. Focus on the message, not the lettering.

empath75 9y ago

> As someone with PhD studies in Information Theory In order for them to come up with a business plan that made 100% logical sense to anyone with expertise on the topic, they'd have had to come up with a legitimate business plan, and at that point, they could have just launched a startup instead of making a sitcom.

tnecniv 9y ago
That would make for a great interview "Well, we wanted to write this show that was a satire of the culture in Silicon Valley, but in the process we revolutionized the field of information theory."

eduren 9y ago

While I agree that the compression breakthrough doesn't make much sense realistically, I think it makes a great premise for the show. The plot needs Richard to actually invent something revolutionary (and thus fail to get it off the ground time and time again), that is also simple enough for the audience to understand. Any algorithmic breakthrough that is real-world feasible would either need too much vague handwaving from the writing ("A brand new way to compute data!"). Or be close enough to reality that neither the layman nor the expert is happy (imagine the show having to explain why the big-O of Richard's new magic sorting algorithm matters). Compression is the logical choice because it's: 1) easier to explain why fitting more data into less space is significant (laymen already have some rough visualization of compression), 2) lends itself to being "black boxed" in the plot ("Look at all the companies we can build with this one breakthrough"), 3) just barely feasible enough that if some SV nerd actually did make the discovery, then events would play out roughly the same as the show depicts (winning techcrunch, getting funding)

monocasa 9y ago
And I'd add that most compressors make a lot of assumptions about the general case that reduce that real world compression ratio when compared to theoretical maximums. That's why stuff like kkrunchy can't be beat in their domains (first and last rule of optimization, "know your data inside and out!"). If someone was to stumble upon a heuristic that gives you noticeable gains for general compression as well, it could be close to as game changing as Richard's stuff is. Obviously we have a good handle on "theoretical maximums" thanks to information theory and so that end of the plot is BS, but looking past that embellishment the show isn't outside the realm of possibility.
thomastjeffery 9y ago
It's perfect, because there is one impossible plot device in the real world. The attention to detail used to accomplish that is very appealing to me. It's my very favorite aspect of the show.

Nimitz14 9y ago

How do you not realize for any subject you pick there'll be another PhD student like you bitching about it?

e12e 9y ago

I always felt that was part of the joke; young programmers chasing the gold at the end of the rainbow.

ericd 9y ago

I once sat on a plane next to someone who was an expert in glass. It ruined every action movie for him, because the glass always shattered all wrong. And sometimes they would cut holes in glass that you couldn't do that with. I'd try to suspend the disbelief :-)