5 ms·

> why does Netflix need X thousand engineers, I could build it in a weekend I would like to hope nobody asks that. Video is the one of the, if the not the hard

> why does Netflix need X thousand engineers, I could build it in a weekend I would like to hope nobody asks that. Video is the one of the, if the not the hardest data plumbing use-case on the internet.

dragontamer 4y ago
I'd say realtime communications is harder. A lot of these tricks being discussed here cannot be applied to Skype calls.
- OJFord 4y ago
  Surely GP would agree, unless you mean even audio-only calls? Otherwise it's just an extra requirement(s) on top of 'video'.
  - jon-wood 4y ago
    Streaming pre-recorded video and streaming realtime video are almost entirely different use cases. Pre-recorded video streaming is, under the hood, really just a high-volume variant of serving up static web pages. You have a few gigabytes of file to send from the server its stored on to the device that wants to play back the video. As this presentation demonstrates that isn't trivial at scale, but the core functionality of sending files over the internet is what it was designed to do from day one. Because you can generally download video across the internet faster than it can be played back its possible to build up a decent sized buffer which allows you to paper over temporary variance in network performance without the customer noticing. Realtime video streaming has two variants. One to many Twitch style video streaming is relatively simple, since you can encode video into files and upload them to a server for people watching to download those files. This is how HLS streaming works, and most of the techniques Netflix use to optimise video delivery can also be applied here at the cost of adding latency between the event being streamed and people consuming it. That latency will often sit at about 30 seconds, and people generally find that acceptable. Skype style realtime video streaming is much harder. You're taking video from one person's camera, and then sending it over the internet to one or more people's device. You can't do any sort of pre-processing on that, or stage the video on servers closer to the consuming users, because you have no way of generating that video until the point your users decide to start talking to each other. Because you can't pre-stage that video you need to be able to establish a network route between the people on a call, potentially in an environment where none of the participants have any open connection from the internet directly to the device they're streaming from. Slight fluctuations in network performance can potentially degrade video delivery to the point of it being unusable. The most common route to deal with that is systems that attempt to establish a direct connection (ideally over a local network) between participants, and if that doesn't work going via relay servers operated by the software provider. These servers provide a single point on the internet all parties can connect to, and then allow passing packets as if they were all on the same network.
    - fomine3 4y ago
      > Skype style realtime video streaming is much harder. I agree but since some optimization is theoretically impossible, perhaps it can be said that optimizable area is much smaller than other services. YouTube (many many videos, high quality, many livestreams, and supports VR) seems to most difficult service to fully optimize.
  - eru 4y ago
    > Otherwise it's just an extra requirement(s) on top of 'video'. I'm not sure what you mean? Real time communication, both video and audio-only, have much lower latency requirements. You can't just buffer ahead when you have some spare bandwidth, like Netflix or YouTube can.
    - OJFord 4y ago
      Yeah that's what I'm kind of facetiously calling 'just an extra requirement'. My point was intended to be that there's the same challenges and more - but it's not something I've thought about in depth (and certainly not had to work with), it maybe wasn't a very good characterisation because it's not the same on the other side either, no large file to serve because at the start of the call it doesn't exist yet for example, so perhaps I take it back.
      - eru 4y ago
        Yeah, jon-wood really did a great write-up of the challenges involved. In any case, it's hard to say what the 'greatest' engineering challenge is. You can make almost any kind of engineering really challenging, if you (or the market..) sets yourself a very low cost-ceiling.
      - bombcar 4y ago
        Video calling is "easier" in that the p2p option is workable for some variant of "Works". It is much much harder because you can't do the "cache everything on the edge" solution. If storage was infinitely cheap and small, Netflix could run their entire business by sending a USB stick with every single movie/tv show they have on it encrypted to you, and everything would play locally. This is basically what they do with their edge servers/CDNs. You can't do that with video calls, because the video/audio didn't exist 1 millisecond ago.
  - dragontamer 4y ago
    The amount of transcoding needed to get a conference call up hurts my brain. If 20 people are talking on Skype, the server needs to receive those 20 streams, decode them, mix the audio together, recode the streams, and then broadcast it back out to all 20 people. I'm not a telecommunications guy, but I had some professors back in college explain how difficult and fundamental the research of "ma-bell" was from the 60s through 80s. I'm talking Erlang, C++, CLOS circuits, etc. etc. The innovations from Bell Labs are nearly endless. Telephone communications is one of the biggest sources of fundamental comp. sci research over the 1950s through 2000s.
    - nordsieck 4y ago
      > I'm talking Erlang, C++, CLOS circuits, etc. etc. The innovations from Bell Labs are nearly endless. A lot of innovations did come out of Bell Labs. But I'm pretty sure Erlang wasn't one of them.
      - dragontamer 4y ago
        Oh, you're right. There seems to have been a glitch in my memory somehow. Still, its Ericsson, which is telecommunications nonetheless.
    - eru 4y ago
      See also Claude Shannon's trifecta of compression, error correcting codes, and cryptography, which also came out of the study of telecommunications. Compression removes redundancy from your message. Error correcting codes introduces carefully controlled redundancies, so your message survives transmission errors, so your recipient can read it. And cryptography deals with 'scrambling' your message, so that no one else can read it (and also deal with authentication etc).

dragontamer 4y ago

I'd say realtime communications is harder. A lot of these tricks being discussed here cannot be applied to Skype calls.

OJFord 4y ago
Surely GP would agree, unless you mean even audio-only calls? Otherwise it's just an extra requirement(s) on top of 'video'.
- jon-wood 4y ago
  Streaming pre-recorded video and streaming realtime video are almost entirely different use cases. Pre-recorded video streaming is, under the hood, really just a high-volume variant of serving up static web pages. You have a few gigabytes of file to send from the server its stored on to the device that wants to play back the video. As this presentation demonstrates that isn't trivial at scale, but the core functionality of sending files over the internet is what it was designed to do from day one. Because you can generally download video across the internet faster than it can be played back its possible to build up a decent sized buffer which allows you to paper over temporary variance in network performance without the customer noticing. Realtime video streaming has two variants. One to many Twitch style video streaming is relatively simple, since you can encode video into files and upload them to a server for people watching to download those files. This is how HLS streaming works, and most of the techniques Netflix use to optimise video delivery can also be applied here at the cost of adding latency between the event being streamed and people consuming it. That latency will often sit at about 30 seconds, and people generally find that acceptable. Skype style realtime video streaming is much harder. You're taking video from one person's camera, and then sending it over the internet to one or more people's device. You can't do any sort of pre-processing on that, or stage the video on servers closer to the consuming users, because you have no way of generating that video until the point your users decide to start talking to each other. Because you can't pre-stage that video you need to be able to establish a network route between the people on a call, potentially in an environment where none of the participants have any open connection from the internet directly to the device they're streaming from. Slight fluctuations in network performance can potentially degrade video delivery to the point of it being unusable. The most common route to deal with that is systems that attempt to establish a direct connection (ideally over a local network) between participants, and if that doesn't work going via relay servers operated by the software provider. These servers provide a single point on the internet all parties can connect to, and then allow passing packets as if they were all on the same network.
  - fomine3 4y ago
    > Skype style realtime video streaming is much harder. I agree but since some optimization is theoretically impossible, perhaps it can be said that optimizable area is much smaller than other services. YouTube (many many videos, high quality, many livestreams, and supports VR) seems to most difficult service to fully optimize.
- eru 4y ago
  > Otherwise it's just an extra requirement(s) on top of 'video'. I'm not sure what you mean? Real time communication, both video and audio-only, have much lower latency requirements. You can't just buffer ahead when you have some spare bandwidth, like Netflix or YouTube can.
  - OJFord 4y ago
    Yeah that's what I'm kind of facetiously calling 'just an extra requirement'. My point was intended to be that there's the same challenges and more - but it's not something I've thought about in depth (and certainly not had to work with), it maybe wasn't a very good characterisation because it's not the same on the other side either, no large file to serve because at the start of the call it doesn't exist yet for example, so perhaps I take it back.
    - eru 4y ago
      Yeah, jon-wood really did a great write-up of the challenges involved. In any case, it's hard to say what the 'greatest' engineering challenge is. You can make almost any kind of engineering really challenging, if you (or the market..) sets yourself a very low cost-ceiling.
    - bombcar 4y ago
      Video calling is "easier" in that the p2p option is workable for some variant of "Works". It is much much harder because you can't do the "cache everything on the edge" solution. If storage was infinitely cheap and small, Netflix could run their entire business by sending a USB stick with every single movie/tv show they have on it encrypted to you, and everything would play locally. This is basically what they do with their edge servers/CDNs. You can't do that with video calls, because the video/audio didn't exist 1 millisecond ago.
- dragontamer 4y ago
  The amount of transcoding needed to get a conference call up hurts my brain. If 20 people are talking on Skype, the server needs to receive those 20 streams, decode them, mix the audio together, recode the streams, and then broadcast it back out to all 20 people. I'm not a telecommunications guy, but I had some professors back in college explain how difficult and fundamental the research of "ma-bell" was from the 60s through 80s. I'm talking Erlang, C++, CLOS circuits, etc. etc. The innovations from Bell Labs are nearly endless. Telephone communications is one of the biggest sources of fundamental comp. sci research over the 1950s through 2000s.
  - nordsieck 4y ago
    > I'm talking Erlang, C++, CLOS circuits, etc. etc. The innovations from Bell Labs are nearly endless. A lot of innovations did come out of Bell Labs. But I'm pretty sure Erlang wasn't one of them.
    - dragontamer 4y ago
      Oh, you're right. There seems to have been a glitch in my memory somehow. Still, its Ericsson, which is telecommunications nonetheless.
  - eru 4y ago
    See also Claude Shannon's trifecta of compression, error correcting codes, and cryptography, which also came out of the study of telecommunications. Compression removes redundancy from your message. Error correcting codes introduces carefully controlled redundancies, so your message survives transmission errors, so your recipient can read it. And cryptography deals with 'scrambling' your message, so that no one else can read it (and also deal with authentication etc).