7 ms·

Pull quote: We run compute on the cloud and have no real-time requirements. I was asked by the Chair, “How much complexity increase is acceptable?” I was not

Pull quote: We run compute on the cloud and have no real-time requirements. I was asked by the Chair, “How much complexity increase is acceptable?” I was not prepared for the question, so did some quick math in my mind estimating an upper bound and said “At the worst case 100X.” The room of about a hundred video standardization experts burst out laughing. I looked at the Chair perplexed, and he says, “Don’t worry they are happy that they can try-out new things. People typically say 3X.” We were all immersed in the video codec space and yet my views surprised them and vice versa.

carbonatedmilk 8y ago
This makes entire sense to me: When you're going to stream the episode of Stranger things to 15million people, who really cares what the one-time cost to encode is? Surely you'd take algorithmic complexity that was 100x or even 1000x the baseline if it provided you with even a few kB of bandwidth savings?
- rayiner 8y ago
  Netflix is probably at the extreme end. Remember: the same codecs are used to compress the videos your cell phone takes.
  - haikuginger 8y ago
    Hardware implementations of codecs can be made that perform encode at low power if a certain level of quality/efficiency can be given up. This is already the case for AVC/HEVC on mobile devices where storage and power considerations overwhelm the possible quality and coding efficiency advances that could be available from a highly-intensive, highly-tunable CPU-based encode. In comparison, if you look at digital cinema, video is dumped as raw, unencoded pixels onto high-speed, high-power SSDs without any compression; this way, the original image can be manipulated losslessly before going through an intensive encode process for an optimal quality/space balance for end-user media delivery.
    - rayiner 8y ago
      Right, but designing a codec that is scalable in complexity to accommodate real-time hardware encoding on one end, and 100x Netflix encoding on the other end probably isn't easy.
      - Twirrim 8y ago
        Why does it have to be the same codec? Use the right tool for the job.
        Slartie 8y ago
        The encoding part may well differ, but having the same bitstream format and thus being able to use the same decoders for a very wide range of efficiency/time trade-offs has huge advantages. The necessity of using hardware implementations of video decoders in many scenarios requires a big initial investment into a new codecs' decoding part before even the first frame of video may be displayed by it. This is why pretty much anyone has standardized on h264, and now on h265, as the video codec of choice, despite using very different encoder software or hardware to realize hugely differing trade-offs on the encoding side of things.
    - tatersolid 8y ago
      The very popular RED digital cinema cameras have always shot in compressed form using a low-latency wavelet-based codec (presumably with some ASIC or GPU acceleration on board). http://www.red.com/learn/red-101/redcode-file-format http://www.red.com/learn/red-101/redcode-file-format http://www.theblackandblue.com/2012/05/02/shooting-red-epic-7/ http://www.theblackandblue.com/2012/05/02/shooting-red-epic-... What cameras shoot uncompressed 8K HDR video? That doesn’t even seem possible given current SSD bus speeds. Note that many high-quality lossless formats are actually still compressed (they subtract current frame data from the previous frame and then apply LZ4 for example).
      - vardump 8y ago
        > What cameras shoot uncompressed 8K HDR video? That doesn’t even seem possible given current SSD bus speeds. 12-bit bayer RGB matrix 8k @24 fps: 7680 * 4320 * 12bit * 24/s = 1.2 GB/s uncompressed. Currently SSDs go up to 3 GB/s. And you could have a RAID 0 array made out of multiple SSDs. So it is possible. Entirely another matter whether it makes sense. One hour of video at this bitrate requires 4.3 TB.
- lostcolony 8y ago
  You still care, since at some point the cost of the extra encoder-hours used is going to outpace the bandwidth savings. I'm guessing she mentally assumed 'at no bandwidth savings, how much margin do we have?', since the amount saved wasn't ever quantified.
  - martincmartin 8y ago
    It's not bandwidth savings. It's growing the user base. There are lots of people on low bandwidth connections. Maybe now they don't find Netflix watchable, because the video quality is to crappy. But if you can fit higher quality video into the same bandwidth, they'd become users. I agree with your overall point though, that at some point the extra cost for encode isn't worth it.
    - alephnil 8y ago
      Netflix is in fact working very hard to optimize the quality per bandwidth at low bitrates. Megha Manohara from Netflix had a conference talk about it last year: https://www.youtube.com/watch?v=S9E9xUbcYAM https://www.youtube.com/watch?v=S9E9xUbcYAM Their goal is to have video of acceptable quality at 250 kbit/s, which makes Netflix an option for a lot more people worldwide.
    - spenczar5 8y ago
      That's part of it, but I think you'd be astonished at how much IP transit can end up costing when you're distributing high-quality video. It's pretty staggering how much bandwidth can cost. I would not be surprised in the slightest if at least 1/4 of Netflix's total engineering spend is on network transit, so any savings that are per-video can have a huge impact. Source: I work at a company that distributes a lot of high-quality video :)
  - derf_ 8y ago
    There are a couple of things that make Netflix somewhat unique. 1) They have a fairly limited catalog (contrast with the constant ingestion of new content by Youtube, for example) 2) The cheapest way for them to ensure they have sufficient capacity to handle peak loads leaves them with a lot of extra compute during non-peak times. That excess compute is essentially free.
- bufferoverflow 8y ago
  Bandwidth is extremely cheap, so 15 million by a few kB is just 15+ GB, which costs pennies in terms of bandwidth. That doesn't buy you that much encoding compute time.
  - Yetanfou 8y ago
    Bandwidth can be cheap, all depending on the connection you're using. It can also be extremely expensive: [ Dutch ] https://www.telegraaf.nl/vrij/2167192/zoontje-duitsers-kijkt-films-tijdens-cruise-12-000-euro https://www.telegraaf.nl/vrij/2167192/zoontje-duitsers-kijkt... If your Dutch isn't up to this let me summarise: A German family took a boat trip from Kiel to Oslo. Their 12yo son watched a few movies (or clips, no idea) while on board. Total data consumption was 470 MB. They got a bill for €12.000,- which comes down to just over €25 per MB. Bandwidth can be very expensive as well. Of course this is an extreme case but any byte shaved off an encoded stream can end up saving some people a lot of money.
  - phlakaton 8y ago
    For you, perhaps, it's cheap. For the use cases they are talking about, it's expensive and limited. Agreed that, if all they can do is shave a "few kB" off a video, it's probably not worth the investment. But what if they can buy a 5–10% bandwidth improvement that actually looks better on the device they're targeting?
  - kemitche 8y ago
    Netflix isn't necessarily trying to reduce the bandwidth on _their_ side. They want to be able to deliver higher quality to users on lower bandwidth connections. The better that goes, the larger their prospective customer base.
    - dick_sucker2 8y ago
      That's a big issue in rural areas with microwave links. Everyone who has one of these links is limited by bandwidth and they all want to watch Netflix.
- maherbeg 8y ago
  Netflix also encodes to hundreds of combinations of bit rates and resolutions. If improving the quality for a given bitrate lets them reduce some of the complexity (by shrinking the combinations needed to serve users), then there are pretty nice advantages there.
nine_k 8y ago
Suddenly they're in the same space as 3D-rendered movies: they can spend a lot of time once to make the result good for a million of spectators. It, of course, only works if you render widely replicated material. Even an average Youtube video is likely not like that, to say nothing of the encoding done in a consumer-grade camera / phone.
- lev99 8y ago
  > encoding done in a consumer-grade camera / phone. Just brainstorming here, but encoding local photos/videos might be an acceptable task for a cellphone to do when it is fully charged and plugged in.
  - admax88q 8y ago
    That would severely limit the amount of photos and video you can capture between charging sessions. Most phones don't have a ton of storage, and raw video/photos are huge.
    - icebraining 8y ago
      Plus storage might not be fast enough to store raw video in realtime. And you'd kill it much sooner with all the writing (larger raw files + reencoded files).
jascenso 8y ago
Note that they are speaking about encoding complexity. The point is that decoder complexity increases are also needed (although, not as much !) and every time that encoder complexity has increased in the past, decoder complexity also grew. The really interesting question is how much decoder complexity increase is acceptable.
da_chicken 8y ago
I'm someone with an intermediate level of coding experience, but I have no idea what exactly is meant here by "encoding complexity". What's the difference between 3X and 100X? What's the encoding complexity of h.264 vs MPEG-2? Is it just saying that they're OK with the encoding algorithm taking 100 times as much processing power or taking 100 times as many processor cycles to compress a given data stream?

carbonatedmilk 8y ago

This makes entire sense to me: When you're going to stream the episode of Stranger things to 15million people, who really cares what the one-time cost to encode is? Surely you'd take algorithmic complexity that was 100x or even 1000x the baseline if it provided you with even a few kB of bandwidth savings?

rayiner 8y ago
Netflix is probably at the extreme end. Remember: the same codecs are used to compress the videos your cell phone takes.
- haikuginger 8y ago
  Hardware implementations of codecs can be made that perform encode at low power if a certain level of quality/efficiency can be given up. This is already the case for AVC/HEVC on mobile devices where storage and power considerations overwhelm the possible quality and coding efficiency advances that could be available from a highly-intensive, highly-tunable CPU-based encode. In comparison, if you look at digital cinema, video is dumped as raw, unencoded pixels onto high-speed, high-power SSDs without any compression; this way, the original image can be manipulated losslessly before going through an intensive encode process for an optimal quality/space balance for end-user media delivery.
  - rayiner 8y ago
    Right, but designing a codec that is scalable in complexity to accommodate real-time hardware encoding on one end, and 100x Netflix encoding on the other end probably isn't easy.
    - Twirrim 8y ago
      Why does it have to be the same codec? Use the right tool for the job.
      - Slartie 8y ago
        The encoding part may well differ, but having the same bitstream format and thus being able to use the same decoders for a very wide range of efficiency/time trade-offs has huge advantages. The necessity of using hardware implementations of video decoders in many scenarios requires a big initial investment into a new codecs' decoding part before even the first frame of video may be displayed by it. This is why pretty much anyone has standardized on h264, and now on h265, as the video codec of choice, despite using very different encoder software or hardware to realize hugely differing trade-offs on the encoding side of things.
  - tatersolid 8y ago
    The very popular RED digital cinema cameras have always shot in compressed form using a low-latency wavelet-based codec (presumably with some ASIC or GPU acceleration on board). http://www.red.com/learn/red-101/redcode-file-format http://www.red.com/learn/red-101/redcode-file-format http://www.theblackandblue.com/2012/05/02/shooting-red-epic-7/ http://www.theblackandblue.com/2012/05/02/shooting-red-epic-... What cameras shoot uncompressed 8K HDR video? That doesn’t even seem possible given current SSD bus speeds. Note that many high-quality lossless formats are actually still compressed (they subtract current frame data from the previous frame and then apply LZ4 for example).
    - vardump 8y ago
      > What cameras shoot uncompressed 8K HDR video? That doesn’t even seem possible given current SSD bus speeds. 12-bit bayer RGB matrix 8k @24 fps: 7680 * 4320 * 12bit * 24/s = 1.2 GB/s uncompressed. Currently SSDs go up to 3 GB/s. And you could have a RAID 0 array made out of multiple SSDs. So it is possible. Entirely another matter whether it makes sense. One hour of video at this bitrate requires 4.3 TB.
lostcolony 8y ago
You still care, since at some point the cost of the extra encoder-hours used is going to outpace the bandwidth savings. I'm guessing she mentally assumed 'at no bandwidth savings, how much margin do we have?', since the amount saved wasn't ever quantified.
- martincmartin 8y ago
  It's not bandwidth savings. It's growing the user base. There are lots of people on low bandwidth connections. Maybe now they don't find Netflix watchable, because the video quality is to crappy. But if you can fit higher quality video into the same bandwidth, they'd become users. I agree with your overall point though, that at some point the extra cost for encode isn't worth it.
  - alephnil 8y ago
    Netflix is in fact working very hard to optimize the quality per bandwidth at low bitrates. Megha Manohara from Netflix had a conference talk about it last year: https://www.youtube.com/watch?v=S9E9xUbcYAM https://www.youtube.com/watch?v=S9E9xUbcYAM Their goal is to have video of acceptable quality at 250 kbit/s, which makes Netflix an option for a lot more people worldwide.
  - spenczar5 8y ago
    That's part of it, but I think you'd be astonished at how much IP transit can end up costing when you're distributing high-quality video. It's pretty staggering how much bandwidth can cost. I would not be surprised in the slightest if at least 1/4 of Netflix's total engineering spend is on network transit, so any savings that are per-video can have a huge impact. Source: I work at a company that distributes a lot of high-quality video :)
- derf_ 8y ago
  There are a couple of things that make Netflix somewhat unique. 1) They have a fairly limited catalog (contrast with the constant ingestion of new content by Youtube, for example) 2) The cheapest way for them to ensure they have sufficient capacity to handle peak loads leaves them with a lot of extra compute during non-peak times. That excess compute is essentially free.
bufferoverflow 8y ago
Bandwidth is extremely cheap, so 15 million by a few kB is just 15+ GB, which costs pennies in terms of bandwidth. That doesn't buy you that much encoding compute time.
- Yetanfou 8y ago
  Bandwidth can be cheap, all depending on the connection you're using. It can also be extremely expensive: [ Dutch ] https://www.telegraaf.nl/vrij/2167192/zoontje-duitsers-kijkt-films-tijdens-cruise-12-000-euro https://www.telegraaf.nl/vrij/2167192/zoontje-duitsers-kijkt... If your Dutch isn't up to this let me summarise: A German family took a boat trip from Kiel to Oslo. Their 12yo son watched a few movies (or clips, no idea) while on board. Total data consumption was 470 MB. They got a bill for €12.000,- which comes down to just over €25 per MB. Bandwidth can be very expensive as well. Of course this is an extreme case but any byte shaved off an encoded stream can end up saving some people a lot of money.
- phlakaton 8y ago
  For you, perhaps, it's cheap. For the use cases they are talking about, it's expensive and limited. Agreed that, if all they can do is shave a "few kB" off a video, it's probably not worth the investment. But what if they can buy a 5–10% bandwidth improvement that actually looks better on the device they're targeting?
- kemitche 8y ago
  Netflix isn't necessarily trying to reduce the bandwidth on _their_ side. They want to be able to deliver higher quality to users on lower bandwidth connections. The better that goes, the larger their prospective customer base.
  - dick_sucker2 8y ago
    That's a big issue in rural areas with microwave links. Everyone who has one of these links is limited by bandwidth and they all want to watch Netflix.
maherbeg 8y ago
Netflix also encodes to hundreds of combinations of bit rates and resolutions. If improving the quality for a given bitrate lets them reduce some of the complexity (by shrinking the combinations needed to serve users), then there are pretty nice advantages there.

nine_k 8y ago

Suddenly they're in the same space as 3D-rendered movies: they can spend a lot of time once to make the result good for a million of spectators. It, of course, only works if you render widely replicated material. Even an average Youtube video is likely not like that, to say nothing of the encoding done in a consumer-grade camera / phone.

lev99 8y ago
> encoding done in a consumer-grade camera / phone. Just brainstorming here, but encoding local photos/videos might be an acceptable task for a cellphone to do when it is fully charged and plugged in.
- admax88q 8y ago
  That would severely limit the amount of photos and video you can capture between charging sessions. Most phones don't have a ton of storage, and raw video/photos are huge.
  - icebraining 8y ago
    Plus storage might not be fast enough to store raw video in realtime. And you'd kill it much sooner with all the writing (larger raw files + reencoded files).

jascenso 8y ago

Note that they are speaking about encoding complexity. The point is that decoder complexity increases are also needed (although, not as much !) and every time that encoder complexity has increased in the past, decoder complexity also grew. The really interesting question is how much decoder complexity increase is acceptable.

da_chicken 8y ago

I'm someone with an intermediate level of coding experience, but I have no idea what exactly is meant here by "encoding complexity". What's the difference between 3X and 100X? What's the encoding complexity of h.264 vs MPEG-2? Is it just saying that they're OK with the encoding algorithm taking 100 times as much processing power or taking 100 times as many processor cycles to compress a given data stream?