4 ms·

The project is still underway, but I think decoding is more important than encoding, especially with live streams. Given a Twitch stream, for example, the encod

by Jhsto 9y ago

The project is still underway, but I think decoding is more important than encoding, especially with live streams. Given a Twitch stream, for example, the encoding is only done by one peer while the decoding is done by thousands or (as of recent) millions.

vanderZwan 9y ago
Counter-argument: encoding is the bottleneck in live-streaming
- nerdymanchild 9y ago
  Depends on how parallelizable the encoding algorithm is.
  - simcop2387 9y ago
    With a live stream, it's generally not parallelizable at all. You won't have future frames to work with unless you add a huge delay in the stream, which generally doesn't mesh well with doing something live.
    - nerdymanchild 9y ago
      5 minute delays are common on twitch, especially to circumvent stream snipers. Totally live video is a liability actually and I would guess most live broadcasts have a delay. In any case, I wasn't referring to inter-frame parallelizability, I was referring to intra-frame parallelizability which doesn't require a delay.
      - bitbang 9y ago
        Iter-frame parallelizability is made possible (at least in the context of h264) by using slices. It comes with a minor penalty in the resulting data compression ratio.
      - SahAssar 9y ago
        Twitch is just one example, and I'd assume most live video viewed would benefit from a lower delay. That goes for everything from security cameras to sports to video chat. I'd bet that for the vast majority of live video a lower delay would not be a "liability", but rather a plus.
      - cdash 9y ago
        I am not going to say it never happens but purposely induced delays are very rare. You have to remember that Twitch is a community and the people watching communicate with the streamer through chat. Getting a response to something you say in chat 5 minutes later is not a workable solution. Now, this works fine for large broadcasts where there is no two way communication between a streamer and their followers or in some type of competitive situations, but that is not the norm.
        nerdymanchild 9y ago
        All the streamers I watch respond to chat in digest form after finishing a game
- Jhsto 9y ago
  Please do elaborate.
  - djrogers 9y ago
    It can take an enormous amount of compute power to encode 4k video at 1:1 speed with these more complex codecs. Multiply for multiple bitrates and resolutions. If you can't encode at 1:1, you add significant delays to 'live' streaming, thus making it not 'live'. Case in point, a 40 second 1080p clip would take over 8 hours to encode on an i7-4800 - that's less than 2 frames per minute. You need a lot of horsepower to cut that down to 40s / 60 FPS. [1] https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/ https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live...
  - vanderZwan 9y ago
    Like the other comment said: the point is live streaming, so the encoding has to be done in real-time. Being the slower of the two steps (assuming comparable hardware between sender and recipient), it becomes the performance bottleneck when measured as frames per second. When measured in, say, total sum of all CPU cycles and hence total energy spent, then yes: the decoder is the bigger deal. So your argument holds for non-live videos.
- usefulcat 9y ago
  But you only have to encode once, or once per client type as opposed to per-client. And the more efficient decoding is, the wider the range of potential clients and/or the better the quality for a given amount of bandwidth.
  - vanderZwan 9y ago
    > But you only have to encode once In real-time. If the encoder is too slow for that it is the bottle-neck.
    - deleted 9y ago
      [deleted]
    - usefulcat 9y ago
      I was thinking of the case where the encoding is being done by a centralized 'broadcaster', as opposed to individual users streaming from their own machines; for the latter case I agree with you.
      - vanderZwan 9y ago
        Ah ok! I'm a bit sceptical of how well that would work in practice, though: 1. the uploader would have to quickly send data to the server, which requires fast encoding in a different format (so lower-quality than AV1, and introducing artefacts from a different lossy encoding) 2. this low quality then gets recompressed with a different codec with different artefacts, and possibly worse compression due to the artefacts introduced by the first codec When uploading to video sites that are not live, the idea is usually to use as high-quality an input as possible first, in which case this won't be much of a problem. But I might be mistaken on how bad this affects live-streaming! Perhaps the first codec throws out information in a way that smooths out the video, which then makes re-encoding with AV1 faster and have better compression with minimal extra loss of details.
    - chemag 9y ago
      Live-streaming is typically a type of broadcast. Some latency (a few or even tens of seconds) is typically OK.
shmerl 9y ago
Efficient encoding is very important for WebRTC and video communication in general. And AV1 is proposed for it. So hopefully some major improvements are planned.

by Jhsto 9y ago

vanderZwan 9y ago

Counter-argument: encoding is the bottleneck in live-streaming

nerdymanchild 9y ago
Depends on how parallelizable the encoding algorithm is.
- simcop2387 9y ago
  With a live stream, it's generally not parallelizable at all. You won't have future frames to work with unless you add a huge delay in the stream, which generally doesn't mesh well with doing something live.
  - nerdymanchild 9y ago
    5 minute delays are common on twitch, especially to circumvent stream snipers. Totally live video is a liability actually and I would guess most live broadcasts have a delay. In any case, I wasn't referring to inter-frame parallelizability, I was referring to intra-frame parallelizability which doesn't require a delay.
    - bitbang 9y ago
      Iter-frame parallelizability is made possible (at least in the context of h264) by using slices. It comes with a minor penalty in the resulting data compression ratio.
    - SahAssar 9y ago
      Twitch is just one example, and I'd assume most live video viewed would benefit from a lower delay. That goes for everything from security cameras to sports to video chat. I'd bet that for the vast majority of live video a lower delay would not be a "liability", but rather a plus.
    - cdash 9y ago
      I am not going to say it never happens but purposely induced delays are very rare. You have to remember that Twitch is a community and the people watching communicate with the streamer through chat. Getting a response to something you say in chat 5 minutes later is not a workable solution. Now, this works fine for large broadcasts where there is no two way communication between a streamer and their followers or in some type of competitive situations, but that is not the norm.
      - nerdymanchild 9y ago
        All the streamers I watch respond to chat in digest form after finishing a game
Jhsto 9y ago
Please do elaborate.
- djrogers 9y ago
  It can take an enormous amount of compute power to encode 4k video at 1:1 speed with these more complex codecs. Multiply for multiple bitrates and resolutions. If you can't encode at 1:1, you add significant delays to 'live' streaming, thus making it not 'live'. Case in point, a 40 second 1080p clip would take over 8 hours to encode on an i7-4800 - that's less than 2 frames per minute. You need a lot of horsepower to cut that down to 40s / 60 FPS. [1] https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/ https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live...
- vanderZwan 9y ago
  Like the other comment said: the point is live streaming, so the encoding has to be done in real-time. Being the slower of the two steps (assuming comparable hardware between sender and recipient), it becomes the performance bottleneck when measured as frames per second. When measured in, say, total sum of all CPU cycles and hence total energy spent, then yes: the decoder is the bigger deal. So your argument holds for non-live videos.
usefulcat 9y ago
But you only have to encode once, or once per client type as opposed to per-client. And the more efficient decoding is, the wider the range of potential clients and/or the better the quality for a given amount of bandwidth.
- vanderZwan 9y ago
  > But you only have to encode once In real-time. If the encoder is too slow for that it is the bottle-neck.
  - deleted 9y ago
    [deleted]
  - usefulcat 9y ago
    I was thinking of the case where the encoding is being done by a centralized 'broadcaster', as opposed to individual users streaming from their own machines; for the latter case I agree with you.
    - vanderZwan 9y ago
      Ah ok! I'm a bit sceptical of how well that would work in practice, though: 1. the uploader would have to quickly send data to the server, which requires fast encoding in a different format (so lower-quality than AV1, and introducing artefacts from a different lossy encoding) 2. this low quality then gets recompressed with a different codec with different artefacts, and possibly worse compression due to the artefacts introduced by the first codec When uploading to video sites that are not live, the idea is usually to use as high-quality an input as possible first, in which case this won't be much of a problem. But I might be mistaken on how bad this affects live-streaming! Perhaps the first codec throws out information in a way that smooths out the video, which then makes re-encoding with AV1 faster and have better compression with minimal extra loss of details.
  - chemag 9y ago
    Live-streaming is typically a type of broadcast. Some latency (a few or even tens of seconds) is typically OK.

shmerl 9y ago

Efficient encoding is very important for WebRTC and video communication in general. And AV1 is proposed for it. So hopefully some major improvements are planned.