5 ms·

So not a good format for live streaming then?

Jhsto 8y ago
The project is still underway, but I think decoding is more important than encoding, especially with live streams. Given a Twitch stream, for example, the encoding is only done by one peer while the decoding is done by thousands or (as of recent) millions.
- vanderZwan 8y ago
  Counter-argument: encoding is the bottleneck in live-streaming
  - nerdymanchild 8y ago
    Depends on how parallelizable the encoding algorithm is.
    - simcop2387 8y ago
      With a live stream, it's generally not parallelizable at all. You won't have future frames to work with unless you add a huge delay in the stream, which generally doesn't mesh well with doing something live.
      - nerdymanchild 8y ago
        5 minute delays are common on twitch, especially to circumvent stream snipers. Totally live video is a liability actually and I would guess most live broadcasts have a delay. In any case, I wasn't referring to inter-frame parallelizability, I was referring to intra-frame parallelizability which doesn't require a delay.
        bitbang 8y ago
        Iter-frame parallelizability is made possible (at least in the context of h264) by using slices. It comes with a minor penalty in the resulting data compression ratio.
        SahAssar 8y ago
        Twitch is just one example, and I'd assume most live video viewed would benefit from a lower delay. That goes for everything from security cameras to sports to video chat. I'd bet that for the vast majority of live video a lower delay would not be a "liability", but rather a plus.
        cdash 8y ago
        I am not going to say it never happens but purposely induced delays are very rare. You have to remember that Twitch is a community and the people watching communicate with the streamer through chat. Getting a response to something you say in chat 5 minutes later is not a workable solution. Now, this works fine for large broadcasts where there is no two way communication between a streamer and their followers or in some type of competitive situations, but that is not the norm.
        nerdymanchild 8y ago
        All the streamers I watch respond to chat in digest form after finishing a game
  - Jhsto 8y ago
    Please do elaborate.
    - djrogers 8y ago
      It can take an enormous amount of compute power to encode 4k video at 1:1 speed with these more complex codecs. Multiply for multiple bitrates and resolutions. If you can't encode at 1:1, you add significant delays to 'live' streaming, thus making it not 'live'. Case in point, a 40 second 1080p clip would take over 8 hours to encode on an i7-4800 - that's less than 2 frames per minute. You need a lot of horsepower to cut that down to 40s / 60 FPS. [1] https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/ https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live...
    - vanderZwan 8y ago
      Like the other comment said: the point is live streaming, so the encoding has to be done in real-time. Being the slower of the two steps (assuming comparable hardware between sender and recipient), it becomes the performance bottleneck when measured as frames per second. When measured in, say, total sum of all CPU cycles and hence total energy spent, then yes: the decoder is the bigger deal. So your argument holds for non-live videos.
  - usefulcat 8y ago
    But you only have to encode once, or once per client type as opposed to per-client. And the more efficient decoding is, the wider the range of potential clients and/or the better the quality for a given amount of bandwidth.
    - vanderZwan 8y ago
      > But you only have to encode once In real-time. If the encoder is too slow for that it is the bottle-neck.
      - deleted 8y ago
        [deleted]
      - usefulcat 8y ago
        I was thinking of the case where the encoding is being done by a centralized 'broadcaster', as opposed to individual users streaming from their own machines; for the latter case I agree with you.
        vanderZwan 8y ago
        Ah ok! I'm a bit sceptical of how well that would work in practice, though: 1. the uploader would have to quickly send data to the server, which requires fast encoding in a different format (so lower-quality than AV1, and introducing artefacts from a different lossy encoding) 2. this low quality then gets recompressed with a different codec with different artefacts, and possibly worse compression due to the artefacts introduced by the first codec When uploading to video sites that are not live, the idea is usually to use as high-quality an input as possible first, in which case this won't be much of a problem. But I might be mistaken on how bad this affects live-streaming! Perhaps the first codec throws out information in a way that smooths out the video, which then makes re-encoding with AV1 faster and have better compression with minimal extra loss of details.
      - chemag 8y ago
        Live-streaming is typically a type of broadcast. Some latency (a few or even tens of seconds) is typically OK.
- shmerl 8y ago
  Efficient encoding is very important for WebRTC and video communication in general. And AV1 is proposed for it. So hopefully some major improvements are planned.
vanderZwan 8y ago
For now
Nadya 8y ago
>The Twitch test set is entirely live-streamed video game content, and we see solid gains here as well. From the article, near the end.
clouddrover 8y ago
Bitmovin produced an AV1 live stream a year ago using 200 cores: https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/ https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live... And six months ago they had improved on that to do live streaming with 32 cores: https://bitmovin.com/constantly-evolving-video-landscape-display-ibc-2017/ https://bitmovin.com/constantly-evolving-video-landscape-dis... Twitch wants to use AV1 for live streaming. Here's a talk on the "switching frame" feature of AV1 for use in live streaming: https://www.youtube.com/watch?v=o5sJX6VA34o https://www.youtube.com/watch?v=o5sJX6VA34o
tigraine 8y ago
Bitmovin and Mozilla recently also did a Webinar about AV1: https://bitmovin.com/an-introduction-to-av1/ https://bitmovin.com/an-introduction-to-av1/

The project is still underway, but I think decoding is more important than encoding, especially with live streams. Given a Twitch stream, for example, the encoding is only done by one peer while the decoding is done by thousands or (as of recent) millions.

vanderZwan 8y ago
Counter-argument: encoding is the bottleneck in live-streaming
- nerdymanchild 8y ago
  Depends on how parallelizable the encoding algorithm is.
  - simcop2387 8y ago
    With a live stream, it's generally not parallelizable at all. You won't have future frames to work with unless you add a huge delay in the stream, which generally doesn't mesh well with doing something live.
    - nerdymanchild 8y ago
      5 minute delays are common on twitch, especially to circumvent stream snipers. Totally live video is a liability actually and I would guess most live broadcasts have a delay. In any case, I wasn't referring to inter-frame parallelizability, I was referring to intra-frame parallelizability which doesn't require a delay.
      - bitbang 8y ago
        Iter-frame parallelizability is made possible (at least in the context of h264) by using slices. It comes with a minor penalty in the resulting data compression ratio.
      - SahAssar 8y ago
        Twitch is just one example, and I'd assume most live video viewed would benefit from a lower delay. That goes for everything from security cameras to sports to video chat. I'd bet that for the vast majority of live video a lower delay would not be a "liability", but rather a plus.
      - cdash 8y ago
        I am not going to say it never happens but purposely induced delays are very rare. You have to remember that Twitch is a community and the people watching communicate with the streamer through chat. Getting a response to something you say in chat 5 minutes later is not a workable solution. Now, this works fine for large broadcasts where there is no two way communication between a streamer and their followers or in some type of competitive situations, but that is not the norm.
        nerdymanchild 8y ago
        All the streamers I watch respond to chat in digest form after finishing a game
- Jhsto 8y ago
  Please do elaborate.
  - djrogers 8y ago
    It can take an enormous amount of compute power to encode 4k video at 1:1 speed with these more complex codecs. Multiply for multiple bitrates and resolutions. If you can't encode at 1:1, you add significant delays to 'live' streaming, thus making it not 'live'. Case in point, a 40 second 1080p clip would take over 8 hours to encode on an i7-4800 - that's less than 2 frames per minute. You need a lot of horsepower to cut that down to 40s / 60 FPS. [1] https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/ https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live...
  - vanderZwan 8y ago
    Like the other comment said: the point is live streaming, so the encoding has to be done in real-time. Being the slower of the two steps (assuming comparable hardware between sender and recipient), it becomes the performance bottleneck when measured as frames per second. When measured in, say, total sum of all CPU cycles and hence total energy spent, then yes: the decoder is the bigger deal. So your argument holds for non-live videos.
- usefulcat 8y ago
  But you only have to encode once, or once per client type as opposed to per-client. And the more efficient decoding is, the wider the range of potential clients and/or the better the quality for a given amount of bandwidth.
  - vanderZwan 8y ago
    > But you only have to encode once In real-time. If the encoder is too slow for that it is the bottle-neck.
    - deleted 8y ago
      [deleted]
    - usefulcat 8y ago
      I was thinking of the case where the encoding is being done by a centralized 'broadcaster', as opposed to individual users streaming from their own machines; for the latter case I agree with you.
      - vanderZwan 8y ago
        Ah ok! I'm a bit sceptical of how well that would work in practice, though: 1. the uploader would have to quickly send data to the server, which requires fast encoding in a different format (so lower-quality than AV1, and introducing artefacts from a different lossy encoding) 2. this low quality then gets recompressed with a different codec with different artefacts, and possibly worse compression due to the artefacts introduced by the first codec When uploading to video sites that are not live, the idea is usually to use as high-quality an input as possible first, in which case this won't be much of a problem. But I might be mistaken on how bad this affects live-streaming! Perhaps the first codec throws out information in a way that smooths out the video, which then makes re-encoding with AV1 faster and have better compression with minimal extra loss of details.
    - chemag 8y ago
      Live-streaming is typically a type of broadcast. Some latency (a few or even tens of seconds) is typically OK.
shmerl 8y ago
Efficient encoding is very important for WebRTC and video communication in general. And AV1 is proposed for it. So hopefully some major improvements are planned.

vanderZwan 8y ago

For now

Nadya 8y ago

>The Twitch test set is entirely live-streamed video game content, and we see solid gains here as well. From the article, near the end.

clouddrover 8y ago

Bitmovin produced an AV1 live stream a year ago using 200 cores: https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/ https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live... And six months ago they had improved on that to do live streaming with 32 cores: https://bitmovin.com/constantly-evolving-video-landscape-display-ibc-2017/ https://bitmovin.com/constantly-evolving-video-landscape-dis... Twitch wants to use AV1 for live streaming. Here's a talk on the "switching frame" feature of AV1 for use in live streaming: https://www.youtube.com/watch?v=o5sJX6VA34o https://www.youtube.com/watch?v=o5sJX6VA34o

tigraine 8y ago

Bitmovin and Mozilla recently also did a Webinar about AV1: https://bitmovin.com/an-introduction-to-av1/ https://bitmovin.com/an-introduction-to-av1/