5 ms·

Show HN: AV1 and WebRTC

72 points by Sean-Der 5y ago

Sean-Der 5y ago
I just finished adding AV1 support to Pion WebRTC. What AV1 is able to do is pretty amazing! I spent a lot of my time with H264 because of legacy devices/hardware acceleration. Seeing the jump in quality + reduction in bandwidth usage it is going to be hard to go back.
- out_of_protocol 5y ago
  Did you try VP9? Huge jump in quality as well. Slightly worser than AV1, available for ages (not apple though?)
  - judge2020 5y ago
    As far as I can tell, vp9 hardware decoding should be available[0][1], but safari will fallback to software decoding[2]. I can't find much of a definitive source for Apple's supported codecs. 0: https://9to5mac.com/2020/06/24/apple-adds-webp-hdr-support-and-more-to-safari-with-ios-14-and-macos-big-sur/ https://9to5mac.com/2020/06/24/apple-adds-webp-hdr-support-a... 1: https://en.wikipedia.org/wiki/Apple_M1#Other_features https://en.wikipedia.org/wiki/Apple_M1#Other_features 2: https://trac.webkit.org/changeset/284523/webkit/ https://trac.webkit.org/changeset/284523/webkit/
  - alphabettsy 5y ago
    VP9 isn’t better than HEVC though is it? And HEVC has hardware encoding and decoding support on most modern devices.
    - ww520 5y ago
      HEVC is loaded with patents. VP9 is Google’s response to avoid the minefield.
      - giantrobot 5y ago
        The device's hardware encoders/decoders have licenses already. Unless you're shipping your own you're not likely have to pay any royalties. If you're so successful the MPEG-LA comes a' knockin' you should have the money to pay the royalties. The whole purpose of the MPEG-LA is to be a single entity to approach about licenses held by a bunch of companies. Avoiding paying licenses means avoiding codecs well supported by mobile hardware.
        out_of_protocol 5y ago
        1. For HEVC it's not a single patent pool but many, most of them don't have clearly defined pricing 2. Firefox don't support hevc nor it will be in the future P.S. https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#vp9 https://developer.mozilla.org/en-US/docs/Web/Media/Formats/V... VP9 is supported by all major browsers
        alphabettsy 5y ago
        Sure, but few devices have VP9 hardware encoding which means re-encoding to view or less efficient software decoding.
        cma 5y ago
        You end up with artificially limited numbers of encode streams and stuff with that, at least with nvenc (I think they doubled the limit at the beginning of the pandemic though).
simulate-me 5y ago
What is the target use-case of pion? Until AV1 support is added to libwebrtc, how can this be used that wouldn't be better served by using RTP directly?
- Sean-Der 5y ago
  AV1 support is already available in libwebrtc! So when I started building Pion the target use case was to make it easier to build scalable servers. Instead of interacting with a WebRTC servers REST API to query information/load balance I wanted to have it all in one code base. It also is really useful to have Media+Transport decoupled. Lots of use cases I didn't realize grew out of that. * Teleoperation/robotics (https://github.com/Ragnar-H/TelloGo https://github.com/Ragnar-H/TelloGo) * Control remote software (https://github.com/m1k1o/neko https://github.com/m1k1o/neko) * Cross platform file sharing (https://github.com/saljam/webwormhole https://github.com/saljam/webwormhole) * Sending pre-recorded media (RTMP/HLS/RTSP -> WebRTC) * Custom DataChannel servers/bridges (https://snowflake.torproject.org/ https://snowflake.torproject.org/) Lots of other cool ones in https://github.com/pion/awesome-pion https://github.com/pion/awesome-pion I need to update it. It has been a bit since I have looked through https://pkg.go.dev/github.com/pion/webrtc/v3?tab=importedby https://pkg.go.dev/github.com/pion/webrtc/v3?tab=importedby
  - simulate-me 5y ago
    Thanks that's helpful. Does pion support RTCP? I've tried sending pre-recorded media over WebRTC before. The issue I ran into is that I needed to send keyframes too often leading to a bitrate that was too high for users with slower internet connections. For whatever reason, not all browsers seemed to support keyframe intra refresh, so there were large bitrate spikes whenever a keyframe was sent. RTCP would have solved this problem by allowing clients to request additional keyframes when needed rather than using a high keyframe interval.
    - Sean-Der 5y ago
      Yes it does support RTCP. https://github.com/pion/rtcp https://github.com/pion/rtcp We also provide https://github.com/pion/interceptor https://github.com/pion/interceptor. Interceptor is implementations of common RTCP workflows. Things like Congestion Feedback, NACK generation and Congestion Controllers. The idea is that you can use the ones we provide, or use your own!
HenrikB 5y ago
"AOMedia Video 1 (AV1) is an open, royalty-free video coding format initially designed for video transmissions over the Internet." [0] [0] https://en.wikipedia.org/wiki/AV1 https://en.wikipedia.org/wiki/AV1

72 points by Sean-Der 5y ago

I just finished adding AV1 support to Pion WebRTC. What AV1 is able to do is pretty amazing! I spent a lot of my time with H264 because of legacy devices/hardware acceleration. Seeing the jump in quality + reduction in bandwidth usage it is going to be hard to go back.

out_of_protocol 5y ago
Did you try VP9? Huge jump in quality as well. Slightly worser than AV1, available for ages (not apple though?)
- judge2020 5y ago
  As far as I can tell, vp9 hardware decoding should be available[0][1], but safari will fallback to software decoding[2]. I can't find much of a definitive source for Apple's supported codecs. 0: https://9to5mac.com/2020/06/24/apple-adds-webp-hdr-support-and-more-to-safari-with-ios-14-and-macos-big-sur/ https://9to5mac.com/2020/06/24/apple-adds-webp-hdr-support-a... 1: https://en.wikipedia.org/wiki/Apple_M1#Other_features https://en.wikipedia.org/wiki/Apple_M1#Other_features 2: https://trac.webkit.org/changeset/284523/webkit/ https://trac.webkit.org/changeset/284523/webkit/
- alphabettsy 5y ago
  VP9 isn’t better than HEVC though is it? And HEVC has hardware encoding and decoding support on most modern devices.
  - ww520 5y ago
    HEVC is loaded with patents. VP9 is Google’s response to avoid the minefield.
    - giantrobot 5y ago
      The device's hardware encoders/decoders have licenses already. Unless you're shipping your own you're not likely have to pay any royalties. If you're so successful the MPEG-LA comes a' knockin' you should have the money to pay the royalties. The whole purpose of the MPEG-LA is to be a single entity to approach about licenses held by a bunch of companies. Avoiding paying licenses means avoiding codecs well supported by mobile hardware.
      - out_of_protocol 5y ago
        1. For HEVC it's not a single patent pool but many, most of them don't have clearly defined pricing 2. Firefox don't support hevc nor it will be in the future P.S. https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#vp9 https://developer.mozilla.org/en-US/docs/Web/Media/Formats/V... VP9 is supported by all major browsers
        alphabettsy 5y ago
        Sure, but few devices have VP9 hardware encoding which means re-encoding to view or less efficient software decoding.
      - cma 5y ago
        You end up with artificially limited numbers of encode streams and stuff with that, at least with nvenc (I think they doubled the limit at the beginning of the pandemic though).

simulate-me 5y ago

What is the target use-case of pion? Until AV1 support is added to libwebrtc, how can this be used that wouldn't be better served by using RTP directly?

Sean-Der 5y ago
AV1 support is already available in libwebrtc! So when I started building Pion the target use case was to make it easier to build scalable servers. Instead of interacting with a WebRTC servers REST API to query information/load balance I wanted to have it all in one code base. It also is really useful to have Media+Transport decoupled. Lots of use cases I didn't realize grew out of that. * Teleoperation/robotics (https://github.com/Ragnar-H/TelloGo https://github.com/Ragnar-H/TelloGo) * Control remote software (https://github.com/m1k1o/neko https://github.com/m1k1o/neko) * Cross platform file sharing (https://github.com/saljam/webwormhole https://github.com/saljam/webwormhole) * Sending pre-recorded media (RTMP/HLS/RTSP -> WebRTC) * Custom DataChannel servers/bridges (https://snowflake.torproject.org/ https://snowflake.torproject.org/) Lots of other cool ones in https://github.com/pion/awesome-pion https://github.com/pion/awesome-pion I need to update it. It has been a bit since I have looked through https://pkg.go.dev/github.com/pion/webrtc/v3?tab=importedby https://pkg.go.dev/github.com/pion/webrtc/v3?tab=importedby
- simulate-me 5y ago
  Thanks that's helpful. Does pion support RTCP? I've tried sending pre-recorded media over WebRTC before. The issue I ran into is that I needed to send keyframes too often leading to a bitrate that was too high for users with slower internet connections. For whatever reason, not all browsers seemed to support keyframe intra refresh, so there were large bitrate spikes whenever a keyframe was sent. RTCP would have solved this problem by allowing clients to request additional keyframes when needed rather than using a high keyframe interval.
  - Sean-Der 5y ago
    Yes it does support RTCP. https://github.com/pion/rtcp https://github.com/pion/rtcp We also provide https://github.com/pion/interceptor https://github.com/pion/interceptor. Interceptor is implementations of common RTCP workflows. Things like Congestion Feedback, NACK generation and Congestion Controllers. The idea is that you can use the ones we provide, or use your own!

HenrikB 5y ago

"AOMedia Video 1 (AV1) is an open, royalty-free video coding format initially designed for video transmissions over the Internet." [0] [0] https://en.wikipedia.org/wiki/AV1 https://en.wikipedia.org/wiki/AV1