3 ms·

Thank you for this informative answer. :- ) I guess a complementary question is in order: since you believe that the WebRTC protocol itself is not incompatible

Thank you for this informative answer. :- ) I guess a complementary question is in order: since you believe that the WebRTC protocol itself is not incompatible with the mechanisms that enable Salsify's performance, how much work would it be to adapt Salsify incrementally into a WebRTC implementation? (Ignoring mandatory codec support for a moment). I've been developing an Opus codec mode for Bluetooth A2DP, and I have a similar sort of situation. I've been looking (casually) in to using more interesting rate control based on channel performance to improve QoS with Bluetooth A2DP. I think Opus has a property you could only dream of: strict limits on frame size. :- ) Added: just found that the video codec uses the VP8 bitstream, that was not immediately clear to me; seems to me that that is a substantial selling point that should be made more clear on the webpage. Since your interface doesn't require any change to the bitstream of a conventional codec, it's a heck of a lot closer to public use than I initially thought!

keithwinstein 7y ago
This is an interesting question! I think getting our codebase to use the WebRTC framing and setup would probably not be that hard. We could probably use the same libraries that WebRTC.org is built on (libjingle, etc.), and maybe even just take a lot of their code. I don't think that means that a Salsify sender would be interoperable with, like, a Chrome receiver, though (even though we're just using VP8) -- we'd have to implement at least the receiver side of Salsify's congestion-control protocol inside WebRTC.org/Chrome, and Salsify sometimes likes to encode a VP8 frame that has to be interpreted relative to a certain (prior) decoder state. On #2, honestly I've worked with libopus a bit for Puffer and it's just really pleasant to work with. As you say, a lot of the difficulties of interacting with a video encoder you just don't have in this context. It's also pretty easy to get "gapless/clickless" back-to-back playback of audio excerpts that were encoded completely independently, unlike with video where this is a huge pain in the neck and usually requires a SAP/IDR/closed GOP/sequence header+I-frame+P-frame (which takes a lot of bytes so you can't do it very often). See https://github.com/StanfordSNR/puffer/blob/master/src/opus-encoder/opus-encoder.cc https://github.com/StanfordSNR/puffer/blob/master/src/opus-e... if you are interested for more.

keithwinstein 7y ago

This is an interesting question! I think getting our codebase to use the WebRTC framing and setup would probably not be that hard. We could probably use the same libraries that WebRTC.org is built on (libjingle, etc.), and maybe even just take a lot of their code. I don't think that means that a Salsify sender would be interoperable with, like, a Chrome receiver, though (even though we're just using VP8) -- we'd have to implement at least the receiver side of Salsify's congestion-control protocol inside WebRTC.org/Chrome, and Salsify sometimes likes to encode a VP8 frame that has to be interpreted relative to a certain (prior) decoder state. On #2, honestly I've worked with libopus a bit for Puffer and it's just really pleasant to work with. As you say, a lot of the difficulties of interacting with a video encoder you just don't have in this context. It's also pretty easy to get "gapless/clickless" back-to-back playback of audio excerpts that were encoded completely independently, unlike with video where this is a huge pain in the neck and usually requires a SAP/IDR/closed GOP/sequence header+I-frame+P-frame (which takes a lot of bytes so you can't do it very often). See https://github.com/StanfordSNR/puffer/blob/master/src/opus-encoder/opus-encoder.cc https://github.com/StanfordSNR/puffer/blob/master/src/opus-e... if you are interested for more.