3 ms·

We use a technique called simulcast. It consists on making every participant "work a bit harder" for the good of the bunch. That is, every participant sends 3

by saghul 7y ago

We use a technique called simulcast. It consists on making every participant "work a bit harder" for the good of the bunch. That is, every participant sends 3 separate video resolutions to the server: 720p, 480p and 180p (this may change due to bandwidth constraints). Then the server will only forward the approopriate layer to each other participant. So, if you are only seeing me in a thumbnail it will only forward the 180p layer. If I become the active speaker (or you choose to pin me to the large view) the server will immediately switch to forwarding the 720p layer. In addtion, we use SVC with temporal layers, so thumbnails may be given at just 15fps or even less. We do have a trick up our sleeve (but IIRC it haad to be disabled for the time being) which involves disabling the ssimulcast layers that nobody is requesting. That is, if nobody iss seeing me in the large view, why send the 720p layer at all? Hope that helps!

applecrazy 7y ago
Naive (and perhaps stupid) question here, why don't you request just the 720p feed from each participant and then rescale the streams as clients need them?
- endgame 7y ago
  The server would have to downscale the received stream. That's at least one round of downscaling for each participant, possibly more if you want to send different quality versions of the same stream down to different participants.
  - folmar 7y ago
    And while your device is quite capable of rescaling your stream -- even a lowest-tier smartphone will do it with no strain on its GPU -- its much more of a strain on the server to rescale 100's/1000's of stream simultaneously.
    - nerpderp82 7y ago
      Ram bandwidth is easier to manage and provision than physical bandwidth.
    - littlestymaar 7y ago
      Additionally: AFAIK you can't easily rescale an encoded stream: you need to decode it, rescale it and then re-encode it. For every single stream! That would be horribly computationally expensive.
gregmac 7y ago
SVC spatial and quality layers [1] sound like a really good solution to the bandwidth issue. From my (extremely limited) understanding, basically if you skip certain packets you get a lower resolution/quality stream. A client sends a single stream at the best quality it can tolerate, and then the SFU can forward whichever layers to each client depending on what resolution that client wants. What's the state of this in jitsi? I can only find limited info about SVC [2] and that is only on temporal layer. How much bandwidth does it even save in practice - maybe it's not worth the complexity trade-off? [1] http://webrtchacks.staging.wpengine.com/chrome-vp9-svc/ http://webrtchacks.staging.wpengine.com/chrome-vp9-svc/ [2] https://github.com/jitsi/jitsi-videobridge/blob/master/doc/svc.md https://github.com/jitsi/jitsi-videobridge/blob/master/doc/s...
littlestymaar 7y ago
Thanks for your interesting answer. It doesn't really address my question though, so I will rephrase it: If I take a 720p video of a webcam and encode it to be delivered as progressive live streaming, the resulting stream is going to be less than one Mbps : because the image doesn't move much, I don't need many key-frame, one every 4 seconds is more than enough, and I've seen streams with no more than one every 30s (live streaming of harbours' CCTV cameras, don't ask me why). But of course it won't be realtime either, and you're gonna have a few seconds of latency. While this is OK for live streaming, it is certainly not for a video chat room. What I'd like to know is why the latency requirement reduces that much the encoding efficiency. Do you have an idea ?

by saghul 7y ago

applecrazy 7y ago

Naive (and perhaps stupid) question here, why don't you request just the 720p feed from each participant and then rescale the streams as clients need them?

endgame 7y ago
The server would have to downscale the received stream. That's at least one round of downscaling for each participant, possibly more if you want to send different quality versions of the same stream down to different participants.
- folmar 7y ago
  And while your device is quite capable of rescaling your stream -- even a lowest-tier smartphone will do it with no strain on its GPU -- its much more of a strain on the server to rescale 100's/1000's of stream simultaneously.
  - nerpderp82 7y ago
    Ram bandwidth is easier to manage and provision than physical bandwidth.
  - littlestymaar 7y ago
    Additionally: AFAIK you can't easily rescale an encoded stream: you need to decode it, rescale it and then re-encode it. For every single stream! That would be horribly computationally expensive.

gregmac 7y ago

SVC spatial and quality layers [1] sound like a really good solution to the bandwidth issue. From my (extremely limited) understanding, basically if you skip certain packets you get a lower resolution/quality stream. A client sends a single stream at the best quality it can tolerate, and then the SFU can forward whichever layers to each client depending on what resolution that client wants. What's the state of this in jitsi? I can only find limited info about SVC [2] and that is only on temporal layer. How much bandwidth does it even save in practice - maybe it's not worth the complexity trade-off? [1] http://webrtchacks.staging.wpengine.com/chrome-vp9-svc/ http://webrtchacks.staging.wpengine.com/chrome-vp9-svc/ [2] https://github.com/jitsi/jitsi-videobridge/blob/master/doc/svc.md https://github.com/jitsi/jitsi-videobridge/blob/master/doc/s...

littlestymaar 7y ago

Thanks for your interesting answer. It doesn't really address my question though, so I will rephrase it: If I take a 720p video of a webcam and encode it to be delivered as progressive live streaming, the resulting stream is going to be less than one Mbps : because the image doesn't move much, I don't need many key-frame, one every 4 seconds is more than enough, and I've seen streams with no more than one every 30s (live streaming of harbours' CCTV cameras, don't ask me why). But of course it won't be realtime either, and you're gonna have a few seconds of latency. While this is OK for live streaming, it is certainly not for a video chat room. What I'd like to know is why the latency requirement reduces that much the encoding efficiency. Do you have an idea ?