5 ms·

They probably do run an encoder on the server. When in a many-users meeting, everyone but the one speaking have reduced resolution and bitrate. This suggests th

by orisho 6y ago

They probably do run an encoder on the server. When in a many-users meeting, everyone but the one speaking have reduced resolution and bitrate. This suggests they are encoding a low bitrate and high bitrate stream for each user, and switching as needed.

Tepix 6y ago
Still, they only need to do two encodings per user, not 2x users ²
ody4242 6y ago
or they expect the client to send a low res stream to the server, and they only do the "mixing" on the server side.
bronco21016 6y ago
I’m guessing the client sends multiple resolutions and bitrates. The server selectively forwards the stream most appropriate at the time. It would be incredibly cost prohibitive to run servers with live transcoding at their scale with a widely used free tier.
- simias 6y ago
  What you say is plausible but encoding multiple streams on weaker clients would be challenging I think (doesn't it run on tablets and the like?). Also I don't know if it's a reasonable comparison point but Youtube does transcode server-side into a whole bunch of of different resolutions and video standards, going as high as 8K these days I think. Given the amount of video they receive every single second it's pretty wild how much processing power that must represent.
  - andy_ppp 6y ago
    I suggest the server tells the client which bitrate/compression to send, but watermarks probably don’t need a full re-encode on the server, could easily be in metadata or only key frames which would be fine. Scary though, I’m sure this will happen in all media at some point. The best way to get around this of course is use someone else’s account to leak from...
    - meowface 6y ago
      I imagine the only truly secure way to do it would be to transcribe the meeting (automatically or manually) and leak the transcription. The video/audio could be shared with one or two trusted journalists to prove it's legitimate (barring sophisticated deepfakes), and the journalists could post the transcript. Maybe in theory even a transcription could be unsafe if the server actually manipulated the words in real-time in some way so that different listeners would hear different word synonyms at certain points, but that seems very convoluted and unlikely.
  - kevincox 6y ago
    This isn't challenging. It is done for a majority of conference software. Essentially the client uploads the streams in such a way that they "depend" on each other. You can imagine there is a base stream at 480p 15fps, then you can add on some packets to get 480 30fps, and further to get 720fps. https://webrtcglossary.com/svc/ https://webrtcglossary.com/svc/
- vinay_ys 6y ago
  One can easily verify this by looking at their bandwidth utilization. It is highly unlikely they are uploading multiple streams from every participant and just muxing the right-sized stream for the other participants to download. Bandwidth costs might be more expensive than the transcoding costs. Remember, you can use ASIC accelerated transcoders which can be quite cost-effective. If they really want to cut corners, they can simply command-and-control every participant client to send low-res stream and restream to every other participant without full transcoding – at least for free tier.
  - deleted 6y ago
    [deleted]
  - deleted 6y ago
    [deleted]
  - bronco21016 6y ago
    I don't believe you would be able to see it in your bandwidth utilization. I suppose you would be able to estimate which stream you're receiving based on number of callers in a conference but your outgoing stream is always going to be the same, assuming you're not network constrained. The stream is a single H264/SVC stream which contains layers of varying quality. It's sent to the Zoom "cloud" where the appropriately sized layer is extracted and forwarded to clients. It's called selective forwarding and it's essentially how all of the WebRTC style video conferencing solutions work once there are more than 2-3 participants.
- mcintyre1994 6y ago
  > It would be incredibly cost prohibitive to run servers with live transcoding at their scale with a widely used free tier. From the article, it looks like video watermark is only available for a meeting with only signed in users and locked to a particular domain. I don't know their pricing model, but speculating - I wonder if those features mean it's only available on some sort of premium tier and they could live encode only those meetings?
  - ridaj 6y ago
    There are more reasons than just cost to do just for signed in users on a given domain. - Privacy (expected for consumers, debatable for domains). - Willingness to pay (if your stuff is worth protecting, might as well charge for it). - Decoding the watermarks: I assume this is only done on demand by authenticated users at a very low rate (otherwise if it's freely available to anyone, you give leakers an oracle that lets them determine whether their media scrubbing / fuzzing has indeed removed the watermarks, and these systems are more effective when they have no way to check / must live in fear that they haven't scrubbed it right)
yk 6y ago
That would costs actual resources server side. My guess is, that their entire architecture is designed to do as much as possible on the clients, because otherwise they could not offer a free service.
cbsmith 6y ago
That kind of doesn't work if you have end-to-end encryption turned on.

by orisho 6y ago

Tepix 6y ago

Still, they only need to do two encodings per user, not 2x users ²

ody4242 6y ago

or they expect the client to send a low res stream to the server, and they only do the "mixing" on the server side.

bronco21016 6y ago

I’m guessing the client sends multiple resolutions and bitrates. The server selectively forwards the stream most appropriate at the time. It would be incredibly cost prohibitive to run servers with live transcoding at their scale with a widely used free tier.

simias 6y ago
What you say is plausible but encoding multiple streams on weaker clients would be challenging I think (doesn't it run on tablets and the like?). Also I don't know if it's a reasonable comparison point but Youtube does transcode server-side into a whole bunch of of different resolutions and video standards, going as high as 8K these days I think. Given the amount of video they receive every single second it's pretty wild how much processing power that must represent.
- andy_ppp 6y ago
  I suggest the server tells the client which bitrate/compression to send, but watermarks probably don’t need a full re-encode on the server, could easily be in metadata or only key frames which would be fine. Scary though, I’m sure this will happen in all media at some point. The best way to get around this of course is use someone else’s account to leak from...
  - meowface 6y ago
    I imagine the only truly secure way to do it would be to transcribe the meeting (automatically or manually) and leak the transcription. The video/audio could be shared with one or two trusted journalists to prove it's legitimate (barring sophisticated deepfakes), and the journalists could post the transcript. Maybe in theory even a transcription could be unsafe if the server actually manipulated the words in real-time in some way so that different listeners would hear different word synonyms at certain points, but that seems very convoluted and unlikely.
- kevincox 6y ago
  This isn't challenging. It is done for a majority of conference software. Essentially the client uploads the streams in such a way that they "depend" on each other. You can imagine there is a base stream at 480p 15fps, then you can add on some packets to get 480 30fps, and further to get 720fps. https://webrtcglossary.com/svc/ https://webrtcglossary.com/svc/
vinay_ys 6y ago
One can easily verify this by looking at their bandwidth utilization. It is highly unlikely they are uploading multiple streams from every participant and just muxing the right-sized stream for the other participants to download. Bandwidth costs might be more expensive than the transcoding costs. Remember, you can use ASIC accelerated transcoders which can be quite cost-effective. If they really want to cut corners, they can simply command-and-control every participant client to send low-res stream and restream to every other participant without full transcoding – at least for free tier.
- deleted 6y ago
  [deleted]
- deleted 6y ago
  [deleted]
- bronco21016 6y ago
  I don't believe you would be able to see it in your bandwidth utilization. I suppose you would be able to estimate which stream you're receiving based on number of callers in a conference but your outgoing stream is always going to be the same, assuming you're not network constrained. The stream is a single H264/SVC stream which contains layers of varying quality. It's sent to the Zoom "cloud" where the appropriately sized layer is extracted and forwarded to clients. It's called selective forwarding and it's essentially how all of the WebRTC style video conferencing solutions work once there are more than 2-3 participants.
mcintyre1994 6y ago
> It would be incredibly cost prohibitive to run servers with live transcoding at their scale with a widely used free tier. From the article, it looks like video watermark is only available for a meeting with only signed in users and locked to a particular domain. I don't know their pricing model, but speculating - I wonder if those features mean it's only available on some sort of premium tier and they could live encode only those meetings?
- ridaj 6y ago
  There are more reasons than just cost to do just for signed in users on a given domain. - Privacy (expected for consumers, debatable for domains). - Willingness to pay (if your stuff is worth protecting, might as well charge for it). - Decoding the watermarks: I assume this is only done on demand by authenticated users at a very low rate (otherwise if it's freely available to anyone, you give leakers an oracle that lets them determine whether their media scrubbing / fuzzing has indeed removed the watermarks, and these systems are more effective when they have no way to check / must live in fear that they haven't scrubbed it right)

yk 6y ago

That would costs actual resources server side. My guess is, that their entire architecture is designed to do as much as possible on the clients, because otherwise they could not offer a free service.

cbsmith 6y ago

That kind of doesn't work if you have end-to-end encryption turned on.