3 ms·
I'm not aware of any of the big players rendering virtual backgrounds on the server. It's all client-side. A couple years ago when I worked at Meta, I built th
by pavlov 3y ago
I'm not aware of any of the big players rendering virtual backgrounds on the server. It's all client-side.
A couple years ago when I worked at Meta, I built the server bits for a demo that was a hybrid solution. The client did the segmentation as usual but instead of sending the final composite, it sent the raw alpha channel (i.e. the transparency mask) in addition to the video stream. The composite was then rendered on a server. This added an interesting level of flexibility to the composition, as you could change the background independently and move/scale the participant's masked video to adapt the layout on the fly. But I don't suppose it ever went into production.
- bobdvb 3y agoTeams allows you to put people in a virtual lecture theatre called "Together Mode" where the individuals are cut out. I presume it sends the alpha along with the video.
- michaelmior 3y agoZoom has a very similar feature which I would also assume is implemented like this.
- pavlov 3y agoMaybe? What happens when some people on the call are not sending the alpha though? Background segmentation remains a hardware-dependent feature. Without the alpha, you can either skip the participant, render them unmasked, do the expensive segmentation on the receiver side (where the rendering happens), or do some kind of fast approximation for a mask. Personally I would probably try to implement this feature first without the alpha channels, maybe just using face detection to get a passable mask cheaply. In a layout with a grid of people and a fancy complex background design, the requirements for mask quality are not quite as strict. There are compositing tricks that could obfuscate the cheap masking.