4 ms·

Good article, thanks. BTW, this is the method Jitsi-meet uses. They also use BodyPix. https://github.com/jitsi/jitsi-meet/blob/master/react/features/stream-effe

Good article, thanks. BTW, this is the method Jitsi-meet uses. They also use BodyPix. https://github.com/jitsi/jitsi-meet/blob/master/react/features/stream-effects/blur/JitsiStreamBlurEffect.js https://github.com/jitsi/jitsi-meet/blob/master/react/featur...

BenTheElder 6y ago
Thanks for sharing -- With very similar settings too! (Not far from the defaults unsurprisingly). Injecting this into a web client seems like the sweet spot effort wise.
shrx 6y ago
Where is the virtual background option in Jitsi? I couldn't find it.
- benoliver999 6y ago
  On Jitsi Meet, in the browser, there is a menu on the bottom right. Click it and there are loads of options there including blur background.
  - porjo 6y ago
    I'm confused. Parent was asking about virtual background. I've checked with meet.jit.si using Chrome and there is no virtual background option in the menu.
    - shrx 6y ago
      It seems that Jitsi uses the tensorflow NN to detect the background, but only for the purpose of blurring it, not replacing it with a virtual background.
- monksy 6y ago
  It's probably an easier solution to create a processed video device
thelastbender12 6y ago
thank you for the link! Do you know if browser extensions can access the webcam too?
TuringNYC 6y ago
Thanks for the share @knotty66. Curious if you can discuss the technicals of this a bit? In the case of Jitsi-meet mobile apps, it seems advantageous that BodyPix models are mobile-first, but did that make sense for the desktop version as well? I've always been disappointed that BodyPix and some similar models are mobile-first and mobile-only on TensorFlow (https://blog.tensorflow.org/2019/11/updated-bodypix-2.html https://blog.tensorflow.org/2019/11/updated-bodypix-2.html) -- are these models just not used much on server-side settings? There seems to be very little documentation on doing this server side.
tpetry 6y ago
Sadly bodypic is very slow. You're only achieving 8-15fps, so any motion is very sluggish. Additionally the detection is on many cases very inaccurate: Too much or too less is detected belonging to the body so you can really only use a blur filter because a virtual background looks very strange if parts of your shoulders/hair or half an ear is missing etc. But i am really impressed by the detection of Microsoft Teams, i would love having this quality and speed available in a browser.

BenTheElder 6y ago

Thanks for sharing -- With very similar settings too! (Not far from the defaults unsurprisingly). Injecting this into a web client seems like the sweet spot effort wise.

shrx 6y ago

Where is the virtual background option in Jitsi? I couldn't find it.

benoliver999 6y ago
On Jitsi Meet, in the browser, there is a menu on the bottom right. Click it and there are loads of options there including blur background.
- porjo 6y ago
  I'm confused. Parent was asking about virtual background. I've checked with meet.jit.si using Chrome and there is no virtual background option in the menu.
  - shrx 6y ago
    It seems that Jitsi uses the tensorflow NN to detect the background, but only for the purpose of blurring it, not replacing it with a virtual background.
monksy 6y ago
It's probably an easier solution to create a processed video device

thelastbender12 6y ago

thank you for the link! Do you know if browser extensions can access the webcam too?

TuringNYC 6y ago

Thanks for the share @knotty66. Curious if you can discuss the technicals of this a bit? In the case of Jitsi-meet mobile apps, it seems advantageous that BodyPix models are mobile-first, but did that make sense for the desktop version as well? I've always been disappointed that BodyPix and some similar models are mobile-first and mobile-only on TensorFlow (https://blog.tensorflow.org/2019/11/updated-bodypix-2.html https://blog.tensorflow.org/2019/11/updated-bodypix-2.html) -- are these models just not used much on server-side settings? There seems to be very little documentation on doing this server side.

tpetry 6y ago

Sadly bodypic is very slow. You're only achieving 8-15fps, so any motion is very sluggish. Additionally the detection is on many cases very inaccurate: Too much or too less is detected belonging to the body so you can really only use a blur filter because a virtual background looks very strange if parts of your shoulders/hair or half an ear is missing etc. But i am really impressed by the detection of Microsoft Teams, i would love having this quality and speed available in a browser.