5 ms·

Show HN: BokehCamera – Blur webcam background photorealistically using RealSense

60 points by dheera 5y ago

ricardobeat 5y ago
Are there any readily available stereo cameras besides RealSense? Turning this into a product for video conferencing would be a huge hit, been thinking of it for a while.
- dheera 5y ago
  OP here! Here are a few: - Structure Core: https://structure.io/structure-core https://structure.io/structure-core Closest to RealSense in functionality that I've tried so far, but pricier. Software is in a private developer portal instead of a Git repo where everyone can submit issues and pull requests and comment, so it's not as easy to get help as RealSense when you run into issues. - Mynt Eye: https://www.mynteye.com/ https://www.mynteye.com/ From my understanding the S models eat into your GPU and the D models have on-board processing, but the D models seem sold out. - ZED: https://www.stereolabs.com/zed/ https://www.stereolabs.com/zed/ They eat into your GPU. No on-board processing. I'd be very interested to know about others, especially if there are more that can do on-board processing, since I typically need my entire GPU for other tasks.
itake 5y ago
seems like RealSense is winding down? https://www.crn.com/news/components-peripherals/intel-says-it-s-winding-down-realsense-camera-business https://www.crn.com/news/components-peripherals/intel-says-i...
- tim-fan 5y ago
  Yes, although it's not as bad as it initially seemed; the stereo product lines will be continued. https://www.linkedin.com/posts/andersgj_i-can-share-this-more-detailed-communication-activity-6834279212426371072-ovuj https://www.linkedin.com/posts/andersgj_i-can-share-this-mor...
deleted 5y ago
[deleted]
black_knight 5y ago
Nice! If you are not on a budget, you can use a mirrorless camera with a big aperture (say f1.6), wide angle lens as a "webcam" by connecting the HDMI out of the camera to the computer using an Elgato Camlink. That will also get you a nicely blurred background. I do this, and a lot of people do a double take whenever I enter a zoom call. Best commment so far was "What is happening? Are you in a movie?".
- dheera 5y ago
  Yep this is a neat trick too. Also I must say (as a hobbyist photographer myself) that even if you're on a budget, if you want to go the optical route, you can easily grab a cheap used DSLR like a Canon 60D which often goes for <$200 used, and slap on an old M42 Russian lens with a conversion adapter and get fantastic bokeh for video calls. The mildly annoying part is most DSLRs don't have an AC adapter input to allow them to be constantly on, but you can buy a "fake" Canon battery that sits in the battery compartment and wires in AC adapter input.
  - black_knight 5y ago
    > The mildly annoying part is most DSLRs don't have an AC adapter input to allow them to be constantly on, > but you can buy a "fake" Canon battery that sits in the battery compartment and wires in AC adapter input. Yeah, I left out that detail. One should also make sure that the camera has a clean HDMI output. I can't use my old EOS 500D for this since the HDMI output is grody with focus aids and stuff.
    - brippalcharrid 5y ago
      You don't even need to use HDMI or have a capture card; you can connect the camera with USB and use gphoto2/v4l2loopback to present it to the OS as a video input. With older cameras that have USB2, you will be limited to a lower bandwidth and resolution (200Mbps -> 1024x680), but it still looks fantastic, especially with high-quality lenses and shallow depth of field that enables video codecs to encode the out-of-focus-areas efficiently and concentrate on detail in the focal plane.
vzaliva 5y ago
Looks great, but my video conferencing setup is on Mac. If there is something similar for MacOS?
teawrecks 5y ago
You're going to have photographers arguing that this is merely depth of field, and for it to be called bokeh you need light sources/reflections to bloom in the shape of the aperture. Which would be a super cool feature.
- dheera 5y ago
  Yeah that would be cool! I guess that's actually not that hard to implement since I just need to do some sort of 2D convolution with the aperture shape, instead of just cv2.blur. The tricky thing is that in order to get different amount of blur at different depths, I interpolate between precomputed 3 levels of blur, since that vectorizes well at -O3, I'm not sure how to compute a per-pixel varying blur efficiently without going to the GPU.
- underwater 5y ago
  That's what I thought this was doing, based on the description. The reason that Meet backgrounds look fake is because they do a Gaussian Blur instead of simulating a lens. A physical bokeh will blow out highlights, whereas a Gaussian Blur will smooth out bright spots.

60 points by dheera 5y ago

Are there any readily available stereo cameras besides RealSense? Turning this into a product for video conferencing would be a huge hit, been thinking of it for a while.

dheera 5y ago
OP here! Here are a few: - Structure Core: https://structure.io/structure-core https://structure.io/structure-core Closest to RealSense in functionality that I've tried so far, but pricier. Software is in a private developer portal instead of a Git repo where everyone can submit issues and pull requests and comment, so it's not as easy to get help as RealSense when you run into issues. - Mynt Eye: https://www.mynteye.com/ https://www.mynteye.com/ From my understanding the S models eat into your GPU and the D models have on-board processing, but the D models seem sold out. - ZED: https://www.stereolabs.com/zed/ https://www.stereolabs.com/zed/ They eat into your GPU. No on-board processing. I'd be very interested to know about others, especially if there are more that can do on-board processing, since I typically need my entire GPU for other tasks.

itake 5y ago

seems like RealSense is winding down? https://www.crn.com/news/components-peripherals/intel-says-it-s-winding-down-realsense-camera-business https://www.crn.com/news/components-peripherals/intel-says-i...

tim-fan 5y ago
Yes, although it's not as bad as it initially seemed; the stereo product lines will be continued. https://www.linkedin.com/posts/andersgj_i-can-share-this-more-detailed-communication-activity-6834279212426371072-ovuj https://www.linkedin.com/posts/andersgj_i-can-share-this-mor...

deleted 5y ago

[deleted]

black_knight 5y ago

Nice! If you are not on a budget, you can use a mirrorless camera with a big aperture (say f1.6), wide angle lens as a "webcam" by connecting the HDMI out of the camera to the computer using an Elgato Camlink. That will also get you a nicely blurred background. I do this, and a lot of people do a double take whenever I enter a zoom call. Best commment so far was "What is happening? Are you in a movie?".

dheera 5y ago
Yep this is a neat trick too. Also I must say (as a hobbyist photographer myself) that even if you're on a budget, if you want to go the optical route, you can easily grab a cheap used DSLR like a Canon 60D which often goes for <$200 used, and slap on an old M42 Russian lens with a conversion adapter and get fantastic bokeh for video calls. The mildly annoying part is most DSLRs don't have an AC adapter input to allow them to be constantly on, but you can buy a "fake" Canon battery that sits in the battery compartment and wires in AC adapter input.
- black_knight 5y ago
  > The mildly annoying part is most DSLRs don't have an AC adapter input to allow them to be constantly on, > but you can buy a "fake" Canon battery that sits in the battery compartment and wires in AC adapter input. Yeah, I left out that detail. One should also make sure that the camera has a clean HDMI output. I can't use my old EOS 500D for this since the HDMI output is grody with focus aids and stuff.
  - brippalcharrid 5y ago
    You don't even need to use HDMI or have a capture card; you can connect the camera with USB and use gphoto2/v4l2loopback to present it to the OS as a video input. With older cameras that have USB2, you will be limited to a lower bandwidth and resolution (200Mbps -> 1024x680), but it still looks fantastic, especially with high-quality lenses and shallow depth of field that enables video codecs to encode the out-of-focus-areas efficiently and concentrate on detail in the focal plane.

vzaliva 5y ago

Looks great, but my video conferencing setup is on Mac. If there is something similar for MacOS?

teawrecks 5y ago

You're going to have photographers arguing that this is merely depth of field, and for it to be called bokeh you need light sources/reflections to bloom in the shape of the aperture. Which would be a super cool feature.

dheera 5y ago
Yeah that would be cool! I guess that's actually not that hard to implement since I just need to do some sort of 2D convolution with the aperture shape, instead of just cv2.blur. The tricky thing is that in order to get different amount of blur at different depths, I interpolate between precomputed 3 levels of blur, since that vectorizes well at -O3, I'm not sure how to compute a per-pixel varying blur efficiently without going to the GPU.
underwater 5y ago
That's what I thought this was doing, based on the description. The reason that Meet backgrounds look fake is because they do a Gaussian Blur instead of simulating a lens. A physical bokeh will blow out highlights, whereas a Gaussian Blur will smooth out bright spots.