5 ms·

Pure speculation: when combined with the LIDAR depth sensor, the two cameras probably don't need as much physical separation to accurately create a depth map. T

Pure speculation: when combined with the LIDAR depth sensor, the two cameras probably don't need as much physical separation to accurately create a depth map. The bigger problem is the inpainting needed to generate hidden detail when the movie is viewed from angles that are different from the one it was actually filmed from.

TaylorAlexander 3y ago
I was wondering about the use of the lidar sensor. Notably they do not say they are using it, but maybe they just wanted to keep it simple? Idk seems weird not to use lidar but also seems weird not to mention it if they are using it.
aYsY4dDQ2NrcNzA 3y ago
3D movies have existed for decades, without adjustable viewing angles.
- mlyle 3y ago
  But if you have eyes 50mm apart, and source material from cameras 15mm apart (plus other depth information), you'll need to in-paint a small amount where your eyes could see "around" something and the cameras can't.
  - coldtea 3y ago
    Or you can make do with 15mm-apart worth of "around"? It's still more "spatial"/3D than a regular (single lens) image. Plus this has a wide lens and a "regular" lens (actually both wide iirc but one is ultrawide), so it's not like 2 equal lenses 50mm apart like in regular stereoscopic "3d" video.
    - mlyle 3y ago
      > Or you can make do with 15mm-apart worth of "around"? You need to move the close objects further apart in left/right than they are in the camera. Then you need to fill the newly empty areas with something. > Plus this has a wide lens and a "regular" lens (actually both wide iirc but one is ultrawide), so it's not like 2 equal lenses 50mm apart like in regular stereoscopic "3d" video. This doesn't affect anything.
  - aYsY4dDQ2NrcNzA 3y ago
    I was responding to > The bigger problem is the inpainting needed to generate hidden detail when the movie is viewed from angles that are different from the one it was actually filmed from.
    - mlyle 3y ago
      Yes-- so to even create a fixed view with viewports that are further apart than the real cameras are, you have ot inpaint hidden detail.
jauntywundrkind 3y ago
My understanding is that very few consumer lidar sensors work well in daylight. It's hard to send out & detect significantly meaningful pulses of light, when there's sunlight all around. I have an Intel L515 which is pretty remarkable in that sometimes you can get some depth finding outdoors. This is just a hobby item for me, I'm not an expert, but this launched as a fairly impressively long range & capable $350 USB3 system, and it seems like the market hasn't much comparable to it. Phones certainly I'd expect to be significantly worse.
- coldtea 3y ago
  >My understanding is that very few consumer lidar sensors work well in daylight. It's hard to send out & detect significantly meaningful pulses of light, when there's sunlight all around. Aren't many "self driving car" sensors lidar? This would imply they can work in daylight - perhaps they don't necessarily depend on light on the sunlight spectrum? (Or perhaps you don't consuder them consumer? Though those cars are consumer products, they're not made for military or industrial use)
  - jauntywundrkind 3y ago
    Many cars are lidar,, but they use much stronger, bigger, and higher power lasers, on very expensive and precise rotator assemblies. The L515 I mentioned was somewhat advanced at least for it's day because it used MEMS to steer its light source. That gave it leading class performance/size but it's still big and kinda hot-ish. Maybe we can keep scaling that kind of system performance to smaller sizes but even this package was pretty cutting edge & gave much better falloff than many competing systems, and was still largely an indoor sensor.
coldtea 3y ago
>The bigger problem is the inpainting needed to generate hidden detail when the movie is viewed from angles that are different from the one it was actually filmed from. It's for spatial video, not for holographic video. When you see a 3d movie in a cinema, it's not like you can look at it from widely different anges and go peek from the side or behind the actors or whatever...
- laserlight 3y ago
  Given that iPhone cameras are ~2.5 mm apart, there needs to be some amount of in-painting when building the stereo image, so that it looks like it is taken with cameras that are ~6.5 mm apart.

TaylorAlexander 3y ago

I was wondering about the use of the lidar sensor. Notably they do not say they are using it, but maybe they just wanted to keep it simple? Idk seems weird not to use lidar but also seems weird not to mention it if they are using it.

aYsY4dDQ2NrcNzA 3y ago

3D movies have existed for decades, without adjustable viewing angles.

mlyle 3y ago
But if you have eyes 50mm apart, and source material from cameras 15mm apart (plus other depth information), you'll need to in-paint a small amount where your eyes could see "around" something and the cameras can't.
- coldtea 3y ago
  Or you can make do with 15mm-apart worth of "around"? It's still more "spatial"/3D than a regular (single lens) image. Plus this has a wide lens and a "regular" lens (actually both wide iirc but one is ultrawide), so it's not like 2 equal lenses 50mm apart like in regular stereoscopic "3d" video.
  - mlyle 3y ago
    > Or you can make do with 15mm-apart worth of "around"? You need to move the close objects further apart in left/right than they are in the camera. Then you need to fill the newly empty areas with something. > Plus this has a wide lens and a "regular" lens (actually both wide iirc but one is ultrawide), so it's not like 2 equal lenses 50mm apart like in regular stereoscopic "3d" video. This doesn't affect anything.
- aYsY4dDQ2NrcNzA 3y ago
  I was responding to > The bigger problem is the inpainting needed to generate hidden detail when the movie is viewed from angles that are different from the one it was actually filmed from.
  - mlyle 3y ago
    Yes-- so to even create a fixed view with viewports that are further apart than the real cameras are, you have ot inpaint hidden detail.

jauntywundrkind 3y ago

My understanding is that very few consumer lidar sensors work well in daylight. It's hard to send out & detect significantly meaningful pulses of light, when there's sunlight all around. I have an Intel L515 which is pretty remarkable in that sometimes you can get some depth finding outdoors. This is just a hobby item for me, I'm not an expert, but this launched as a fairly impressively long range & capable $350 USB3 system, and it seems like the market hasn't much comparable to it. Phones certainly I'd expect to be significantly worse.

coldtea 3y ago
>My understanding is that very few consumer lidar sensors work well in daylight. It's hard to send out & detect significantly meaningful pulses of light, when there's sunlight all around. Aren't many "self driving car" sensors lidar? This would imply they can work in daylight - perhaps they don't necessarily depend on light on the sunlight spectrum? (Or perhaps you don't consuder them consumer? Though those cars are consumer products, they're not made for military or industrial use)
- jauntywundrkind 3y ago
  Many cars are lidar,, but they use much stronger, bigger, and higher power lasers, on very expensive and precise rotator assemblies. The L515 I mentioned was somewhat advanced at least for it's day because it used MEMS to steer its light source. That gave it leading class performance/size but it's still big and kinda hot-ish. Maybe we can keep scaling that kind of system performance to smaller sizes but even this package was pretty cutting edge & gave much better falloff than many competing systems, and was still largely an indoor sensor.

coldtea 3y ago

>The bigger problem is the inpainting needed to generate hidden detail when the movie is viewed from angles that are different from the one it was actually filmed from. It's for spatial video, not for holographic video. When you see a 3d movie in a cinema, it's not like you can look at it from widely different anges and go peek from the side or behind the actors or whatever...

laserlight 3y ago
Given that iPhone cameras are ~2.5 mm apart, there needs to be some amount of in-painting when building the stereo image, so that it looks like it is taken with cameras that are ~6.5 mm apart.