5 ms·

Can anyone who has seen these demos confirm if they are 6 DoF (the perspective changes with viewer position) or merely stereo? The latter is boring and easy to

Can anyone who has seen these demos confirm if they are 6 DoF (the perspective changes with viewer position) or merely stereo? The latter is boring and easy to implement, the former much more compelling and technically difficult.

stavros 3y ago
How can they be 6 DOF if they're recorded with two cameras?
- andybak 3y ago
  AI magic. (They aren't 6dof but it is possible)
  - starcraft2wol 3y ago
    AI is not magic, and still constrained by math. If you need 3 variables and only have 2, AI isn't going to do it.
    - andybak 3y ago
      Unless we're talking about different things - it already exists. There's two parts to it - estimating a depth map ("monocular depth estimation") and inpainting the occluded parts.
- Torkel 3y ago
  They have a lidar. So they have very accurate depth. This gives some limited possibility at extrapolating and showing things from different viewing angles. I think this is also how they can give a good stereo experience from cameras that are closer than the average human IPD.
  - j4yav 3y ago
    It would have to be AI generated, right? The cameras can't see behind what they can't see behind.
    - Torkel 3y ago
      Maybe, but not necessarily. Good oldfashioned computer vision can go a long way. When it comes to photography "AI" is a spice best used in moderation, cf eg https://www.theverge.com/2023/12/2/23985299/iphone-bridal-photo-three-poses-explanation-panorama-photoshop-generative-ai https://www.theverge.com/2023/12/2/23985299/iphone-bridal-ph...
  - alexdbird 3y ago
    Only for objects up to 5 metres away, though those might be where the most is gained from accuracy.
    - Torkel 3y ago
      Human depth perception only works up to ~7 meters so should be ok.
- ibrahimsow1 3y ago
  I like to imagine with 3D gaussian splatting of the videos. Not compute possible yet.
- kobalsky 3y ago
  something like this? https://guanjunwu.github.io/4dgs/ https://guanjunwu.github.io/4dgs/
deleted 3y ago
[deleted]
dagmx 3y ago
They are stereo using the MV-HEVC format https://developer.apple.com/av-foundation/HEVC-Stereo-Video-Profile.pdf https://developer.apple.com/av-foundation/HEVC-Stereo-Video-... And while stereo video as a format is simple, IMHO it’s still quite a feat of engineering to capture it from the current phone generation. There have been stereo Android phones before but they were engineered from the get go with stereo in mind, and sacrificed other elements to achieve it

stavros 3y ago

How can they be 6 DOF if they're recorded with two cameras?

andybak 3y ago
AI magic. (They aren't 6dof but it is possible)
- starcraft2wol 3y ago
  AI is not magic, and still constrained by math. If you need 3 variables and only have 2, AI isn't going to do it.
  - andybak 3y ago
    Unless we're talking about different things - it already exists. There's two parts to it - estimating a depth map ("monocular depth estimation") and inpainting the occluded parts.
Torkel 3y ago
They have a lidar. So they have very accurate depth. This gives some limited possibility at extrapolating and showing things from different viewing angles. I think this is also how they can give a good stereo experience from cameras that are closer than the average human IPD.
- j4yav 3y ago
  It would have to be AI generated, right? The cameras can't see behind what they can't see behind.
  - Torkel 3y ago
    Maybe, but not necessarily. Good oldfashioned computer vision can go a long way. When it comes to photography "AI" is a spice best used in moderation, cf eg https://www.theverge.com/2023/12/2/23985299/iphone-bridal-photo-three-poses-explanation-panorama-photoshop-generative-ai https://www.theverge.com/2023/12/2/23985299/iphone-bridal-ph...
- alexdbird 3y ago
  Only for objects up to 5 metres away, though those might be where the most is gained from accuracy.
  - Torkel 3y ago
    Human depth perception only works up to ~7 meters so should be ok.
ibrahimsow1 3y ago
I like to imagine with 3D gaussian splatting of the videos. Not compute possible yet.
kobalsky 3y ago
something like this? https://guanjunwu.github.io/4dgs/ https://guanjunwu.github.io/4dgs/

deleted 3y ago

[deleted]

dagmx 3y ago

They are stereo using the MV-HEVC format https://developer.apple.com/av-foundation/HEVC-Stereo-Video-Profile.pdf https://developer.apple.com/av-foundation/HEVC-Stereo-Video-... And while stereo video as a format is simple, IMHO it’s still quite a feat of engineering to capture it from the current phone generation. There have been stereo Android phones before but they were engineered from the get go with stereo in mind, and sacrificed other elements to achieve it