3 ms·

Curiosity, what is the difference between 4D or 6DoF (six degrees of freedom)? Sounds a lot like the 6DoF work that Lytro did back in 2012, although this obviou

Curiosity, what is the difference between 4D or 6DoF (six degrees of freedom)? Sounds a lot like the 6DoF work that Lytro did back in 2012, although this obviously is coming at the problem from the other direction, generating it rather than capturing it.

moralestapia 2y ago
Move in 3D space + rotate in 3D space, I think. But w/ time should it be 7?
deckar01 2y ago
Lytro added 2 spatial dimensions of info to 2D image capture: the angles the light was traveling at when it entered the camera. They could simulate the image with different camera parameters, which was good for changing depth of field after the fact, but the occlusion information was limited by the diameter of the aperture. They tried to make depth maps, but that extra data was not a silver bullet. As far as I could tell, they were still fundamentally COLMAPing, they just had extra hints to guess with.
- ryandamm 2y ago
  This is spot-on. Note that the aperture on the camera was quite large, I want to say on the order of 100mm? They sourced really exotic hardware for that cinema camera. They also had the "Immerge," which was a ~1m diameter, hexagonal array of 2D cameras. They got the 4D data from having a 2D (spatially distributed) array of 2D samples (each camera's views). It's under sampled, because they threw out most of the light, but using 3D as a prior for reconstructing missing rays is generally pretty effective. But I also understand a lot of what they demoed at first was smoke and mirrors, plus a lot of traditional 3D VFX workflows. Still impressive as hell at the time, it's just that the tech has progressed significantly since ~2018.
  - PaulHoule 2y ago
    I got as Lytro Illium off Ebay at a reasonable price but it is a bit of a white elephant. I was hoping to shoot stereograms but I haven't been able to do it with the stock software (I just get two images that look the same with no clear disparity) I've seen open source software for plentopic images which might be able to generate a point cloud but I've only gotten one good shot of the Lytro which was similar to a shot I took with this crazy lens https://7artisans.store/products/7artisans-50mm-f0-95-large-aperture-manual-focus-prime-fixed-lens-aps-c-for-sony-e-nikon-z-fuji-fx-canon-eos-m-olympus-m43 https://7artisans.store/products/7artisans-50mm-f0-95-large-...
littlestymaar 2y ago
The scene itself moves over time, hence the 4D. Vanilla gaussian splating already give you 6 degrees of freedom since you have a full 3D scene.

moralestapia 2y ago

Move in 3D space + rotate in 3D space, I think. But w/ time should it be 7?

deckar01 2y ago

Lytro added 2 spatial dimensions of info to 2D image capture: the angles the light was traveling at when it entered the camera. They could simulate the image with different camera parameters, which was good for changing depth of field after the fact, but the occlusion information was limited by the diameter of the aperture. They tried to make depth maps, but that extra data was not a silver bullet. As far as I could tell, they were still fundamentally COLMAPing, they just had extra hints to guess with.

ryandamm 2y ago
This is spot-on. Note that the aperture on the camera was quite large, I want to say on the order of 100mm? They sourced really exotic hardware for that cinema camera. They also had the "Immerge," which was a ~1m diameter, hexagonal array of 2D cameras. They got the 4D data from having a 2D (spatially distributed) array of 2D samples (each camera's views). It's under sampled, because they threw out most of the light, but using 3D as a prior for reconstructing missing rays is generally pretty effective. But I also understand a lot of what they demoed at first was smoke and mirrors, plus a lot of traditional 3D VFX workflows. Still impressive as hell at the time, it's just that the tech has progressed significantly since ~2018.
- PaulHoule 2y ago
  I got as Lytro Illium off Ebay at a reasonable price but it is a bit of a white elephant. I was hoping to shoot stereograms but I haven't been able to do it with the stock software (I just get two images that look the same with no clear disparity) I've seen open source software for plentopic images which might be able to generate a point cloud but I've only gotten one good shot of the Lytro which was similar to a shot I took with this crazy lens https://7artisans.store/products/7artisans-50mm-f0-95-large-aperture-manual-focus-prime-fixed-lens-aps-c-for-sony-e-nikon-z-fuji-fx-canon-eos-m-olympus-m43 https://7artisans.store/products/7artisans-50mm-f0-95-large-...

littlestymaar 2y ago

The scene itself moves over time, hence the 4D. Vanilla gaussian splating already give you 6 degrees of freedom since you have a full 3D scene.