3 ms·

I think the true revolutionary change will be when we get easy to capture radiance fields or Gaussian splats. Being able to explore the “photo” with 6DOF will b

I think the true revolutionary change will be when we get easy to capture radiance fields or Gaussian splats. Being able to explore the “photo” with 6DOF will be a significant upgrade from normal 2d work. Stereoscopic 3d is interesting, but feels a little off putting when moving your head doesn’t move your location in the scene.

mikewarot 3y ago
Lytro could have changed photography, but they went with an optics system that was about 6" too small in diameter. It needed to be something big and clunky, but capable of doing amazing things with the captured light fields. There wasn't enough change of perspective across the lens to be useful at the form factor they chose.
- IanCal 3y ago
  You might be interested in it's bigger sibling, the 40k 11ft cinema camera https://www.lightfieldlab.com/blogposts/the-promise-of-light-field-cinematography https://www.lightfieldlab.com/blogposts/the-promise-of-light... I can't find a specific link to one I've seen but there's some great videos about this truly insane bit of tech.
  - mikewarot 3y ago
    I've made images with a virtual light field bigger than that.. focus is incredibly short, which is awesome. These 2 images were derived from the same set of photos taken about 2 blocks away from the subjects in Chicago https://flic.kr/p/2n1uYqj https://flic.kr/p/2n1AhXa In this case the virtual aperture was about 20 feet. For most subjects 1 foot is more appropriate. It's a time intensive process, but cost nothing otherwise.
- consumer451 3y ago
  Based on my quick reading of TFA, an iPhone 15 Pro uses two sensors very close together for 3D, and the wide angle lens lowers the res to 1080p. I wonder if as a kludge, we will see a 3rd party app and dual iPhone holder which then creates a single 3D video/image, allowing more human-like pupillary distance at 8k from the two main sensors.
lelag 3y ago
I totally agree with your assessment, Nerf and Gaussian are much more interesting and I'm sure they will be huge in the future. But I don't think it will ever be easy to capture a clean scene randomly, at least not with anything that moves like people in the scene. The issue is that in order for the Nerf/3DGS to be clean, that is to not have too many zones where the data is not sufficient to train the model for that particular spot, you need to have pictures from every angles to train the scene. For a totally static scene, it will work: walk around the scene until you have all the angles, and you are done. The capture won't even need to be perfect because diffusion models will be able to fill in the gap for what you missed. However, if you are talking about recording a memory, presumably with several moving people and objects, it's going to be a challenge as you can't freeze time (or I can't at least). For planned production, it won't be an issue, the trick will be to setup enough cameras around the scene to shoot from all the angles, but I don't think we can expect to ever have a single magical device that you can walk around with and record Nerfs on the fly.
- scheeseman486 3y ago
  Instead of photographing from every angle you could capture with an array of cameras placed on the outside of a ~1m spheroid, similar to how Google captured their lightfield demos. Generating guassian splats with that data would allow for high quality, perspective correct imagery from any virtual camera placed within that sphere, sort of like a 360 degree photo (and eventually video) that looks totally convincing. Move your head outside the sphere and the effect would break, but I feel 1x1m would be enough to allow for enough freedom of movement to be comfortable. Capture could perhaps be somewhat portable with some clever engineering, think those expanding sphere ball toys.

mikewarot 3y ago

Lytro could have changed photography, but they went with an optics system that was about 6" too small in diameter. It needed to be something big and clunky, but capable of doing amazing things with the captured light fields. There wasn't enough change of perspective across the lens to be useful at the form factor they chose.

IanCal 3y ago
You might be interested in it's bigger sibling, the 40k 11ft cinema camera https://www.lightfieldlab.com/blogposts/the-promise-of-light-field-cinematography https://www.lightfieldlab.com/blogposts/the-promise-of-light... I can't find a specific link to one I've seen but there's some great videos about this truly insane bit of tech.
- mikewarot 3y ago
  I've made images with a virtual light field bigger than that.. focus is incredibly short, which is awesome. These 2 images were derived from the same set of photos taken about 2 blocks away from the subjects in Chicago https://flic.kr/p/2n1uYqj https://flic.kr/p/2n1AhXa In this case the virtual aperture was about 20 feet. For most subjects 1 foot is more appropriate. It's a time intensive process, but cost nothing otherwise.
consumer451 3y ago
Based on my quick reading of TFA, an iPhone 15 Pro uses two sensors very close together for 3D, and the wide angle lens lowers the res to 1080p. I wonder if as a kludge, we will see a 3rd party app and dual iPhone holder which then creates a single 3D video/image, allowing more human-like pupillary distance at 8k from the two main sensors.

lelag 3y ago

I totally agree with your assessment, Nerf and Gaussian are much more interesting and I'm sure they will be huge in the future. But I don't think it will ever be easy to capture a clean scene randomly, at least not with anything that moves like people in the scene. The issue is that in order for the Nerf/3DGS to be clean, that is to not have too many zones where the data is not sufficient to train the model for that particular spot, you need to have pictures from every angles to train the scene. For a totally static scene, it will work: walk around the scene until you have all the angles, and you are done. The capture won't even need to be perfect because diffusion models will be able to fill in the gap for what you missed. However, if you are talking about recording a memory, presumably with several moving people and objects, it's going to be a challenge as you can't freeze time (or I can't at least). For planned production, it won't be an issue, the trick will be to setup enough cameras around the scene to shoot from all the angles, but I don't think we can expect to ever have a single magical device that you can walk around with and record Nerfs on the fly.

scheeseman486 3y ago
Instead of photographing from every angle you could capture with an array of cameras placed on the outside of a ~1m spheroid, similar to how Google captured their lightfield demos. Generating guassian splats with that data would allow for high quality, perspective correct imagery from any virtual camera placed within that sphere, sort of like a 360 degree photo (and eventually video) that looks totally convincing. Move your head outside the sphere and the effect would break, but I feel 1x1m would be enough to allow for enough freedom of movement to be comfortable. Capture could perhaps be somewhat portable with some clever engineering, think those expanding sphere ball toys.