4 ms·

Two points are always on a single axis… it’s not like we lose our stereo vision when we tilt our heads.

dagmx 3y ago
I really don’t think you’ve thought through your curt response. 1. Clearly the array of cameras on these phones is along two axes, so the pedantic definition of two points on a line is irrelevant 2. You’d be shooting everything at a Dutch angle to maintain stereo if they weren’t aligned horizontally or vertically. 3. The sensors are rectangles and need to be aligned along their longest axes to maximize the most field of view.
- charcircuit 3y ago
  >3. The sensors are rectangles and need to be aligned along their longest axes to maximize the most field of view. The iPhone 14 could offer a lower FoV. The video format encodes the FoV of the recording already.
  - dagmx 3y ago
    It’s already cutting the resolution to 1080p to crop into the Ultrawide. How much lower would you go? Bear in mind this has to play back on one of the highest ppd displays on the market when the Vision Pro releases.
    - SirMaster 3y ago
      Just because it's the highest PPD doesn't mean the resolution is actually all that high. Resolution has always been my main complaint about VR headsets and they are nowhere near the PPD needed to simulate watching high resolution content on a high resolution screen in front of you yet. The Vision Pro is supposed to have 23MP, and an aspect ratio of 1.165:1 which suggests 3660 pixels across each eye. However, what most people fail to think about is the FOV. The headset's entire FOV is about 100 degrees horizontally. THX standards recommend around 35 degrees for video horizontal FOV, but if people like a really immersive FOV for their video I think it's safe to push this up to around 50 degrees. This means that a virtual theater screen within your FOV on the headset has only half the headset's resolution to work with. So that makes the width of a virtual 50 FOV horizontal screen 1830 pixels wide, which is less than even 1080p video at 1920 pixels wide. And this is probably a bit high too as there is typically some amount of pixels you can't even see through the lens of a VR headset because you don't normally want to be able to see the edge of the panel inside the headset. Rather you see the edge of the optics. So that 3660 panel width is probably more like 3460, then divide that in half to 1730 pixels wide for your virtual screen. And that's if you like a 50 FOV wide virtual screen. That would be like sitting 5 feet away from a 65" TV which is arguably closer than the vast majority of people sit. So if you are watching video at a pretty immersive 50 degree FOV, in an Apple Vision Pro, the native resolution of the video needs to only be about 1730x973 (assuming 16:9 video) at most to max out the PPD of the headset for reasonably sized virtual video screens.
      - dagmx 3y ago
        That’s all well and good (and good info) but you’re not factoring in pixel alignment. There’s loss because you need to account for the video being at a non-aligned offset at any given view from the native screen pixels, and the video may be presented at different planar angles or distances from the user. Therefore there would be some level of interpolation causing loss of clarity. In general, you’d want to double resolutions as a form of super sampling. The 1080p here would be just about cutting it then at the initial 35 degrees you mentioned. I think for the natural FoV of the phone camera and the types of content people tend to shoot (families etc) that a 50 degree view might be uncomfortably large. Perhaps for further afield content like landscapes etc
        SirMaster 3y ago
        I guess I'm not sure why the video should need to be higher resolution for super-sampling than the pixel space it will be rendered on. Why it would need to be higher resolution than the native pixel space that the virtual screen will be rendered over? Changing the angle or rotation of your head to the virtual screen only makes the virtual screen technically even lower resolution, and would make it so that you would notice an even lesser difference between higher and lower resolution videos being played on that virtual screen. I chose 50 FOV because I think it's reasonable near upper-bound. I am heavily into home theater, and I have tried various VR headsets to replicated a home theater projector setup. Personally I am a fan of sitting to my projector (or TV) screens where the width of the screen is equal to my viewing distance. This results in a horizontal FOV of 53.1 degrees. Now I often invite people over to my theater, or travel around and visit other people's HTs and I have found that most HT enthusiasts have no problems with this rough screen width-to-distance FOV for viewing movies. However maybe HT enthusiasts are not very representative of the masses, but anyway I was only using it as more of an upper-bound and what I have been looking for in a headset. If I want a virtual HT screen that has the native resolution of at least 1080p, and the headset has over 100 degree FOV (which most do), then I see that I need a headset to have more than 4000 pixels wide per eye, which sadly just doesn't exist yet. But, at least in my experience, I don't think the resolution of the video itself needs to be substantially higher than the pixel space of the virtual screen. I think that regular real-time machine learning video up-scaling techniques are sufficient enough these days to render the video in a way that wont cause bad artifacts when the virtual screen pixels are skewed because the headset wearer is tilting or turning their head in relation to the position of the virtual screen. Now I am fully aware that VR signal input also needs to be a higher resolution source than the native headset pixels, because there is some warping done to the image to make it look correct through the lenses, but that should only need to be applied to the 3D environment space the user is seeing. That means that the video being imposed into the virtual screen simply needs to be stretched via up-scale to a higher resolution and then this will be downscaled via the headset warp more or less back to its native resolution. But what you are saying does make sense and I am sure is somewhat of a factor. But then also, lower resolution video can still look quite fine once there is enough pixels to get good detail.
    - charcircuit 3y ago
      3D videos can be played on more devices than the vision pro and just because the vision pro has a high ppd that doesn't mean that the content will be rendering images with that high of a ppd.
      - dagmx 3y ago
        But Apple aren’t making this for other devices, they’re making it for their own ecosystem. That it can play elsewhere is not a consideration.
- tsunamifury 3y ago
  Angle of the sensor also makes no difference other than aligned effective resolution. iPhone already crops in several modes also digitally stabilizes ie rotates. Worked with lots of dimensional photography measurements with Pixel. Nothing here is limiting really. Maybe there is some quality giveups. This isn’t film projector 3D where you lose the effect if your eyes aren’t aligned with the dual projectors. This is reconstructed 3D from two points of measurement. Do You think the effect will only work if your phone is held 100% 90 degree level? I mean your blowing me of smugly but all those points can be overcome with just a crop and tilt. It shows the typical Hackernews mentality “your wrong even though I have no experience in this space and only thought through this superficially.” Apple wants to sell more hardware.
  - dagmx 3y ago
    Again, I think you’re not thinking this through, and all your hubris of having experience in the space is a call to authority rather than any actual logical correctness. 1. Nobody would want to hold the phone at an angle to shoot all the time. 2. It’s already reducing resolution to 1080p because it’s cropping in on the Ultrawide when its axis aligned. How much lower would you go to do an angled shot? The resolution does matter because these will be playing back on one of the highest ppd headsets on the market. Apple may want to sell more hardware, but that’s not particularly interesting insight. But at the same time, your theories of how it could be done on the 14 Pro would make for a rather poor user experience all around.
    - tsunamifury 3y ago
      1. The censors do not need to be physically 90 degree aligned to get depth data. And the the crop can be rotated so you do not need to hold the phone at a dutch angle... 2. The censors are 24mp and 12mp, way way way above 1080p (2.1mp). A tilted crop sample will be no issue at all. Also you are using the term 'appeal to authority' wrong as I am not referencing an outside authority here, I am referencing myself. I'm sorry I've made you a bit angry here, but just calm it down a bit and think it through. The overlayed images of two cameras are slightly mis-aligned, but the overlapping data is more than enough to produce a 2.1mp image. This can work both in traditional stereoscopic or with 3D generated depth data. I think one thing that threw me though is the the advertising images for this are clearly deceptive. They show 3D generated video that can be viewed from multiple angles, when in actually HVEC is just stereoscopic.
mlyle 3y ago
Two points are always on a single axis, but the two image sensors will still be rectangles at an odd angle relative to that axis.

I really don’t think you’ve thought through your curt response. 1. Clearly the array of cameras on these phones is along two axes, so the pedantic definition of two points on a line is irrelevant 2. You’d be shooting everything at a Dutch angle to maintain stereo if they weren’t aligned horizontally or vertically. 3. The sensors are rectangles and need to be aligned along their longest axes to maximize the most field of view.

charcircuit 3y ago
>3. The sensors are rectangles and need to be aligned along their longest axes to maximize the most field of view. The iPhone 14 could offer a lower FoV. The video format encodes the FoV of the recording already.
- dagmx 3y ago
  It’s already cutting the resolution to 1080p to crop into the Ultrawide. How much lower would you go? Bear in mind this has to play back on one of the highest ppd displays on the market when the Vision Pro releases.
  - SirMaster 3y ago
    Just because it's the highest PPD doesn't mean the resolution is actually all that high. Resolution has always been my main complaint about VR headsets and they are nowhere near the PPD needed to simulate watching high resolution content on a high resolution screen in front of you yet. The Vision Pro is supposed to have 23MP, and an aspect ratio of 1.165:1 which suggests 3660 pixels across each eye. However, what most people fail to think about is the FOV. The headset's entire FOV is about 100 degrees horizontally. THX standards recommend around 35 degrees for video horizontal FOV, but if people like a really immersive FOV for their video I think it's safe to push this up to around 50 degrees. This means that a virtual theater screen within your FOV on the headset has only half the headset's resolution to work with. So that makes the width of a virtual 50 FOV horizontal screen 1830 pixels wide, which is less than even 1080p video at 1920 pixels wide. And this is probably a bit high too as there is typically some amount of pixels you can't even see through the lens of a VR headset because you don't normally want to be able to see the edge of the panel inside the headset. Rather you see the edge of the optics. So that 3660 panel width is probably more like 3460, then divide that in half to 1730 pixels wide for your virtual screen. And that's if you like a 50 FOV wide virtual screen. That would be like sitting 5 feet away from a 65" TV which is arguably closer than the vast majority of people sit. So if you are watching video at a pretty immersive 50 degree FOV, in an Apple Vision Pro, the native resolution of the video needs to only be about 1730x973 (assuming 16:9 video) at most to max out the PPD of the headset for reasonably sized virtual video screens.
    - dagmx 3y ago
      That’s all well and good (and good info) but you’re not factoring in pixel alignment. There’s loss because you need to account for the video being at a non-aligned offset at any given view from the native screen pixels, and the video may be presented at different planar angles or distances from the user. Therefore there would be some level of interpolation causing loss of clarity. In general, you’d want to double resolutions as a form of super sampling. The 1080p here would be just about cutting it then at the initial 35 degrees you mentioned. I think for the natural FoV of the phone camera and the types of content people tend to shoot (families etc) that a 50 degree view might be uncomfortably large. Perhaps for further afield content like landscapes etc
      - SirMaster 3y ago
        I guess I'm not sure why the video should need to be higher resolution for super-sampling than the pixel space it will be rendered on. Why it would need to be higher resolution than the native pixel space that the virtual screen will be rendered over? Changing the angle or rotation of your head to the virtual screen only makes the virtual screen technically even lower resolution, and would make it so that you would notice an even lesser difference between higher and lower resolution videos being played on that virtual screen. I chose 50 FOV because I think it's reasonable near upper-bound. I am heavily into home theater, and I have tried various VR headsets to replicated a home theater projector setup. Personally I am a fan of sitting to my projector (or TV) screens where the width of the screen is equal to my viewing distance. This results in a horizontal FOV of 53.1 degrees. Now I often invite people over to my theater, or travel around and visit other people's HTs and I have found that most HT enthusiasts have no problems with this rough screen width-to-distance FOV for viewing movies. However maybe HT enthusiasts are not very representative of the masses, but anyway I was only using it as more of an upper-bound and what I have been looking for in a headset. If I want a virtual HT screen that has the native resolution of at least 1080p, and the headset has over 100 degree FOV (which most do), then I see that I need a headset to have more than 4000 pixels wide per eye, which sadly just doesn't exist yet. But, at least in my experience, I don't think the resolution of the video itself needs to be substantially higher than the pixel space of the virtual screen. I think that regular real-time machine learning video up-scaling techniques are sufficient enough these days to render the video in a way that wont cause bad artifacts when the virtual screen pixels are skewed because the headset wearer is tilting or turning their head in relation to the position of the virtual screen. Now I am fully aware that VR signal input also needs to be a higher resolution source than the native headset pixels, because there is some warping done to the image to make it look correct through the lenses, but that should only need to be applied to the 3D environment space the user is seeing. That means that the video being imposed into the virtual screen simply needs to be stretched via up-scale to a higher resolution and then this will be downscaled via the headset warp more or less back to its native resolution. But what you are saying does make sense and I am sure is somewhat of a factor. But then also, lower resolution video can still look quite fine once there is enough pixels to get good detail.
  - charcircuit 3y ago
    3D videos can be played on more devices than the vision pro and just because the vision pro has a high ppd that doesn't mean that the content will be rendering images with that high of a ppd.
    - dagmx 3y ago
      But Apple aren’t making this for other devices, they’re making it for their own ecosystem. That it can play elsewhere is not a consideration.
tsunamifury 3y ago
Angle of the sensor also makes no difference other than aligned effective resolution. iPhone already crops in several modes also digitally stabilizes ie rotates. Worked with lots of dimensional photography measurements with Pixel. Nothing here is limiting really. Maybe there is some quality giveups. This isn’t film projector 3D where you lose the effect if your eyes aren’t aligned with the dual projectors. This is reconstructed 3D from two points of measurement. Do You think the effect will only work if your phone is held 100% 90 degree level? I mean your blowing me of smugly but all those points can be overcome with just a crop and tilt. It shows the typical Hackernews mentality “your wrong even though I have no experience in this space and only thought through this superficially.” Apple wants to sell more hardware.
- dagmx 3y ago
  Again, I think you’re not thinking this through, and all your hubris of having experience in the space is a call to authority rather than any actual logical correctness. 1. Nobody would want to hold the phone at an angle to shoot all the time. 2. It’s already reducing resolution to 1080p because it’s cropping in on the Ultrawide when its axis aligned. How much lower would you go to do an angled shot? The resolution does matter because these will be playing back on one of the highest ppd headsets on the market. Apple may want to sell more hardware, but that’s not particularly interesting insight. But at the same time, your theories of how it could be done on the 14 Pro would make for a rather poor user experience all around.
  - tsunamifury 3y ago
    1. The censors do not need to be physically 90 degree aligned to get depth data. And the the crop can be rotated so you do not need to hold the phone at a dutch angle... 2. The censors are 24mp and 12mp, way way way above 1080p (2.1mp). A tilted crop sample will be no issue at all. Also you are using the term 'appeal to authority' wrong as I am not referencing an outside authority here, I am referencing myself. I'm sorry I've made you a bit angry here, but just calm it down a bit and think it through. The overlayed images of two cameras are slightly mis-aligned, but the overlapping data is more than enough to produce a 2.1mp image. This can work both in traditional stereoscopic or with 3D generated depth data. I think one thing that threw me though is the the advertising images for this are clearly deceptive. They show 3D generated video that can be viewed from multiple angles, when in actually HVEC is just stereoscopic.

mlyle 3y ago

Two points are always on a single axis, but the two image sensors will still be rectangles at an odd angle relative to that axis.