4 ms·

This is not the same as the Vision Pro's display. This paper describes tracking a single other person and displaying a perspective-correct rendering for them, b

This is not the same as the Vision Pro's display. This paper describes tracking a single other person and displaying a perspective-correct rendering for them, but the Vision Pro displays a perspective-correct rendering for many viewpoints at once using a lenticular screen. Apple's solution works for >1 people at the same time and doesn't require any external tracking (though it's already doing the external tracking regardless), at the cost of lower resolution and only being correct in one dimension vs two.

ramboldio 3y ago
Adding multiple viewpoints is actually sth Meta first proposed, based on the paper from above: https://research.facebook.com/blog/2021/08/display-systems-research-reverse-passthrough-vr/ https://research.facebook.com/blog/2021/08/display-systems-r...
- jessriedel 3y ago
  More detail from that post: > There are several established ways to display 3D images. For this research, we used a microlens-array light field display because it’s thin, simple to construct, and based on existing consumer LCD technology. These displays use a tiny grid of lenses that send light from different LCD pixels out in different directions, with the effect that an observer sees a different image when looking at the display from different directions. The perspective of the images shift naturally so that any number of people in the room can look at the light field display and see the correct perspective for their location. > As with any early stage research prototype, this hardware still carries significant limitations: First, the viewing angle can’t be too severe, and second, the prototype can only show objects in sharp focus that are within a few centimeters of the physical screen surface. Conversations take place face-to-face, which naturally limits reverse passthrough viewing angles. And the wearer’s face is only a few centimeters from the physical screen surface, so the technology works well for this case — and will work even better if VR headsets continue to shrink in size, using methods such as holographic optics.
jessriedel 3y ago
Do you know where one could read more about Apple's technique? I don't know much about lenticular displays or why the trick only works in one direction (presumably the horizontal one).
- ramboldio 3y ago
  Facebooks paper on their technology is quite amazing: https://dl.acm.org/doi/10.1145/3450550.3465338 https://dl.acm.org/doi/10.1145/3450550.3465338 I'm guessing (?) Apple's approach is similar.
- ladberg 3y ago
  Think of it like those movie posters or bookmarks that change as you move from side to side, but with a screen behind it. The Wikipedia article might explain it better: https://en.wikipedia.org/wiki/Lenticular_lens https://en.wikipedia.org/wiki/Lenticular_lens It could work in both dimensions but you're sacrificing even more resolution by doing it that way. For example imagine you have a 1000x1000 pixel display (I just made this resolution up) and you stick a 1D lenticular screen on top with a pitch of 10 pixels. You've effectively split the display into 10 separate 100x1000 displays that are each view from a different angle. You could instead use a 2D lenticular screen and split it up into 100 100x100 displays viewable from a different angle in a 10x10 grid at virtually no extra $ cost. However, you're displaying at 1/10th the resolution just to be able to support perspective-correct views from above or below, which are way less common than from the side.
cubefox 3y ago
Please provide a source which says that Apple's solution works for more than one person. I'm pretty sure they didn't say anything about that.
- ladberg 3y ago
  https://www.youtube.com/live/GYkq9Rgoj8E?t=6729 https://www.youtube.com/live/GYkq9Rgoj8E?t=6729
  - cubefox 3y ago
    Okay, "each person" does indeed sound like "multiple at the same time", but it could also mean "any single person". Then the lenticular lens display would only need to produce two images, for the left and the right eye.
    - jonhohle 3y ago
      That would be similar to the 3DS, right?
      - cubefox 3y ago
        Like the "New 3DS", yes. The original 3DS had no eye tracking, and a parallax barrier instead of a lenticular lens.

ramboldio 3y ago

Adding multiple viewpoints is actually sth Meta first proposed, based on the paper from above: https://research.facebook.com/blog/2021/08/display-systems-research-reverse-passthrough-vr/ https://research.facebook.com/blog/2021/08/display-systems-r...

jessriedel 3y ago
More detail from that post: > There are several established ways to display 3D images. For this research, we used a microlens-array light field display because it’s thin, simple to construct, and based on existing consumer LCD technology. These displays use a tiny grid of lenses that send light from different LCD pixels out in different directions, with the effect that an observer sees a different image when looking at the display from different directions. The perspective of the images shift naturally so that any number of people in the room can look at the light field display and see the correct perspective for their location. > As with any early stage research prototype, this hardware still carries significant limitations: First, the viewing angle can’t be too severe, and second, the prototype can only show objects in sharp focus that are within a few centimeters of the physical screen surface. Conversations take place face-to-face, which naturally limits reverse passthrough viewing angles. And the wearer’s face is only a few centimeters from the physical screen surface, so the technology works well for this case — and will work even better if VR headsets continue to shrink in size, using methods such as holographic optics.

jessriedel 3y ago

Do you know where one could read more about Apple's technique? I don't know much about lenticular displays or why the trick only works in one direction (presumably the horizontal one).

ramboldio 3y ago
Facebooks paper on their technology is quite amazing: https://dl.acm.org/doi/10.1145/3450550.3465338 https://dl.acm.org/doi/10.1145/3450550.3465338 I'm guessing (?) Apple's approach is similar.
ladberg 3y ago
Think of it like those movie posters or bookmarks that change as you move from side to side, but with a screen behind it. The Wikipedia article might explain it better: https://en.wikipedia.org/wiki/Lenticular_lens https://en.wikipedia.org/wiki/Lenticular_lens It could work in both dimensions but you're sacrificing even more resolution by doing it that way. For example imagine you have a 1000x1000 pixel display (I just made this resolution up) and you stick a 1D lenticular screen on top with a pitch of 10 pixels. You've effectively split the display into 10 separate 100x1000 displays that are each view from a different angle. You could instead use a 2D lenticular screen and split it up into 100 100x100 displays viewable from a different angle in a 10x10 grid at virtually no extra $ cost. However, you're displaying at 1/10th the resolution just to be able to support perspective-correct views from above or below, which are way less common than from the side.

cubefox 3y ago

Please provide a source which says that Apple's solution works for more than one person. I'm pretty sure they didn't say anything about that.

ladberg 3y ago
https://www.youtube.com/live/GYkq9Rgoj8E?t=6729 https://www.youtube.com/live/GYkq9Rgoj8E?t=6729
- cubefox 3y ago
  Okay, "each person" does indeed sound like "multiple at the same time", but it could also mean "any single person". Then the lenticular lens display would only need to produce two images, for the left and the right eye.
  - jonhohle 3y ago
    That would be similar to the 3DS, right?
    - cubefox 3y ago
      Like the "New 3DS", yes. The original 3DS had no eye tracking, and a parallax barrier instead of a lenticular lens.