12 ms·

This is a fun experiment, but reminded me why I didn't like 3D movies: You need to look right at where they want you to look - so in this example you need to l

This is a fun experiment, but reminded me why I didn't like 3D movies: You need to look right at where they want you to look - so in this example you need to look right at the center point of the image. If your gave drifts off to the side then it looks bad. The same with 3D movies, if you don't want to look directly at what the director wanted you to look at, you end up getting seasick. Do VR headsets have gaze tracking these days?

nullifidian 6y ago
>gaze tracking these days? They do (e.g. Vive Pro Eye, HP Reverb G2 Omnicept Edition). And checking that on google is much faster than writing this sentence.
082349872349872 6y ago
the same with 2D movies. Anime taught me to always look in backgrounds, a habit which makes it clear that live action frequently uses focus to force attention.
- milgrim 6y ago
  Some Netflix series are really bad in this regard. Sometimes the upper and lower part of the picture are really blurry. Maybe they do this to achieve a more cinematic look? But to me it feels like forced tunnel vision and it makes me really uncomfortable.
ansible 6y ago
> The same with 3D movies, if you don't want to look directly at what the director wanted you to look at, you end up getting seasick. They could do statistical gaze tracking on a 2D version of the movie scene using a small test audience, and then render the 3D scene based on the results.
- Pfhreak 6y ago
  There will likely always be people impacted by this. For me, human faces are generally not the interesting part of the screen. I'm looking at clothing, set design, fx, etc. I noticed this constantly watching avatar in 3D. I happen to also struggle with human face recognition, so it's likely I'm just wired different.
w-m 6y ago
Some games (not sure if all) on the Oculus Quest use fixed foveated rendering, no gaze tracking. If you look to the side of the display, you can notice the rendering getting blocky. If you search for it, you can make out a border from sharp rendering to blocky rendering, but it's seldom noticeable in normal gaming. Since the field of view is limited in today's VR displays anyhow, you don't lose much from the foveated rendering: if you look to the side of display, much of your own field of view is now black, which is more limiting than the fact that the few pixels that you still see are rendered in a blocky fashion.
- JonathanFly 6y ago
  In Carmack's talk this year he said fovisted rendering was basically a mistake. A lot of it was due to hardware implementation details, but it didn't save much GPU resources for the quality hit you took. One nice benefit of the Quest 2 is that it can run Quest 2 games without be it. It's still a feature in the Oculus runtime but they only recommend it in specific situations now, and they try and manage the quality outside of the game because way too many developers just cranked it to Max and it made their games look terrible.
  - taneq 6y ago
    I don't remember him saying it was a mistake, just that it wasn't the magic panacea that people were expecting to catapult us to full retina-resolution VR. It was still a significant improvement (and combined with eye tracking, didn't constitute a quality hit) iirc.
    - jayd16 6y ago
      He was saying fixed foveated is not a significant improvement if you set it to a level that isn't perceptible. You can easily tell when games have it cranked up and the quality drop off at the edge is very severe. Perf improvements are only moderate.
      - taneq 6y ago
1000100_1000101 6y ago
I don't think any current headsets do gaze tracking, but it is something Sony is looking at. https://venturebeat.com/2019/05/23/playstations-dominic-mallinson-on-next-gen-vr-wireless-gaze-tracking-foveated-rendering-and-ar/ https://venturebeat.com/2019/05/23/playstations-dominic-mall...
- Apofis 6y ago
  That would certainly give VR headsets a huge performance boost.
- cjsawyer 6y ago
  Vive Pro Eye and Varjo VR2 are some examples. I expect the functionality to be a part of most next gen headsets
underanalyzer 6y ago
I would definitely not dismiss this technique b/c of my quick prototype. There's a ton of work that has gone into the foveated rendering you will see on quest. Check out this paper: https://ai.facebook.com/blog/deepfovea-using-deep-learning-for-foveated-reconstruction-in-ar-vr https://ai.facebook.com/blog/deepfovea-using-deep-learning-f... (that I link to in my post) to see a what a real version of this technique looks like.
kristiandupont 6y ago
When you say "3D movies", do you mean animated ones? Because they just simulate shallow depth of field which you also get from real lenses (given the right configuration). It's completely normal to use it as a tool to direct attention in movies.
- jayd16 6y ago
  They mean stereoscopic movies. It is a normal camera trick but it doesn't work well in 3D? Adjusting your eye's stereo focus differently than the lens focus can be too straining.

nullifidian 6y ago

>gaze tracking these days? They do (e.g. Vive Pro Eye, HP Reverb G2 Omnicept Edition). And checking that on google is much faster than writing this sentence.

082349872349872 6y ago

the same with 2D movies. Anime taught me to always look in backgrounds, a habit which makes it clear that live action frequently uses focus to force attention.

milgrim 6y ago
Some Netflix series are really bad in this regard. Sometimes the upper and lower part of the picture are really blurry. Maybe they do this to achieve a more cinematic look? But to me it feels like forced tunnel vision and it makes me really uncomfortable.

ansible 6y ago

> The same with 3D movies, if you don't want to look directly at what the director wanted you to look at, you end up getting seasick. They could do statistical gaze tracking on a 2D version of the movie scene using a small test audience, and then render the 3D scene based on the results.

Pfhreak 6y ago
There will likely always be people impacted by this. For me, human faces are generally not the interesting part of the screen. I'm looking at clothing, set design, fx, etc. I noticed this constantly watching avatar in 3D. I happen to also struggle with human face recognition, so it's likely I'm just wired different.

w-m 6y ago

Some games (not sure if all) on the Oculus Quest use fixed foveated rendering, no gaze tracking. If you look to the side of the display, you can notice the rendering getting blocky. If you search for it, you can make out a border from sharp rendering to blocky rendering, but it's seldom noticeable in normal gaming. Since the field of view is limited in today's VR displays anyhow, you don't lose much from the foveated rendering: if you look to the side of display, much of your own field of view is now black, which is more limiting than the fact that the few pixels that you still see are rendered in a blocky fashion.

JonathanFly 6y ago
In Carmack's talk this year he said fovisted rendering was basically a mistake. A lot of it was due to hardware implementation details, but it didn't save much GPU resources for the quality hit you took. One nice benefit of the Quest 2 is that it can run Quest 2 games without be it. It's still a feature in the Oculus runtime but they only recommend it in specific situations now, and they try and manage the quality outside of the game because way too many developers just cranked it to Max and it made their games look terrible.
- taneq 6y ago
  I don't remember him saying it was a mistake, just that it wasn't the magic panacea that people were expecting to catapult us to full retina-resolution VR. It was still a significant improvement (and combined with eye tracking, didn't constitute a quality hit) iirc.
  - jayd16 6y ago
    He was saying fixed foveated is not a significant improvement if you set it to a level that isn't perceptible. You can easily tell when games have it cranked up and the quality drop off at the edge is very severe. Perf improvements are only moderate.
    - taneq 6y ago

1000100_1000101 6y ago

I don't think any current headsets do gaze tracking, but it is something Sony is looking at. https://venturebeat.com/2019/05/23/playstations-dominic-mallinson-on-next-gen-vr-wireless-gaze-tracking-foveated-rendering-and-ar/ https://venturebeat.com/2019/05/23/playstations-dominic-mall...

Apofis 6y ago
That would certainly give VR headsets a huge performance boost.
cjsawyer 6y ago
Vive Pro Eye and Varjo VR2 are some examples. I expect the functionality to be a part of most next gen headsets

underanalyzer 6y ago

I would definitely not dismiss this technique b/c of my quick prototype. There's a ton of work that has gone into the foveated rendering you will see on quest. Check out this paper: https://ai.facebook.com/blog/deepfovea-using-deep-learning-for-foveated-reconstruction-in-ar-vr https://ai.facebook.com/blog/deepfovea-using-deep-learning-f... (that I link to in my post) to see a what a real version of this technique looks like.

kristiandupont 6y ago

When you say "3D movies", do you mean animated ones? Because they just simulate shallow depth of field which you also get from real lenses (given the right configuration). It's completely normal to use it as a tool to direct attention in movies.

jayd16 6y ago
They mean stereoscopic movies. It is a normal camera trick but it doesn't work well in 3D? Adjusting your eye's stereo focus differently than the lens focus can be too straining.