5 ms·

But 'all' you have to do is track the pupil, and push focus on the object they're looking at. We're already rendering the focal length live in most games, there

by hodwik 10y ago

But 'all' you have to do is track the pupil, and push focus on the object they're looking at. We're already rendering the focal length live in most games, there's no reason we couldn't shift the focal length affording to what the viewer's pupils are pointed at. Alternatively, they could use a modified biology, where the direction the head is pointing chooses the relative DOF, basically you render a fairly wide depth of field around whatever the head is pointed at. Anyway, you can get a pretty lifelike experience with just some basic atmospheric perspective. Look at photography with very wide depth of field [1], so everything is in focus simultaneously. You sorta just get used to it, and accept it. [1] http://cdn.shutterbug.com/images/0814picture03.jpg http://cdn.shutterbug.com/images/0814picture03.jpg

deleted 10y ago
[deleted]
modeless 10y ago
As someone who works on eye tracking for VR headsets, this is much easier said than done. Doing it well requires levels of accuracy, latency, and reliability that are not available in any eye tracker on the market today.
- rjvs 10y ago
  What about SMI already tracking pupils at 250Hz? https://www.youtube.com/watch?v=Qq09BTmjzRs https://www.youtube.com/watch?v=Qq09BTmjzRs That tech is still expensive and I don't know how accurate or reliable but if it is good enough for foveated rendering, I would have expected it to be good enough for focus as well. Care to elaborate as to why it wouldn't work, or is it just beyond your horizon of "today"?
  - modeless 10y ago
    It's not accurate or reliable enough (in addition to being very far from a consumer price point). Reliability in particular is difficult to assess and usually lacking in eye tracking systems. Also just because the cameras are running at 250 Hz doesn't mean the latency is 1/250 seconds or even close to that.
- chimprich 10y ago
  As someone working in the industry, would you care to guess how many years away we are from being able to do this at an acceptable level? Is this something for the next gen of hardware or is it much further away?
- chimprich 10y ago
  As someone working in the industry, would you care to guess how many years away we are from being able to do this at an acceptable level? Is this something for the next gen of hardware or is it much further away?
- rasz_pl 10y ago
  Only reason for lack of availability is size of the market. Tech is already here and you are probably holding it in your hand right now. Avago's will be more than happy to sell you tiny combo chip packing camera/DSP able to capture >10K frames/s, track at >5 meters/s and report position at over 1KHz.
andrewstuart2 10y ago
Except you can't fake the feedback of muscles adjusting focus that the real world requires without different physical focal planes. You can mimic the focal effect, but your brain has the ability to recognize motor inputs to the eyes and act on this information (or lack thereof). I think what we really need is a way to target specific focal distances via some sort of dynamic refraction. Otherwise we'll never get the proper motor feedback needed for immersion.
- nitrogen 10y ago
  So, dynamic lenses on the VR display.
- Natanael_L 10y ago
  Light field displays seems to be the answer. Also much more computationally heavy.
Animats 10y ago
Maybe VR would work better if you just fed the same view to both eyes and ignored stereoscopy. Jaron Lanier used to do that with his original VR rig in the 1980s when one of the machines went down. He said nobody noticed.
- milesokeefe 10y ago
  I can assure you that the tech has advanced enough in the last three decades that you would immediately notice the lack of non-parallax depth perception.
provemewrong 10y ago
The problem is your eyes can focus at different focal planes: on an object, past an object etc. I'd imagine it would be pretty jarring if the software just blurred everything around an object that happened to be at the x, y coordinates where your pupils were pointing.
- jsprogrammer 10y ago
  You project into the scene to discover the distance of the object, then set the aperture and/or focal length of your camera as appropriate.
  - marvin 10y ago
    What do you do when the pupil is ambiguously pointing at an object 30cm in front of your eyes, and a mountain on the horizon?
    - hodwik 10y ago
      Wide DOF
  - ascorbic 10y ago
    This doesn't help. You need to physically change focus in your eye. Your brain uses the feedback from the eye muscles to ascertain distance. If your eye muscles don't need to adjust focus, then they're reporting that everything is at the same distance. If this disagrees with the stereoscopy or the context then you will get nausea.
    - Natanael_L 10y ago
      Light field displays?
    - hodwik 10y ago
      > "Your brain uses the feedback from the eye muscles to ascertain distance." Source? I'm under the impression that focus is a brain-first, muscles-second process. If the object was already in focus, I don't think your eyes would mind.

by hodwik 10y ago

deleted 10y ago

[deleted]

modeless 10y ago

As someone who works on eye tracking for VR headsets, this is much easier said than done. Doing it well requires levels of accuracy, latency, and reliability that are not available in any eye tracker on the market today.

rjvs 10y ago
What about SMI already tracking pupils at 250Hz? https://www.youtube.com/watch?v=Qq09BTmjzRs https://www.youtube.com/watch?v=Qq09BTmjzRs That tech is still expensive and I don't know how accurate or reliable but if it is good enough for foveated rendering, I would have expected it to be good enough for focus as well. Care to elaborate as to why it wouldn't work, or is it just beyond your horizon of "today"?
- modeless 10y ago
  It's not accurate or reliable enough (in addition to being very far from a consumer price point). Reliability in particular is difficult to assess and usually lacking in eye tracking systems. Also just because the cameras are running at 250 Hz doesn't mean the latency is 1/250 seconds or even close to that.
chimprich 10y ago
As someone working in the industry, would you care to guess how many years away we are from being able to do this at an acceptable level? Is this something for the next gen of hardware or is it much further away?
chimprich 10y ago
As someone working in the industry, would you care to guess how many years away we are from being able to do this at an acceptable level? Is this something for the next gen of hardware or is it much further away?
rasz_pl 10y ago
Only reason for lack of availability is size of the market. Tech is already here and you are probably holding it in your hand right now. Avago's will be more than happy to sell you tiny combo chip packing camera/DSP able to capture >10K frames/s, track at >5 meters/s and report position at over 1KHz.

andrewstuart2 10y ago

Except you can't fake the feedback of muscles adjusting focus that the real world requires without different physical focal planes. You can mimic the focal effect, but your brain has the ability to recognize motor inputs to the eyes and act on this information (or lack thereof). I think what we really need is a way to target specific focal distances via some sort of dynamic refraction. Otherwise we'll never get the proper motor feedback needed for immersion.

nitrogen 10y ago
So, dynamic lenses on the VR display.
Natanael_L 10y ago
Light field displays seems to be the answer. Also much more computationally heavy.

Animats 10y ago

Maybe VR would work better if you just fed the same view to both eyes and ignored stereoscopy. Jaron Lanier used to do that with his original VR rig in the 1980s when one of the machines went down. He said nobody noticed.

milesokeefe 10y ago
I can assure you that the tech has advanced enough in the last three decades that you would immediately notice the lack of non-parallax depth perception.

provemewrong 10y ago

The problem is your eyes can focus at different focal planes: on an object, past an object etc. I'd imagine it would be pretty jarring if the software just blurred everything around an object that happened to be at the x, y coordinates where your pupils were pointing.

jsprogrammer 10y ago
You project into the scene to discover the distance of the object, then set the aperture and/or focal length of your camera as appropriate.
- marvin 10y ago
  What do you do when the pupil is ambiguously pointing at an object 30cm in front of your eyes, and a mountain on the horizon?
  - hodwik 10y ago
    Wide DOF
- ascorbic 10y ago
  This doesn't help. You need to physically change focus in your eye. Your brain uses the feedback from the eye muscles to ascertain distance. If your eye muscles don't need to adjust focus, then they're reporting that everything is at the same distance. If this disagrees with the stereoscopy or the context then you will get nausea.
  - Natanael_L 10y ago
    Light field displays?
  - hodwik 10y ago
    > "Your brain uses the feedback from the eye muscles to ascertain distance." Source? I'm under the impression that focus is a brain-first, muscles-second process. If the object was already in focus, I don't think your eyes would mind.