7 ms·

Eye tracking in video games

53 points by Phithagoras 9y ago

WrtCdEvrydy 9y ago
Now we just need cheaper dual eye tracking setups and support for lower quality sections in your screen (just lower quality where you're not looking). Open source community to the rescue!
- rtkwe 9y ago
  What you're talking about is called foveated rendering and it's already used today in VR to reduce the rendering workload around the edges.
  - SimonPStevens 9y ago
    I'm not sure I'd quite class it as in use yet, when none of the major commercially available VR headsets use it. It's certainly being actively researched though and seems likely to be part of a future generation of VR headsets.
    - rtkwe 9y ago
      Eye tracking will require new headsets or new accessories but basic foveated rendering which just goes to the center could be done on today's VR hardware.
- melling 9y ago
  I'd like an open source solution that I could use in my editor to navigate to a particular word, end of line, etc.
  - Crespyl 9y ago
    In my window manager I make heavy use of "focus-follows-mouse" behavior, but what I really want most of the time is "focus-follows-eye". Especially in multi-monitor situations, I could see this being really invisibly helpful.
pandaman 9y ago
No mention of GT5? It tells how impressive is this feature when few people remember a game that shipped 10+M units with it, does not it? I figure it could be much more useful in AR applications than in games but, even there, it's a long shot (remember Amazon Fire Phone?).
- modeless 9y ago
  You're thinking of head tracking. Eye tracking is very different and neither GT5 nor the Fire Phone had it.
  - pandaman 9y ago
    Fire Phone definitely had it (see https://www.wired.com/2014/06/amazon-fire-phone/ https://www.wired.com/2014/06/amazon-fire-phone/ for example), GT5 called the feature "Head Tracking" but, since it is played on a fixed display it was essentially tracking the direction of gaze in that setup, from the POV of gameplay I don't see much difference between that and true eye tracking.
    - dahauns 9y ago
      since it is played on a fixed display it was essentially tracking the direction of gaze in that setup, from the POV of gameplay I don't see much difference between that and true eye tracking. Absolutely not. Head and eye movements might complement each other, but they are completely different in execution. And you can't reliably track gaze from head movements alone. Focusing on distinct objects by looking at them, for example, is something you can only do with eye tracking (except for the most extremely coarse cases where there's only one object in the general direction of the head). BTW: Head tracking in games is nothing new, TrackIR e.g. has been around and a staple in sim racers for ~15 years now.
      - pandaman 9y ago
        In GT5, when you looked in the door mirror, the camera view shifted there. How could that be different with eye tracking?
katamaritaco 9y ago
Although I don't see eye tracking in games as 'the next big thing' in any sense (the few I've tried were severely lacking in the fun department), I would be interested to try playing Soma with this. Soma had some flaws to be sure, but damn did it do some things really well. I'd highly recommend it - eye tracking or no.
- astrobe_ 9y ago
  One great application I can think of is depth of field adjustment. Some games have depth of field simulation, which makes far away scenery blurry. Personally I always disable it because I think it belongs to the uncanny valley: it's nice on screenshots, but it doesn't work so well in-game because the focus doesn't change with where/what you're looking at. Eye tracking, by making this feature dynamic, would I think enhance 3D rendering for relatively cheap.
  - Crespyl 9y ago
    Good use of eye tracking could also allow for foveated rendering, drawing the out-of-focus areas with lower resolution and leaving more time to render the areas of interest.
hashhar 9y ago
For everyone who hasn't played SOMA, give it a try if you like video games and/or philosphy (related to self discovery). The game itself has great mechanics, environment and audio. But the story is what makes me like the game more than anything. It makes you think.
- flanbiscuit 9y ago
  I 2nd this. Loved the game. Played it just a couple months ago for the first time. It's not as scary as the developer's other game, Amnesia: The Dark Descent, but I actually liked SOMA's story more.
  - pizza234 9y ago
    I think the game is significantly misunderstood (although fortunately, still appreciated) - IMHO, the developers wanted to make an unsettling game, rather than a scary one, so the typical comparison with Amnesia, which is a pure horror, is somewhat misplaced.
    - gambiting 9y ago
      I actually really recommend playing it with the mod that makes almost all monsters non-aggressive. It's great for players who are not into horror games, and I would argue that it makes the game even more unsettling once you realize what the monsters are.
- cyxxon 9y ago
  As an off-topic counterpoint: I heard so much about SOMA, but I actually didn't get out of my apartment because of game mechanics (or lack of discoverability thereof). I actually didn't know what to do, as in what to click, drag, toggle, ... nothing realy worked, so I stopped.
  - hashhar 9y ago
    I agree with that. They just kind of throw you in there and hope you'd figure it out. Add to it the fact the mechanics are mostly unique to that game only so there was no familiarity to help.
Arizhel 9y ago
All the best video games were published before 1990.
- dkersten 9y ago
  Citation needed. This is clearly subjective. There are many aspects of games and many games implementing these aspects. Some pre-1990 games excel at some of them and some newer games excel at others. Apples and oranges. For example, as a medium for interactive storytelling, in my own opinion, only very recently have games managed it in a way that I found more engaging than non-interactive mediums in terms of storytelling impact. Part of this, for me, is that decisions actually impact the gameplay and continuing story and things like lifelike facial animation. When engaging visuals are combined with rock solid storytelling, you've got a great movie, but when you add solid interactivity and gameplay, it really becomes something else. This just wasn't possible pre-1990. You're obviously talking about the simple "small number of mechanics" pure gameplay games (the tetrises, bombermans, pacmans, space invaders etc), sure, there are some great and long-lasting games, but there are plenty of amazing modern games too in terms of gameplay (eg Rocket league).
  - Arizhel 9y ago
    No citation needed, it's the truth. But of course it's subjective, just like any opinion, which is why asking for a citation is simply absurd. That's like asking for a citation when someone says "I like ham sandwiches".
    - dkersten 9y ago
      It's not the truth, it's your opinion. You said so yourself, after all.
skocznymroczny 9y ago
I don't think eye tracking will have much use for gameplay. It will have much use for ads though, "your eyes have moved away from the banner for more than 500 milliseconds, please try again, and remember, no looking away for 20 seconds"
- jbrooksuk 9y ago
  Very Black Mirror-esque.

53 points by Phithagoras 9y ago

Now we just need cheaper dual eye tracking setups and support for lower quality sections in your screen (just lower quality where you're not looking). Open source community to the rescue!

rtkwe 9y ago
What you're talking about is called foveated rendering and it's already used today in VR to reduce the rendering workload around the edges.
- SimonPStevens 9y ago
  I'm not sure I'd quite class it as in use yet, when none of the major commercially available VR headsets use it. It's certainly being actively researched though and seems likely to be part of a future generation of VR headsets.
  - rtkwe 9y ago
    Eye tracking will require new headsets or new accessories but basic foveated rendering which just goes to the center could be done on today's VR hardware.
melling 9y ago
I'd like an open source solution that I could use in my editor to navigate to a particular word, end of line, etc.
- Crespyl 9y ago
  In my window manager I make heavy use of "focus-follows-mouse" behavior, but what I really want most of the time is "focus-follows-eye". Especially in multi-monitor situations, I could see this being really invisibly helpful.

pandaman 9y ago

No mention of GT5? It tells how impressive is this feature when few people remember a game that shipped 10+M units with it, does not it? I figure it could be much more useful in AR applications than in games but, even there, it's a long shot (remember Amazon Fire Phone?).

modeless 9y ago
You're thinking of head tracking. Eye tracking is very different and neither GT5 nor the Fire Phone had it.
- pandaman 9y ago
  Fire Phone definitely had it (see https://www.wired.com/2014/06/amazon-fire-phone/ https://www.wired.com/2014/06/amazon-fire-phone/ for example), GT5 called the feature "Head Tracking" but, since it is played on a fixed display it was essentially tracking the direction of gaze in that setup, from the POV of gameplay I don't see much difference between that and true eye tracking.
  - dahauns 9y ago
    since it is played on a fixed display it was essentially tracking the direction of gaze in that setup, from the POV of gameplay I don't see much difference between that and true eye tracking. Absolutely not. Head and eye movements might complement each other, but they are completely different in execution. And you can't reliably track gaze from head movements alone. Focusing on distinct objects by looking at them, for example, is something you can only do with eye tracking (except for the most extremely coarse cases where there's only one object in the general direction of the head). BTW: Head tracking in games is nothing new, TrackIR e.g. has been around and a staple in sim racers for ~15 years now.
    - pandaman 9y ago
      In GT5, when you looked in the door mirror, the camera view shifted there. How could that be different with eye tracking?

katamaritaco 9y ago

Although I don't see eye tracking in games as 'the next big thing' in any sense (the few I've tried were severely lacking in the fun department), I would be interested to try playing Soma with this. Soma had some flaws to be sure, but damn did it do some things really well. I'd highly recommend it - eye tracking or no.

astrobe_ 9y ago
One great application I can think of is depth of field adjustment. Some games have depth of field simulation, which makes far away scenery blurry. Personally I always disable it because I think it belongs to the uncanny valley: it's nice on screenshots, but it doesn't work so well in-game because the focus doesn't change with where/what you're looking at. Eye tracking, by making this feature dynamic, would I think enhance 3D rendering for relatively cheap.
- Crespyl 9y ago
  Good use of eye tracking could also allow for foveated rendering, drawing the out-of-focus areas with lower resolution and leaving more time to render the areas of interest.

hashhar 9y ago

For everyone who hasn't played SOMA, give it a try if you like video games and/or philosphy (related to self discovery). The game itself has great mechanics, environment and audio. But the story is what makes me like the game more than anything. It makes you think.

flanbiscuit 9y ago
I 2nd this. Loved the game. Played it just a couple months ago for the first time. It's not as scary as the developer's other game, Amnesia: The Dark Descent, but I actually liked SOMA's story more.
- pizza234 9y ago
  I think the game is significantly misunderstood (although fortunately, still appreciated) - IMHO, the developers wanted to make an unsettling game, rather than a scary one, so the typical comparison with Amnesia, which is a pure horror, is somewhat misplaced.
  - gambiting 9y ago
    I actually really recommend playing it with the mod that makes almost all monsters non-aggressive. It's great for players who are not into horror games, and I would argue that it makes the game even more unsettling once you realize what the monsters are.
cyxxon 9y ago
As an off-topic counterpoint: I heard so much about SOMA, but I actually didn't get out of my apartment because of game mechanics (or lack of discoverability thereof). I actually didn't know what to do, as in what to click, drag, toggle, ... nothing realy worked, so I stopped.
- hashhar 9y ago
  I agree with that. They just kind of throw you in there and hope you'd figure it out. Add to it the fact the mechanics are mostly unique to that game only so there was no familiarity to help.

Arizhel 9y ago

All the best video games were published before 1990.

dkersten 9y ago
Citation needed. This is clearly subjective. There are many aspects of games and many games implementing these aspects. Some pre-1990 games excel at some of them and some newer games excel at others. Apples and oranges. For example, as a medium for interactive storytelling, in my own opinion, only very recently have games managed it in a way that I found more engaging than non-interactive mediums in terms of storytelling impact. Part of this, for me, is that decisions actually impact the gameplay and continuing story and things like lifelike facial animation. When engaging visuals are combined with rock solid storytelling, you've got a great movie, but when you add solid interactivity and gameplay, it really becomes something else. This just wasn't possible pre-1990. You're obviously talking about the simple "small number of mechanics" pure gameplay games (the tetrises, bombermans, pacmans, space invaders etc), sure, there are some great and long-lasting games, but there are plenty of amazing modern games too in terms of gameplay (eg Rocket league).
- Arizhel 9y ago
  No citation needed, it's the truth. But of course it's subjective, just like any opinion, which is why asking for a citation is simply absurd. That's like asking for a citation when someone says "I like ham sandwiches".
  - dkersten 9y ago
    It's not the truth, it's your opinion. You said so yourself, after all.

skocznymroczny 9y ago

I don't think eye tracking will have much use for gameplay. It will have much use for ads though, "your eyes have moved away from the banner for more than 500 milliseconds, please try again, and remember, no looking away for 20 seconds"

jbrooksuk 9y ago
Very Black Mirror-esque.