4 ms·

I'm a machine learning researcher and I also have a lot of experience analyzing eye movement data. There aren't large labeled datasets that are publicly availa

I'm a machine learning researcher and I also have a lot of experience analyzing eye movement data. There aren't large labeled datasets that are publicly available in this area. Just to classify eye movements into events, you would need images of the eyes or eye velocity to classify eye events into smooth pursuit, fixation, etc. That means someone needs to label all of those in ground truth. To actually determine what someone is looking at, you have to go beyond this to have information about the scene. When looking at a screen, this is usually done with calibration, but things get more complicated when trying to predict what someone is looking at with a mobile eye tracker and a fixed scene camera. Body and head movements complicate things further (e.g., VOR) and many algorithms ignore them.

lazyjeff 10y ago
My research group has been trying to address this problem by using user interactions with a website to create this "ground truth". One assumption is that you have to look where you click. But there are more subtle user interactions that could be helpful. Video and demos and source code: http://webgazer.cs.brown.edu/ http://webgazer.cs.brown.edu/
- charlesdm 10y ago
  It's a good start, but nowhere near as accurate in comparison to the infrared trackers, though? What would it take to get it to that level?
  - stinos 10y ago
    Depends on which aspect you mean. E.g. a proper infrared tracker with high framerate, something like >120Hz, can be used to detect microsaccdes. I don't know much about webcam tech, but I doubt it can reach those framerates and at the same time low noise needed to distinguish those microsaccades. Then again, the research of the meaning of those and whether they're predictive or not isn't conclusive afaik so I wouldn't know any use for VR anyway.
hokkos 10y ago
I was wondering if you could use such a device with a small laser to get where the eyes points, it would also give velocity : https://www.youtube.com/watch?v=BZjtufbRZwA https://www.youtube.com/watch?v=BZjtufbRZwA
- jacquesm 10y ago
  Lasers and eyes don't mix very well. A single stray reflection off something spherical and you could end up with serious eye damage. Warning sign on a friends watercooled laser: 'do not look into laser with remaining eye'.
charlesdm 10y ago
If you don't mind me jumping in here, since people with eye tracking experience seem to be quite scarce: is it viable these days to build a working accurate gaze tracker that would, say, work using just an iPad camera? No hardware whatsoever, just using the camera to somewhat accurately position a dot on the screen. Every tracker I've tried is close to absolute garbage when it comes to accuracy.
- jacquesm 10y ago
  YC invested in a company called 'GazeHawk', but they didn't make it (acquihire by facebook), the page is still up though. http://www.businessinsider.com/facebook-acquires-gazehawk-for-talent-2012-3?international=true&r=US&IR=T http://www.businessinsider.com/facebook-acquires-gazehawk-fo... http://www.gazehawk.com/ http://www.gazehawk.com/
  - charlesdm 10y ago
    Yeah.. I've literally tried 20 solutions, and most can not detect gaze location well. Gazehawk included.
    - jacquesm 10y ago
      That may have been one of the reasons why they didn't make it. Eye tracking is tricky. I worked a bit with two guys developing a commercial eye tracker in the 80's, it was a pretty sophisticated optical set-up that combined the image being looked at and the image of the person looking using a 45 degree angle 50% mirror, that made the job considerably easier. One of the complicating factors was that eye movements have a lot of noise in them that our brain has become very adept at filtering out so what we think our eyes are doing can be quite different from what they're really doing.
- Bartweiss 10y ago
  Not easily. Samsung has something like this, which attempts the minimal task of "see if you're looking at the screen" to prevent dimming while you read. It's... not good. Not even passable, in my experience. That task should be achievable, but anything fancier - even positioning a dot - is probably still a tall order with that single camera.

lazyjeff 10y ago

My research group has been trying to address this problem by using user interactions with a website to create this "ground truth". One assumption is that you have to look where you click. But there are more subtle user interactions that could be helpful. Video and demos and source code: http://webgazer.cs.brown.edu/ http://webgazer.cs.brown.edu/

charlesdm 10y ago
It's a good start, but nowhere near as accurate in comparison to the infrared trackers, though? What would it take to get it to that level?
- stinos 10y ago
  Depends on which aspect you mean. E.g. a proper infrared tracker with high framerate, something like >120Hz, can be used to detect microsaccdes. I don't know much about webcam tech, but I doubt it can reach those framerates and at the same time low noise needed to distinguish those microsaccades. Then again, the research of the meaning of those and whether they're predictive or not isn't conclusive afaik so I wouldn't know any use for VR anyway.

hokkos 10y ago

I was wondering if you could use such a device with a small laser to get where the eyes points, it would also give velocity : https://www.youtube.com/watch?v=BZjtufbRZwA https://www.youtube.com/watch?v=BZjtufbRZwA

jacquesm 10y ago
Lasers and eyes don't mix very well. A single stray reflection off something spherical and you could end up with serious eye damage. Warning sign on a friends watercooled laser: 'do not look into laser with remaining eye'.

charlesdm 10y ago

If you don't mind me jumping in here, since people with eye tracking experience seem to be quite scarce: is it viable these days to build a working accurate gaze tracker that would, say, work using just an iPad camera? No hardware whatsoever, just using the camera to somewhat accurately position a dot on the screen. Every tracker I've tried is close to absolute garbage when it comes to accuracy.

jacquesm 10y ago
YC invested in a company called 'GazeHawk', but they didn't make it (acquihire by facebook), the page is still up though. http://www.businessinsider.com/facebook-acquires-gazehawk-for-talent-2012-3?international=true&r=US&IR=T http://www.businessinsider.com/facebook-acquires-gazehawk-fo... http://www.gazehawk.com/ http://www.gazehawk.com/
- charlesdm 10y ago
  Yeah.. I've literally tried 20 solutions, and most can not detect gaze location well. Gazehawk included.
  - jacquesm 10y ago
    That may have been one of the reasons why they didn't make it. Eye tracking is tricky. I worked a bit with two guys developing a commercial eye tracker in the 80's, it was a pretty sophisticated optical set-up that combined the image being looked at and the image of the person looking using a 45 degree angle 50% mirror, that made the job considerably easier. One of the complicating factors was that eye movements have a lot of noise in them that our brain has become very adept at filtering out so what we think our eyes are doing can be quite different from what they're really doing.
Bartweiss 10y ago
Not easily. Samsung has something like this, which attempts the minimal task of "see if you're looking at the screen" to prevent dimming while you read. It's... not good. Not even passable, in my experience. That task should be achievable, but anything fancier - even positioning a dot - is probably still a tall order with that single camera.