3 ms·

Online WebCam Eye-Tracking API

10 points by szydej 6y ago

myth_drannon 6y ago
Can it replace the eye gaze devices for disabled people? They are expensive... If you can just use regular tablet's camera that would be amazing, even if you can only use the browser.
- movedx 6y ago
  This is a good example of this kind of tech being used for good! I like it.
- TYPE_FASTER 6y ago
  Native iOS and Android apps can do this without using the cloud.
  - myth_drannon 6y ago
    Hmmm, when why someone would pay +1k$ for a camera like dynavox's one ... https://us.tobiidynavox.com/collections/devices/products/pceye https://us.tobiidynavox.com/collections/devices/products/pce...
- szydej 6y ago
  You can Write with your eyes. Try GazeBoard :https://gazerecorder.com/gazeboard/ https://gazerecorder.com/gazeboard/ Eye Tracking and text entry using only an ordinary webcam.
movedx 6y ago
Nice! But no. We don't need Black Mirror to be coming true. We really don't need websites showing adverts which, if looked away from, prevent the website from ever rendering - watch the advert or no website for you! I understand the technical innovation here, but this can be abused a lot. In fact I think it can be abused more than it can be put to good use. How does everyone else feel about this? How can this be used for good and evil?
- Normal_gaussian 6y ago
  Im surprised it works so well. Good: - Video chats give a multitile view and bump the audio of the person you are looking at - websites pre-fetch areas of content you are likely to click on - better designed sites as its way cheaper to get decent attention / confusion metrics for a test audience The bad is advertising / snooping etc. I'm actually quite taken and wondering if I can replace my mouse with gaze tracking. I already use i3 and mouse position for window focus. Would be nice to keep my jands on the keyboard for all apps.
  - wanderingstan 6y ago
    Years ago at CES there was an eye tracking demo that would automatically scroll the page as you read text. It was so subtle; you never actually noticed the text moving, but then you also never had to scroll.
    - szydej 6y ago
      Try GazeScroll Chrome Extension https://gazerecorder.com/gazescroll/ https://gazerecorder.com/gazescroll/
  - szydej 6y ago
    You can control mouse cursor with your eyes. Check out GazePointer: https://gazerecorder.com/gazepointer-beta/ https://gazerecorder.com/gazepointer-beta/
rektide 6y ago
TensorFlow announced their new Facial Landmarks model gained Iris features last month[1]. There's an online browser-only demo[2]. Note that this model doesn't say where the user is looking, it just extracts 2d locations of iris & surrounding face. Older software like WebGazer.js[3] provides more complete apis, reporting where on the screen someone is looking. The approach here seems to be server side. I wonder what tech they are using under the hood. [1] https://blog.tensorflow.org/2020/11/iris-landmark-tracking-in-browser-with-MediaPipe-and-TensorFlowJS.html https://blog.tensorflow.org/2020/11/iris-landmark-tracking-i... [2] https://storage.googleapis.com/tfjs-models/demos/face-landmarks-detection/index.html https://storage.googleapis.com/tfjs-models/demos/face-landma... [3] https://webgazer.cs.brown.edu/ https://webgazer.cs.brown.edu/
wanderingstan 6y ago
There was a company 8ish years ago trying to do the same thing; they ended up getting aqui-hired by Facebook: https://www.zdnet.com/article/facebook-acquires-eye-tracking-startup-gazehawk/ https://www.zdnet.com/article/facebook-acquires-eye-tracking... Since then I always wondered when webcam-eye-tracking would actually be possible. Looks like today is the day!

10 points by szydej 6y ago

Can it replace the eye gaze devices for disabled people? They are expensive... If you can just use regular tablet's camera that would be amazing, even if you can only use the browser.

movedx 6y ago
This is a good example of this kind of tech being used for good! I like it.
TYPE_FASTER 6y ago
Native iOS and Android apps can do this without using the cloud.
- myth_drannon 6y ago
  Hmmm, when why someone would pay +1k$ for a camera like dynavox's one ... https://us.tobiidynavox.com/collections/devices/products/pceye https://us.tobiidynavox.com/collections/devices/products/pce...
szydej 6y ago
You can Write with your eyes. Try GazeBoard :https://gazerecorder.com/gazeboard/ https://gazerecorder.com/gazeboard/ Eye Tracking and text entry using only an ordinary webcam.

movedx 6y ago

Nice! But no. We don't need Black Mirror to be coming true. We really don't need websites showing adverts which, if looked away from, prevent the website from ever rendering - watch the advert or no website for you! I understand the technical innovation here, but this can be abused a lot. In fact I think it can be abused more than it can be put to good use. How does everyone else feel about this? How can this be used for good and evil?

Normal_gaussian 6y ago
Im surprised it works so well. Good: - Video chats give a multitile view and bump the audio of the person you are looking at - websites pre-fetch areas of content you are likely to click on - better designed sites as its way cheaper to get decent attention / confusion metrics for a test audience The bad is advertising / snooping etc. I'm actually quite taken and wondering if I can replace my mouse with gaze tracking. I already use i3 and mouse position for window focus. Would be nice to keep my jands on the keyboard for all apps.
- wanderingstan 6y ago
  Years ago at CES there was an eye tracking demo that would automatically scroll the page as you read text. It was so subtle; you never actually noticed the text moving, but then you also never had to scroll.
  - szydej 6y ago
    Try GazeScroll Chrome Extension https://gazerecorder.com/gazescroll/ https://gazerecorder.com/gazescroll/
- szydej 6y ago
  You can control mouse cursor with your eyes. Check out GazePointer: https://gazerecorder.com/gazepointer-beta/ https://gazerecorder.com/gazepointer-beta/

rektide 6y ago

TensorFlow announced their new Facial Landmarks model gained Iris features last month[1]. There's an online browser-only demo[2]. Note that this model doesn't say where the user is looking, it just extracts 2d locations of iris & surrounding face. Older software like WebGazer.js[3] provides more complete apis, reporting where on the screen someone is looking. The approach here seems to be server side. I wonder what tech they are using under the hood. [1] https://blog.tensorflow.org/2020/11/iris-landmark-tracking-in-browser-with-MediaPipe-and-TensorFlowJS.html https://blog.tensorflow.org/2020/11/iris-landmark-tracking-i... [2] https://storage.googleapis.com/tfjs-models/demos/face-landmarks-detection/index.html https://storage.googleapis.com/tfjs-models/demos/face-landma... [3] https://webgazer.cs.brown.edu/ https://webgazer.cs.brown.edu/

wanderingstan 6y ago

There was a company 8ish years ago trying to do the same thing; they ended up getting aqui-hired by Facebook: https://www.zdnet.com/article/facebook-acquires-eye-tracking-startup-gazehawk/ https://www.zdnet.com/article/facebook-acquires-eye-tracking... Since then I always wondered when webcam-eye-tracking would actually be possible. Looks like today is the day!