9 ms·
WebGazer.js: Eye Tracking on the Browser
- amelius 10y agoThis could be useful to predict where the user will click next, so the website can preload data, with the result that user waiting time is reduced.
- dimal 10y agoI assume it's meant for in-house user testing, not on the live web.
- dyarosla 10y ago100px accuracy. I thought eye tracking could do better than that in general? At 100px accuracy, WebGazer is more useful to advertisers than it is to consumers, as I can't see any consumer-friendly applications that would be useful with that precision. If that's the case, adoption won't grow. Are there other comparable eye tracking libraries out now?
- falcolas 10y ago> as I can't see any consumer-friendly applications that would be useful with that precision How about: - switching focus between windows - application selection a'la mission-control - coarse navigation through a document via a sidebar - real-time inventory management in a HTML5 game - Selective zoom of embedded images Even more fun: Atom is run inside a web browser. Imagine what you could do with eye tracking within an IDE.
- Morgawr 10y ago>Imagine what you could do with eye tracking within an IDE. You provided really good cases for the rest, but for this I can't quite picture a need (other than maybe scrolling, which is arguably faster using vim movements already). Do you have any ideas?
- falcolas 10y agoThere are a number of activities in an IDE which require keystrokes or mouse movement, or changing contexts. The capability to browse through code or documentation without having to change contexts would be valuable to me; and I can't be the only one able to finish typing a thought while my eyes move on to read other code. Minor use cases, perhaps, but something that operates intuitively has the potential to be pretty awesome.
- danneu 10y agoMaybe it could be used for modal keybindings if your gaze changes modes e.g. focus between the panes of an IDE. hjkl moves the cursor when looking at the code, but it traverses the file hierarchy when looking at the sidebar, and enumerates tabs when looking at widgets. Kind of like how keys are reused in Vim yet have similar meanings across each mode. You don't have to remember so many unique keybindings. Some editors let you switch between panes with keybindings, but once you have more than a few or some in weird shapes, you end up needing tmux's `C-b q` pane selector. Seems like gaze could replace most of that.
- dyarosla 10y agoThese would probably work - switching focus between windows - application selection a'la mission-control Whereas these, no way at 100px precision or without external sensor (touch/mouse/etc) would this work at all nicely/without jitter. - coarse navigation through a document via a sidebar - real-time inventory management in a HTML5 game - Selective zoom of embedded images
- nikki-9696 10y agoAs a user, I find this creepy as fk. I would love to know how to disable this sort of "feature" on my phone.
- huskyr 10y agoIf you have an Apple device you don't need to disable it: webcam video streams (aka getUserMedia()) isn't supported on Safari. On other browsers you always need to give permission before the camera is switched on.
- nikki-9696 10y agoOh, great to know, thanks!
- CommanderData 10y agoI think this could be useful with Digital Signage. My client uses HTML to display content and is run in browser which makes it easy to design and deploy content. He says one of the challenges is getting seamless user interactivity with his browser based digital signage. This coupled with web camera recognizing hand gestures e.g. http://hadi.io/gest.js/ http://hadi.io/gest.js/ could help grabbing user interaction inside his apps without too much hassle but right now its a janky solution compared to a real native kinect setup (I don't think its even possible to effectively connect to kinect in browser).
- j_s 10y agoSomeday soon USB devices will be accessible, thanks to the Chromebook. Not sure if that will help accessing the Kinect. https://wicg.github.io/webusb/ https://wicg.github.io/webusb/
- kecks 10y agoIt didn't work for me at all; the red dot never appeared on my screen, though the face-outline in the preview image did fit my face quite well (when it wasn't detecting my chin as my mouth).
- lazyjeff 10y agoHi, one of the authors here. It was probably not very clear on our part, but it's meant to train the eye tracking model as naturally you use the website. So it takes advantage of your interactions over time, like if you're using Gmail. So if you're doing the demo on the blank page, click in a couple of places around the screen (while looking there as you would normally), and you should start seeing the prediction.
- kecks 10y agoOh, thanks, it worked now!
- 33a 10y agoCool idea and opens up a lot of interesting applications. Unfortunately, the demos don't work reliably (on my machine the tracking was completely off the wall).
- benlower 10y agoVery interesting project. Was excited to try this but the quality of the tracking is not there yet. The face detection was off most of the time unless I positioned my camera just right. Then the eye tracking was no good. But they are only using a webcam and doing this right (see http://www.tobii.com/xperience/ http://www.tobii.com/xperience/) requires multiple IR emitters and an IR camera.
- Vespasian 10y agoWell after I figured out that I could use clicks to calibrate the tracker it was somewhat accurate (ca. 100-200 px), however hte variance was quite large. I think it's a really nice idea, however not yet precise enough to work for such tasks as user studies
- falcolas 10y ago100px gives you around 70 zones in your average desktop window. More, when you consider shared hits between zones. That's enough to do basic heat maps of a web site (such as for ad impression tracking), coarse navigation, or integration into things like editors/IDEs.
- grenoire 10y agoI was thinking about ad impressions as well; but God forbid websites are ever able to use my camera for eye tracking by bypassing the webcam permissions (or through the removal of in the future? I expect anything when it comes to content monetisation).
- falcolas 10y agoOr simply refusing access to the content without access to the camera (thinking of Forbe's ill-conceived content blocking).
- grenoire 10y agoStop giving them ideas!
- artursapek 10y agoI will laugh my ass off if it ever comes to this. Hopefully I never have to come back to this comment and weep.
- Drdrdrq 10y agoSticker over camera lens takes care of that - in this age of malware it is good to use it anyhow. Good luck tracking my eyes then! Other than thwt, I see huge potential for this in UX studies.
- fredleblanc 10y agoIn the demo, I had to click on the screen a couple of times to get the dot to start moving around. Then, it seemed to want to follow my mouse quite a bit as I moved and clicked. I was able to greatly move the red dot when I say closer to the webcam, really opened my eyes, and moved left to right, but the action was flipped for me — looking left moved the dot right, and vice versa. (I'm using an Apple Cinema Display… I've had issues before where it automatically flips the screen around in some cases.) Although not very accurate, it's certainly an interesting experiment.
- WalterSear 10y agoDid you calibrate it at all?
- kevinsimper 10y agoHow do you calibrate it?
- WalterSear 10y agoThe instructions were on the page. Get it looking at your face properly, and then look directly at the mouse cursor and click it, on spots around the page. Make sure you don't move your head, just your eyes.
- uptown 10y agoMaybe the site could detect if people have read the instructions.
- wrigby 10y agoI think this would be really useful to UX researchers! The folks I know in that space often set up screen-sharing sessions with users, and watch them interact with the UI in real time. Adding gaze tracking to this process seems like a natural progression, allowing the researchers to see where the user expects a button or UI feature to be, even if the user can't articulate that in real time.
- travjones 10y agoI think this is impressive work. My lab does eye tracking with really expensive equipment locally. It would be interesting to conduct visual/face processing studies through the web and include eye tracking data with this library. Granted the accuracy is low compared to specialized equipment and predictions are highly variable, other contributors could enhance its accuracy. I suspect that the variability in shape, size, and structure of human faces contributed greatly to the high variance/low resolution predictions issue. Maybe categorizing the user's face shape a priori would enhance predictions. That is, the library would first estimate the "type of face" and use those data to inform the eye tracking. This might not make sense--just a thought.
- alistproducer2 10y agoIt didn't work that well with the camera on my HP 8470p but great idea and I'm sure it took a lot of time and effort. I appreciate the effort.
- stephen 10y agoJust curious, but does anyone who pays attention to this space/technology know if eye tracking (with commodity-ish hardware) has gotten to the point of "replacing the mouse"? E.g. at the OS level, the pointer basically follows your gaze.
- GilbertErik 10y agoBUT BUT BUT!!! I spend so much time moving the cursor out of the way so I can read the letter under the pointer.
- trishume 10y agoYou can avoid that easily using more conservative MAGIC (actually a reference to an HCI paper) that only moves the mouse pointer when you want it to.
- trishume 10y agoEyes don't point a things we look at as accurate as a mouse does (they only aim within the fovea, about 1 degree angular, corresponding to about 1cm on the screen) and eye trackers are often far worse. This summer I built a system that uses slight head movements for refinement, allowing hands free mousing as fast as a trackpad. However, eye trackers good enough for the system to be pleasant cost $10000+. I have ideas that would allow it to work with a $100 eye tracker, but even then it has the downside that the computer vision for your mouse will peg one core.
- yetanotherjosh 10y agoAs someone who hates mice and hopes one day to have this sort of tech, I'm trying to understand what you mean by "Eyes don't point at things we look at as accurate as a mouse does" - how is it that I can choose to "look at" an individual pixel on my screen, or an individual serif on a letter, and then choose another one far less than 1cm away, and my eye focus shifts to it? That shift is not detectable by a camera? (Edit update: by "detectable" I mean theoretically detectable by a really good camera and software. In other words, it seems you are arguing the tech is impossible even theoretically due to some aspect of biology. Am I following you correctly there? Thanks)
- kevinsimper 10y agoIt sadly did not work for me either, even though I tried "calibrating" it with moving my mouse around the screen, but when i moved the mouse out of the window it was just as bad as before.
- danneu 10y agoYou need to click. Open the game and follow the mouse with your eyes while you click each 8-directional edge of the game's viewport and then click the center. Now you can let go of the mouse (don't need to move it out of the game) and the game should follow your gaze. This demo is impressive. It's accurate enough for me to isolate the orange ball from the cluster. On a website, the calibration could be done in a modal overlay. "Look at this dot while clicking it wherever it appears". After then Nth click, the modal goes away and the user lands on the website ready to gaze.
- o_____________o 10y agoThe demo you came here for https://webgazer.cs.brown.edu/collision.html https://webgazer.cs.brown.edu/collision.html Click around the screen where you're currently looking to calibrate
- martin-adams 10y agoThat is amazing. I've never used an eye-tracking device before so the experience of hitting the circles with my eyes is very surreal.
- rawnlq 10y agoIt didn't work for me until I did the following: - Zoom in so that my gaze is actually looking at distinct parts of the physical screen - Look at the cursor while clicking a few times to train it - Take off my glasses - Move way closer to the camera and hold relatively still It was a lot of hassle to get it to work but after it did work it was a lot of fun!
- metasean 10y ago> Take off my glasses I was wondering if that was the problem for me, but I'm so near-sighted that if I were close enough to actually see the cursor, there's no way the camera could see my eyes to track them. :-(
- reubensutton 10y agoI wonder if anyone has experimented with using something like this to make an incredibly inexpensive assistive technology interface
- deleted 10y ago[deleted]
- swang 10y agocool trick but we all know who will end up using this the most: advertisers. another way for them to "gauge" their reach and brand appeal. or something. literally i feel like every new thing that comes to the browser, a marketer at a company thinks, 'how can we exploit this to put more junk into the face of other people?' i am not against advertising, i just think most of them have no limits in terms of what they'll exploit at the expense of their own users.
- untog 10y agoWell, advertisers couldn't use this without asking permission for a user's webcam, which would be a non-starter. I choose the less cynical path and say this could be tremendously useful for remote user testing.
- arsenerei 10y agoRemote usability testing would be fantastic. This could be useful for local testing as well. Having a library or tool that could generically sit in front of a site to show where users look when taking action X sounds beneficial to me. I can also imagine on a lighter note this being used for games where you look to where you want to move a character (maybe blink twice for confirmation).
- polartx 10y agoI'm on a thunderbolt display (with integrated camera) sitting back about 3 feet. After 5 or so clicks against the corner regions, tracking became surprisingly accurate. I'm impressed!
- samfisher83 10y agoHow many people are going to be hesitant to allow people to use their webcams? I can't get it to work, but even if I could I would be very hesitant to use it.
- goatic 10y agopretty cool!
- Javadavinci 10y agoThis is interesting. I don't see many practical implications though.
- transpy 10y agoCould this be useful to monitor user sentiment and correlate it with further data from applications/media?
- jhanna 10y agoNice work, well done.. one of the best one's we've seen !!! I am one of the founders at xLabs and we too have been working on self calibrating real time interactive webcam eye tracking for a number of and have a number of demos and commercial products. We'd love your feedback.. check out http://xlabsgaze.com/ http://xlabsgaze.com/ (core tech) and our first commercial product https://eyesdecide.com https://eyesdecide.com (Design effectiveness and usabilty testing SaaS). Our just check out one of our video here https://www.youtube.com/watch?v=kSnAfJWAhtE https://www.youtube.com/watch?v=kSnAfJWAhtE Keep up the good work.
- octopitus 10y agoInteresting. Just curious, but does this library scalable for a facial recognition system? Eg: Authentication system.