> But it does put an enormous amount of pressure on the eye tracking. As far as I can tell so far, the role of precise 2D control has been shifted to the eyes.

> But it does put an enormous amount of pressure on the eye tracking. As far as I can tell so far, the role of precise 2D control has been shifted to the eyes. I've been researching eye tracking for my own project for the past year. I have a Tobii eye tracker which is probably the best eye tracking device for consumers currently (or the only one really). It's much more accurate than trying to repurpose a webcam. The problem with eye tracking in general is what's called the "midas touch" problem. Everything you look at is potentially a target. If you were to simply connect your mouse pointer to your gaze, for example, any sort of hover effect on a web page would be activated simply by glancing at it. [1] Additionally, our eyes are constantly making small movements call saccades [2]. If you track eye movement perfectly, the target will wobble all over the screen like mad. The ways to alleviate this are by expanding the target visually so that the small movements are contained within a "bubble" or by delaying the targeting slightly so the movements can be smoothed out. But this naturally causes inaccuracy and latency. [3] Even then, you can easily get a headache from the effort of trying to fixate your eyes on a small target (trust me). Though Apple is making an effort to predict eye movements to give the user the impression of lower latency and improve accuracy, it's an imperfect solution. Simply put, gazing as an interface will always suffer from latency and unnatural physical effort. Until computers can read our minds, that isn't going to change. Apple decided to incorporate desktop and mobile apps into the device, so it seems this was really their only choice, as they need the equivalent of a pointer or finger to activate on-screen elements. They could do this with hand tracking, but then there's the issue of accuracy as well as clicking, tapping, dragging or swiping - plus the effort of holding your arms up for extended periods. I think it's odd that they decided that voice should not be part of the UI. My preference would be hand tracking a virtual mouse/trackpad (smaller and more familiar movements) plus a simple, "tap" or "swipe" spoken aloud, with the current system for "quiet" operation. But Apple is Apple, and they insist on one way to do things. They have a video of using Safari and it doesn't look particularly efficient/practical to me [4]. But who knows - I haven't tried it yet, maybe Apple's engineers nailed it. I have my doubts. 1. https://uxdesign.cc/the-midas-touch-effect-the-most-unknown-phenomenon-in-ux-design-36827204edd https://uxdesign.cc/the-midas-touch-effect-the-most-unknown-... 2. https://en.m.wikipedia.org/wiki/Saccade https://en.m.wikipedia.org/wiki/Saccade 3. https://help.tobii.com/hc/en-us/articles/210245345-How-to-set-up-Tobii-Ghost https://help.tobii.com/hc/en-us/articles/210245345-How-to-se... 4. https://developer.apple.com/videos/play/wwdc2023/10279/ https://developer.apple.com/videos/play/wwdc2023/10279/