3 ms·
> That eye tracking though also supports dystopian scenarios such as being able to change the world right from under you by only changing things you're not look
by jsheard 3y ago
> That eye tracking though also supports dystopian scenarios such as being able to change the world right from under you by only changing things you're not looking at, "reading your mind" by seeing what catches your eye, etc.
For better or worse Apple isn't allowing this kind of thing, the eye tracking "cursor" is only ever known to the OS itself and apps only receive "click" events with a snapshot of where you're looking when the OS detects the relevent hand gesture. Apps are never allowed to know what you're gazing at passively.
It's a good move for privacy, but it's very limiting for games since they will only have (accurate) head and (not so accurate) hand tracking data to work with.
- sebzim4500 3y agoI suspect there will be ways of getting this information though, it would be incredibly hard to design the foveated renderer such that you can't figure out where the eye is by e.g. positioning different amounts of geometry on the screen and then timing frames.
- jsheard 3y agoApple hasn't made it exactly clear AFAICT, but I suspect that foveated rendering might only work in the managed RealityKit environment where Apple controls the entire rendering pipeline, and not inside applications which implement their own rendering from the ground up using Metal, for exactly that reason. There's nothing in the documentation about foveated rendering in Metal apps, and it is something that engines would have to be explicitly aware of if they do any kind of off-screen rendering.
- dwaite 3y agoI don't believe you get mouseover/"glanceover" events; you have to define your focus visual behavior declaratively. Rendering of that behavior is then not visible to an app with default entitlements. Of course, Apple's store means they can just forbid gaming the system.