5 ms·

Eyetracking Jetpack Joyride, Smash Hit and Dungeon Keeper

11 points by chewxy 13y ago

xyzzy123 13y ago
Using a tablet's built-in camera for gaze tracking is a pretty neat hack :) I wonder if you could make a game which incorporates looking (or not looking at things) in some way?
- chewxy 13y ago
  It's not going to be terribly accurate. I have accuracy of about 80-85%. It loses accuracy towards the edges, and towards the bottom of the screen
  - xyzzy123 13y ago
    Any idea much more accuracy might be achievable given hardware limitations?
    - chewxy 13y ago
      well, it would take a lot of tuning of the code. But even with a lot of careful tuning, there will be limitation in form of the framerate with JavaCameraView or even NativeCameraView. Better algorithms exist. I've tested them too, but they're too slow to work on the onboard hardware. I've worked out an algorithm where a portion of the processing is offloaded to a different machine. Assuming a fast enough internet connection, (or even local LAN), and assuming that the offloaded processing is done with say, graphics cards or something like that, the accuracy will be largely improved to about 90-ish %. It took me quite a loooong time to figure out how to profile code in Android, but the processing bottlenecks are these: finding the center of the eye, and calculating the probable location and shape of the iris. So these are the things that are to be offloaded to be processed. Naturally network latency would be then an issue, but I have some ideas on how to overcome those TL;DR - if it's just relying on onboard hardware limitation, more accurate algorithms cannot be used because it would slow to a crawl. EDIT: also, there is the issue with memory use - my app would consistently crash after about 1 min of use (I used this to my advantage) as it was somehow somewhere leaking memory.
      - xyzzy123 13y ago
        I see, thanks. So a game would probably be hard, since games require low latency. Something might be possible if one could find mechanics which don't require high accuracy...

11 points by chewxy 13y ago

Using a tablet's built-in camera for gaze tracking is a pretty neat hack :) I wonder if you could make a game which incorporates looking (or not looking at things) in some way?

chewxy 13y ago
It's not going to be terribly accurate. I have accuracy of about 80-85%. It loses accuracy towards the edges, and towards the bottom of the screen
- xyzzy123 13y ago
  Any idea much more accuracy might be achievable given hardware limitations?
  - chewxy 13y ago
    well, it would take a lot of tuning of the code. But even with a lot of careful tuning, there will be limitation in form of the framerate with JavaCameraView or even NativeCameraView. Better algorithms exist. I've tested them too, but they're too slow to work on the onboard hardware. I've worked out an algorithm where a portion of the processing is offloaded to a different machine. Assuming a fast enough internet connection, (or even local LAN), and assuming that the offloaded processing is done with say, graphics cards or something like that, the accuracy will be largely improved to about 90-ish %. It took me quite a loooong time to figure out how to profile code in Android, but the processing bottlenecks are these: finding the center of the eye, and calculating the probable location and shape of the iris. So these are the things that are to be offloaded to be processed. Naturally network latency would be then an issue, but I have some ideas on how to overcome those TL;DR - if it's just relying on onboard hardware limitation, more accurate algorithms cannot be used because it would slow to a crawl. EDIT: also, there is the issue with memory use - my app would consistently crash after about 1 min of use (I used this to my advantage) as it was somehow somewhere leaking memory.
    - xyzzy123 13y ago
      I see, thanks. So a game would probably be hard, since games require low latency. Something might be possible if one could find mechanics which don't require high accuracy...