4 ms·

Drowsiness detection with OpenCV

petters 9y ago
Seems like a good tutorial! The example video is shot during daytime, though.
- jcims 9y ago
  Good point. Night time with glasses and reflections could be a mess. Might actually be easier with a near IR camera and a couple small LEDs for illumination, as I believe the retina is much more reflective in that range.
  - gaius 9y ago
    A friend of mine built one of these 5 years ago with an iPhone and on that hardware the darkness problem was insurmountable.
ggus 9y ago
Is it enough to check if the eyes are closing for long enough? I fear that there is a "pre-sleep" status, where one is unable to drive but not with his eyes closed. When sleepy and driving I blink more, I feel loss of motor skill, and I may give off other hints well in advance of finally closing my eyes. When I finally close my eyes it is way too late. To put it bluntly: I fear that checking if the eyes closed for too long is as useful as checking if the airbag has deployed.
- rubyfan 9y ago
  Agreed, I usually try to keep my eyes open to fool myself that I'm not sleepy. My face is probably limp though. From what I've seen from recent neural nets I believe it's possible to detect the pre-sleep phase given a training set of faces prior to dozing. Seems like that would be a hard data set to come by though.
epaga 9y ago
I've wondered before whether something similar could be implemented to help foster focus & "flow", for those of us working at computers. By which I mean less measuring actual drowsiness, but more "lack of focus" as detected by your facial expression (perhaps also mouse movements + current open app?). This likely wouldn't be a binary thing but rather a more fluid "guide" which notices you're beginning to lose focus and guides you back to your "best next action". Obviously intrinsic motivation is far more beneficial than external, but still it seems that something like this could be helpful (to me at least).
yaccz 9y ago
The solution is based on a premise that drowsiness implies closed eyes and the alarm is sound after something like 3 to 4 seconds. First, a lot can happen in 3 seconds. In 100kph this gives you roughly 90 to 120 meters you can still sleep through. Second I doubt the premise based on personal experience. I was driving after an exhausting day. I started to fall asleep and drift into the opposite lane, however I had realized neither at the time but I still clearly had eyes on the road, just unable to react until I realized I am dangerously close to the other lane and woke. Its cool proof of concept but I'm afraid it still has a long way to go. Especially in rigorous research on the premises it can be based on. I guess its better than nothing but false sense of security can also be a problem.
angry_octet 9y ago
For those with curiosity as to practical use, there are commercial products for this, which initially focused on fatigue in truck drivers. https://www.seeingmachines.com/industry-applications/automotive/ https://www.seeingmachines.com/industry-applications/automot... https://www.youtube.com/watch?v=gvN1elQ8NLY https://www.youtube.com/watch?v=gvN1elQ8NLY These systems often use IR illuminators, as the contrast between pupil and iris is dramatic under IR. The IR can also be boosted to compensate for other light in the cabin. Spectacles are obviously impediments.[1] Eye tracking is also great for seeing how people use user interfaces, including VR and AR scenes.[2] Cheap custom systems are available too[3], some of which are open source[4]. 1- https://www.diva-portal.org/smash/get/diva2:306465/FULLTEXT01.pdf https://www.diva-portal.org/smash/get/diva2:306465/FULLTEXT0... 2- https://www.youtube.com/watch?v=rrj1F3rHJd0 https://www.youtube.com/watch?v=rrj1F3rHJd0 3- http://thume.ca/2016/03/24/eye-tracker-reviews-pupil-labs-tobii-eyex-eye-tribe-tobii-x2-30/ http://thume.ca/2016/03/24/eye-tracker-reviews-pupil-labs-to... 4- https://pupil-labs.com/pupil/ https://pupil-labs.com/pupil/
TYPE_FASTER 9y ago
I experimented with something similar using iOS. Let me know if you want the link to the Github repo. I realized quickly that lighting conditions have a huge impact on the success of eye detection. My dad suggested that reading the steering sensor is another good way to detect drowsiness. I've been meaning to take another look now that I have a wifi OBDII adapter that will work with iOS, but haven't had the time.
aadeshbagmar 9y ago
https://www.youtube.com/watch?v=NGCRkPrcQkQ https://www.youtube.com/watch?v=NGCRkPrcQkQ Almost 2 years ago! :)

Seems like a good tutorial! The example video is shot during daytime, though.

jcims 9y ago
Good point. Night time with glasses and reflections could be a mess. Might actually be easier with a near IR camera and a couple small LEDs for illumination, as I believe the retina is much more reflective in that range.
- gaius 9y ago
  A friend of mine built one of these 5 years ago with an iPhone and on that hardware the darkness problem was insurmountable.

ggus 9y ago

Is it enough to check if the eyes are closing for long enough? I fear that there is a "pre-sleep" status, where one is unable to drive but not with his eyes closed. When sleepy and driving I blink more, I feel loss of motor skill, and I may give off other hints well in advance of finally closing my eyes. When I finally close my eyes it is way too late. To put it bluntly: I fear that checking if the eyes closed for too long is as useful as checking if the airbag has deployed.

rubyfan 9y ago
Agreed, I usually try to keep my eyes open to fool myself that I'm not sleepy. My face is probably limp though. From what I've seen from recent neural nets I believe it's possible to detect the pre-sleep phase given a training set of faces prior to dozing. Seems like that would be a hard data set to come by though.

epaga 9y ago

I've wondered before whether something similar could be implemented to help foster focus & "flow", for those of us working at computers. By which I mean less measuring actual drowsiness, but more "lack of focus" as detected by your facial expression (perhaps also mouse movements + current open app?). This likely wouldn't be a binary thing but rather a more fluid "guide" which notices you're beginning to lose focus and guides you back to your "best next action". Obviously intrinsic motivation is far more beneficial than external, but still it seems that something like this could be helpful (to me at least).

yaccz 9y ago

The solution is based on a premise that drowsiness implies closed eyes and the alarm is sound after something like 3 to 4 seconds. First, a lot can happen in 3 seconds. In 100kph this gives you roughly 90 to 120 meters you can still sleep through. Second I doubt the premise based on personal experience. I was driving after an exhausting day. I started to fall asleep and drift into the opposite lane, however I had realized neither at the time but I still clearly had eyes on the road, just unable to react until I realized I am dangerously close to the other lane and woke. Its cool proof of concept but I'm afraid it still has a long way to go. Especially in rigorous research on the premises it can be based on. I guess its better than nothing but false sense of security can also be a problem.

angry_octet 9y ago

For those with curiosity as to practical use, there are commercial products for this, which initially focused on fatigue in truck drivers. https://www.seeingmachines.com/industry-applications/automotive/ https://www.seeingmachines.com/industry-applications/automot... https://www.youtube.com/watch?v=gvN1elQ8NLY https://www.youtube.com/watch?v=gvN1elQ8NLY These systems often use IR illuminators, as the contrast between pupil and iris is dramatic under IR. The IR can also be boosted to compensate for other light in the cabin. Spectacles are obviously impediments.[1] Eye tracking is also great for seeing how people use user interfaces, including VR and AR scenes.[2] Cheap custom systems are available too[3], some of which are open source[4]. 1- https://www.diva-portal.org/smash/get/diva2:306465/FULLTEXT01.pdf https://www.diva-portal.org/smash/get/diva2:306465/FULLTEXT0... 2- https://www.youtube.com/watch?v=rrj1F3rHJd0 https://www.youtube.com/watch?v=rrj1F3rHJd0 3- http://thume.ca/2016/03/24/eye-tracker-reviews-pupil-labs-tobii-eyex-eye-tribe-tobii-x2-30/ http://thume.ca/2016/03/24/eye-tracker-reviews-pupil-labs-to... 4- https://pupil-labs.com/pupil/ https://pupil-labs.com/pupil/

TYPE_FASTER 9y ago

I experimented with something similar using iOS. Let me know if you want the link to the Github repo. I realized quickly that lighting conditions have a huge impact on the success of eye detection. My dad suggested that reading the steering sensor is another good way to detect drowsiness. I've been meaning to take another look now that I have a wifi OBDII adapter that will work with iOS, but haven't had the time.

aadeshbagmar 9y ago

https://www.youtube.com/watch?v=NGCRkPrcQkQ https://www.youtube.com/watch?v=NGCRkPrcQkQ Almost 2 years ago! :)