4 ms·

I'll use this post / opportunity to ask the HN community (I understand that this is a technically complicated, and potentially sensitive subject) I've been ap

I'll use this post / opportunity to ask the HN community (I understand that this is a technically complicated, and potentially sensitive subject) I've been approached by a few people who are doing what they can to lower the cost of prescription glasses. It's a mix of (licensed) ophthalmologists, people interested in offering community health services, etc. What is the current state of the art for pupillary distance (PD) measurement based on face detection / pupil tracking using laptop webcams or mobile (front) cameras? I imagine (obviously?) that there must be scholar research on the acceptable error margins for a PD measurement (depending on the type of vision condition, i.e. farsightedness, etc.)? Would using something like Handsfree.js or https://github.com/esimov/pigo https://github.com/esimov/pigo (which has pupil detection) be a good start, or would these be an ~order of magnitude off in terms of the necessary margins? Thanks a lot.

notum 5y ago
It comes down to physics. Cameras and their lenses come in all shapes and sizes, most of webcams don't expose their geometry in a programmatically accessible way, even if they did, most of them don's have means of determining distance to the subject. We can detect faces, eyes and pupils, but with no reference points we have no idea how big or far apart they are.
- gregsadetsky 5y ago
  Thanks! What about sites (such as Warby Parker) that make you hold up a credit card (which has a standard size) and use that to determine (via what I assume is a rule of three) the PD? [0] I can see of course problems with the card not being “perpendicular” and other 3d distortions. [0] - https://www.warbyparker.com/pd/instructions https://www.warbyparker.com/pd/instructions
mncharity 5y ago
Fwiw, there's https://www.eyeque.com/pdcheck/ https://www.eyeque.com/pdcheck/ . And Google's MediaPipe Iris used iris diameter for distance estimation.[1] EyeQue is perhaps a cautionary tale. "Change the world" "few dollars of optics + phone = D+CYL" become $45 one-person-per-subscription "vision tracker". [1] https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-iris-tracking.html https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-i...
fny 5y ago
There are a lot of problems with measuring pd accurately with a photo. Ideally you need to ensure the pupils are aligned (i.e. the head can't be turned) which is really hard to tell from a photo (there's no depth). The credit card trick somewhat works since you can guage depth given the size and positioning of the card, but the errors still can end up in the 2-3mm range fairly easily. The best mechanism would be to create something like a Google cardboard to guarantee the head is properly positioned and then perform the measurements.
epmaybe 5y ago
This probably isn't very helpful, but I'd imagine underestimating the PD is going to lead to happier patients even if not all that accurate. We can accommodate with crossing our eyes, but the opposite is not usually true.
GoingHandsfree 5y ago
Hello! Handsfree.js doesn't yet have pupillary distance, but I do plan to implement MediaPipe Iris very soon which does: https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-iris-tracking.html https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-i...
joshuamcginnis 5y ago
If my memory serves me correctly, I once a used a digital try-on app from an online glasses retailer and they had me hold a common item with a known width (something like a credit card or drivers license) in front of the cam within a bounding box. It could guess the location of my pupils but allowed me to drag the pupil location markers to the exact centers. Calculating pupillary distance was then just a matter of using the distance in pixels between the pupils and converting them to mm using the ratio of pixels to mm of the card in the previous photo.

notum 5y ago

It comes down to physics. Cameras and their lenses come in all shapes and sizes, most of webcams don't expose their geometry in a programmatically accessible way, even if they did, most of them don's have means of determining distance to the subject. We can detect faces, eyes and pupils, but with no reference points we have no idea how big or far apart they are.

gregsadetsky 5y ago
Thanks! What about sites (such as Warby Parker) that make you hold up a credit card (which has a standard size) and use that to determine (via what I assume is a rule of three) the PD? [0] I can see of course problems with the card not being “perpendicular” and other 3d distortions. [0] - https://www.warbyparker.com/pd/instructions https://www.warbyparker.com/pd/instructions

mncharity 5y ago

Fwiw, there's https://www.eyeque.com/pdcheck/ https://www.eyeque.com/pdcheck/ . And Google's MediaPipe Iris used iris diameter for distance estimation.[1] EyeQue is perhaps a cautionary tale. "Change the world" "few dollars of optics + phone = D+CYL" become $45 one-person-per-subscription "vision tracker". [1] https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-iris-tracking.html https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-i...

fny 5y ago

There are a lot of problems with measuring pd accurately with a photo. Ideally you need to ensure the pupils are aligned (i.e. the head can't be turned) which is really hard to tell from a photo (there's no depth). The credit card trick somewhat works since you can guage depth given the size and positioning of the card, but the errors still can end up in the 2-3mm range fairly easily. The best mechanism would be to create something like a Google cardboard to guarantee the head is properly positioned and then perform the measurements.

epmaybe 5y ago

This probably isn't very helpful, but I'd imagine underestimating the PD is going to lead to happier patients even if not all that accurate. We can accommodate with crossing our eyes, but the opposite is not usually true.

GoingHandsfree 5y ago

Hello! Handsfree.js doesn't yet have pupillary distance, but I do plan to implement MediaPipe Iris very soon which does: https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-iris-tracking.html https://ai.googleblog.com/2020/08/mediapipe-iris-real-time-i...

joshuamcginnis 5y ago

If my memory serves me correctly, I once a used a digital try-on app from an online glasses retailer and they had me hold a common item with a known width (something like a credit card or drivers license) in front of the cam within a bounding box. It could guess the location of my pupils but allowed me to drag the pupil location markers to the exact centers. Calculating pupillary distance was then just a matter of using the distance in pixels between the pupils and converting them to mm using the ratio of pixels to mm of the card in the previous photo.