4 ms·

Does anyone know how this can be technically achieved? Large scale face recognition with 1 photo per person in the training set?

crazygringo 13y ago
I don't think even people can do it from just a single photo in a training set.
Raphael 13y ago
Perhaps with the help of Facebook. It's right in the name.
frenger 13y ago
Well, we know they archive Facebook content so go figure
varelse 13y ago
It could certainly narrow the field. Combine a vector of "features" derived from the single image that with a bunch of other orthogonal "features" such as location and I could see it improving efficiency of manhunts significantly. Just beware of Bonferroni's Principle: http://math.stackexchange.com/questions/352689/bonferronis-principle http://math.stackexchange.com/questions/352689/bonferronis-p...
nathan_long 13y ago
Maybe not in realtime, but they could store all the images and maybe have software presort faces by similarity. Then if they became interested in a specific person, they could input more training images and sift through the set of similar ones. They might find hundreds of pictures of you with timestamps and locations. (This is all speculation; I haven't written image recognition software myself.)
IanCal 13y ago
Why does it need to be a single photo? Mandate that when you get your passport/driving license theres a face scan. Wouldn't be amazing tech and could be introduced fairly easily.
- wicknicks 13y ago
  I didn't realize that. Got my passport long ago, and for the driving license I'm sure they just took a picture. There was no scan (atleast nothing like the laser scan described by bsenftner above).
bsenftner 13y ago
Yes, it exists, and I use it every day. In fact, I have a partnership with the DOD/NSA subcontractors who created it, and I have applied the technology to the automated creation of 3D Avatars given a single photo of a person's face. My version of the tech is available at www.3D-Avatar-Store.com. The system works as follows: a person's head is laser scanned, at the same time dozens of single photos are taken of that person from different angles, different lighting conditions, and different quality cameras. Then a neural net is trained to associate each photo with the laser scan data. After a few thousand trainings, anyone's single photo would generate a reasonable likeness of them in 3D. Today, after over a decade of training and 10's of thousands of scans added to the training database, we get remarkably good quality 3D reconstructions given a single photo of anyone, any age, any ethnicity. There are limitations, such as the face needs to be visible, and it needs to be within +/-30 degrees of facing the camera for the 3D reconstruction to be suitable for facial recognition. And that is the point of the original application of this technology: a facial recognition pre-processor. It corrects the face angle and removes any facial expression, creating a likeness of the face in a passport style photo perfect for facial recognition. But, of course, I'm not using it for that. I use it to auto-magically create 3D avatars for video game studios. Oh, it's really, really fast. My current system is based upon two generations back of neural nets, and they reconstruct from one photo in 0.9 seconds. The latest generation does 144 reconstructions simultaneously given HD video feeds.
- anonymousDan 13y ago
  What's the state of the art in terms of extending such systems to handle faces that are partially obscured (e.g. wearing glasses)? References to further resources in the area accepted and appreciated! I'm going to start work on a face recognition system soon and I'm keen to learn more about their current limitations.
  - bsenftner 13y ago
    It's just an issue of training. I'd expect a similar setup as I describe could be used to associate a series of faces with and without different things attached to their face, and a neural net could be trained to generate, for example, a no eye glasses wearing image from one with eye glasses. We already do that with facial expressions. I think the issues to be improved upon are extending the recognition beyond the face - include ear characteristics to extend recognition beyond +/- 30 degrees, and although my lab tells me analyzing gait is a dead end, I feel there might be something there that can help. Also lighting neutralization is an area that needs some help, which includes illumination balancing without removing important details, shadow removal, and colored illumination white balancing.
therobot24 13y ago
I do it with only a single ocular image (eye, eyebrow, and surrounding skin texture) rather than the full face. Using 1:1 matching environments are common for NIST challenges.
- wicknicks 13y ago
  Do you have any papers on your work? Would be great if you add a link to them.
  - therobot24 13y ago
    http://www.cs.cmu.edu/~vboddeti/papers/icb_2012.pdf http://www.cs.cmu.edu/~vboddeti/papers/icb_2012.pdf http://www.cs.cmu.edu/~vboddeti/papers/ijcb_2011.pdf http://www.cs.cmu.edu/~vboddeti/papers/ijcb_2011.pdf I have two more in the pipe (one still awaiting acceptance, and the other is accepted but yet to be added to IEEEXplore)

crazygringo 13y ago

I don't think even people can do it from just a single photo in a training set.

Raphael 13y ago

Perhaps with the help of Facebook. It's right in the name.

frenger 13y ago

Well, we know they archive Facebook content so go figure

varelse 13y ago

It could certainly narrow the field. Combine a vector of "features" derived from the single image that with a bunch of other orthogonal "features" such as location and I could see it improving efficiency of manhunts significantly. Just beware of Bonferroni's Principle: http://math.stackexchange.com/questions/352689/bonferronis-principle http://math.stackexchange.com/questions/352689/bonferronis-p...

nathan_long 13y ago

Maybe not in realtime, but they could store all the images and maybe have software presort faces by similarity. Then if they became interested in a specific person, they could input more training images and sift through the set of similar ones. They might find hundreds of pictures of you with timestamps and locations. (This is all speculation; I haven't written image recognition software myself.)

IanCal 13y ago

Why does it need to be a single photo? Mandate that when you get your passport/driving license theres a face scan. Wouldn't be amazing tech and could be introduced fairly easily.

wicknicks 13y ago
I didn't realize that. Got my passport long ago, and for the driving license I'm sure they just took a picture. There was no scan (atleast nothing like the laser scan described by bsenftner above).

bsenftner 13y ago

Yes, it exists, and I use it every day. In fact, I have a partnership with the DOD/NSA subcontractors who created it, and I have applied the technology to the automated creation of 3D Avatars given a single photo of a person's face. My version of the tech is available at www.3D-Avatar-Store.com. The system works as follows: a person's head is laser scanned, at the same time dozens of single photos are taken of that person from different angles, different lighting conditions, and different quality cameras. Then a neural net is trained to associate each photo with the laser scan data. After a few thousand trainings, anyone's single photo would generate a reasonable likeness of them in 3D. Today, after over a decade of training and 10's of thousands of scans added to the training database, we get remarkably good quality 3D reconstructions given a single photo of anyone, any age, any ethnicity. There are limitations, such as the face needs to be visible, and it needs to be within +/-30 degrees of facing the camera for the 3D reconstruction to be suitable for facial recognition. And that is the point of the original application of this technology: a facial recognition pre-processor. It corrects the face angle and removes any facial expression, creating a likeness of the face in a passport style photo perfect for facial recognition. But, of course, I'm not using it for that. I use it to auto-magically create 3D avatars for video game studios. Oh, it's really, really fast. My current system is based upon two generations back of neural nets, and they reconstruct from one photo in 0.9 seconds. The latest generation does 144 reconstructions simultaneously given HD video feeds.

anonymousDan 13y ago
What's the state of the art in terms of extending such systems to handle faces that are partially obscured (e.g. wearing glasses)? References to further resources in the area accepted and appreciated! I'm going to start work on a face recognition system soon and I'm keen to learn more about their current limitations.
- bsenftner 13y ago
  It's just an issue of training. I'd expect a similar setup as I describe could be used to associate a series of faces with and without different things attached to their face, and a neural net could be trained to generate, for example, a no eye glasses wearing image from one with eye glasses. We already do that with facial expressions. I think the issues to be improved upon are extending the recognition beyond the face - include ear characteristics to extend recognition beyond +/- 30 degrees, and although my lab tells me analyzing gait is a dead end, I feel there might be something there that can help. Also lighting neutralization is an area that needs some help, which includes illumination balancing without removing important details, shadow removal, and colored illumination white balancing.

therobot24 13y ago

I do it with only a single ocular image (eye, eyebrow, and surrounding skin texture) rather than the full face. Using 1:1 matching environments are common for NIST challenges.

wicknicks 13y ago
Do you have any papers on your work? Would be great if you add a link to them.
- therobot24 13y ago
  http://www.cs.cmu.edu/~vboddeti/papers/icb_2012.pdf http://www.cs.cmu.edu/~vboddeti/papers/icb_2012.pdf http://www.cs.cmu.edu/~vboddeti/papers/ijcb_2011.pdf http://www.cs.cmu.edu/~vboddeti/papers/ijcb_2011.pdf I have two more in the pipe (one still awaiting acceptance, and the other is accepted but yet to be added to IEEEXplore)