4 ms·

Here's the paper: https://www.facebook.com/publications/546316888800776/ https://www.facebook.com/publications/546316888800776/ The methods are quite different

by apu 12y ago

Here's the paper: https://www.facebook.com/publications/546316888800776/ https://www.facebook.com/publications/546316888800776/ The methods are quite different, but I think at root, it's just more being able to take advantage of much more training data. The 97.53% "human performance" number comes from my paper from a few years ago [0]. We initially thought that there's no way we would be able to do that well on this task (manually) either, but then we tried it and our scores were much higher (very close to 100%). So the verification task on LFW is actually not that hard, although in part it's because the dataset consists of public figures and therefore you can recognize many of the people outright if you've seen them before. (We tried to measure this in our human studies but found it to be inconclusive.) 3% is not the right way to look at error rates, btw. See [1] for why. But also, face verification is more of a low-level building block for recognition, because most real-world applications don't directly care about saying whether two faces are the same person or not ("verification"), but "who is this person?" ("recognition"). A simple way to build recognition from verification is to compare a test face against all faces in the database, and then take the one that gets ranked highest. In that (simplistic) scenario, your recognition rate scales (very roughly) as acc^lg(N), where N is the number of people in your database and acc is the verification accuracy. So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you might have a recognition accuracy of ~66% (very roughly!), which is not great. Finally, there are already lots of applications to face recognition other than surveillance, and many more waiting to be discovered. [0] http://homes.cs.washington.edu/~neeraj/projects/faceverification/ http://homes.cs.washington.edu/~neeraj/projects/faceverifica... [1] https://news.ycombinator.com/item?id=7393438 https://news.ycombinator.com/item?id=7393438

pessimizer 12y ago
Found it about 10 secs ahead of your post - was coming back to edit it into my comment:) Thanks. >So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you'd have a recognition accuracy of ~66%, which is not great. Out of 100,000 people, I'd assume that there'd be many pairs that wouldn't be distinguishable by their parents, or even by each other. Wouldn't you eventually run into a quantum effect where the differences between people's faces from image-to-image would be larger than the difference between someone's face and everyone in a corpus of X million people? I'd think that a 66% would crush if you had 5 (for example) images of each person you were trying to identify, especially if they were intentionally taken from very different angles or in very different lighting, and you were trying to identify a person that you had 3 images of. >Finally, there are already lots of applications to face recognition other than surveillance I'm curious about those.
- apu 12y ago
  People are really good at face recognition in real-life, because you have not just a static 2-d view of a person completely out-of-context, but a fully dynamic 3-d view of someone you've probably seen before and you know the lighting environment you're in, meaning you can easily factor out those effects. A computer algorithm operating on single 2d images (like LFW) has none of that. If you were to ask people to do the same task with the same data, they'd probably still do pretty well (much better than computers), but perhaps not perfectly. The differences between image-to-image (same person) and different people is exactly what makes this problem so tough in the general case. It was shown about 20 years ago now that faces span a fairly low-dimensional manifold, and across different parts of this manifold, faces of different people do look much more similar than faces of the same person. I probably shouldn't have explicitly written that formula for recognition, since it's not actually the formula, but more a general scaling rule-of-thumb. But even if we take it as given (hypothetically), there would be several issues. First, remember that 97.53% is on this particular dataset, which is very special in many ways (e.g., it was all collected over the course of a year from photos on Yahoo News, of public figures, with relatively lower-resolution images, and often very distinctive backgrounds). On a more realistic dataset, these numbers would be much lower. Second, having more images of a person does help, but not nearly as much as you'd hope, because now there's an even bigger chance that you might accidentally match against someone else who happened to have their photo taken in the same pose, lighting conditions, and facial expression as your test photo, and it's very tough for algorithms to discount those confounding factors. As for other applications of face recognition, let me describe one broad area that I think is pretty exciting, rather than a bunch of specific instances. One of the big shifts in user interface design is going to be "personalization" (to varying degrees). If a program or device or robot can recognize who you are, it can pro-actively change its settings/behavior/performance/etc. to better suit you. A very simple example is the new Kinect, which does some sort of recognition to load your saved profile/controller preferences. But that's just the tip of the iceberg. And then of course there's personal photo collection management. If your photo organizer knew who everyone was in all your photos, it would be immensely useful in a number of ways. For starters, you could easily search for "photos of me & and my sister" or "my family", etc. You could also do more advanced analysis to start to get at more subtle things. For example, "my high school debate trip" might not immediately seem like it's related to face recognition, but in fact recognition might get you 90% of the way there. Finally, if you've made it this far, I might as well plug my recent paper, "Photo Recall" [1], which looks at how to do advanced searches on your photo collection. Our system doesn't currently handle faces, but if you look at the kinds of queries we can do, it should become clear how they might extend with faces. [1] http://homes.cs.washington.edu/~neeraj/projects/photo-recall/ http://homes.cs.washington.edu/~neeraj/projects/photo-recall...

by apu 12y ago

pessimizer 12y ago

Found it about 10 secs ahead of your post - was coming back to edit it into my comment:) Thanks. >So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you'd have a recognition accuracy of ~66%, which is not great. Out of 100,000 people, I'd assume that there'd be many pairs that wouldn't be distinguishable by their parents, or even by each other. Wouldn't you eventually run into a quantum effect where the differences between people's faces from image-to-image would be larger than the difference between someone's face and everyone in a corpus of X million people? I'd think that a 66% would crush if you had 5 (for example) images of each person you were trying to identify, especially if they were intentionally taken from very different angles or in very different lighting, and you were trying to identify a person that you had 3 images of. >Finally, there are already lots of applications to face recognition other than surveillance I'm curious about those.

apu 12y ago
People are really good at face recognition in real-life, because you have not just a static 2-d view of a person completely out-of-context, but a fully dynamic 3-d view of someone you've probably seen before and you know the lighting environment you're in, meaning you can easily factor out those effects. A computer algorithm operating on single 2d images (like LFW) has none of that. If you were to ask people to do the same task with the same data, they'd probably still do pretty well (much better than computers), but perhaps not perfectly. The differences between image-to-image (same person) and different people is exactly what makes this problem so tough in the general case. It was shown about 20 years ago now that faces span a fairly low-dimensional manifold, and across different parts of this manifold, faces of different people do look much more similar than faces of the same person. I probably shouldn't have explicitly written that formula for recognition, since it's not actually the formula, but more a general scaling rule-of-thumb. But even if we take it as given (hypothetically), there would be several issues. First, remember that 97.53% is on this particular dataset, which is very special in many ways (e.g., it was all collected over the course of a year from photos on Yahoo News, of public figures, with relatively lower-resolution images, and often very distinctive backgrounds). On a more realistic dataset, these numbers would be much lower. Second, having more images of a person does help, but not nearly as much as you'd hope, because now there's an even bigger chance that you might accidentally match against someone else who happened to have their photo taken in the same pose, lighting conditions, and facial expression as your test photo, and it's very tough for algorithms to discount those confounding factors. As for other applications of face recognition, let me describe one broad area that I think is pretty exciting, rather than a bunch of specific instances. One of the big shifts in user interface design is going to be "personalization" (to varying degrees). If a program or device or robot can recognize who you are, it can pro-actively change its settings/behavior/performance/etc. to better suit you. A very simple example is the new Kinect, which does some sort of recognition to load your saved profile/controller preferences. But that's just the tip of the iceberg. And then of course there's personal photo collection management. If your photo organizer knew who everyone was in all your photos, it would be immensely useful in a number of ways. For starters, you could easily search for "photos of me & and my sister" or "my family", etc. You could also do more advanced analysis to start to get at more subtle things. For example, "my high school debate trip" might not immediately seem like it's related to face recognition, but in fact recognition might get you 90% of the way there. Finally, if you've made it this far, I might as well plug my recent paper, "Photo Recall" [1], which looks at how to do advanced searches on your photo collection. Our system doesn't currently handle faces, but if you look at the kinds of queries we can do, it should become clear how they might extend with faces. [1] http://homes.cs.washington.edu/~neeraj/projects/photo-recall/ http://homes.cs.washington.edu/~neeraj/projects/photo-recall...