7 ms·
A Face Recognition Algorithm That Finally Outperforms Humans
- chriskanan 12y agoThis isn't really face recognition, this is face verification. In computer vision, face recognition usually means tell me who this person is (in psychology it means "have you seen this person before?"). Face verification gives an algorithm (or a person) two images of faces and asks, "Are these the same person?" Their result is impressive, and it improves a bit over Facebook's recent result on the same dataset with their DeepFace system (97.35% for DeepFace vs 97.53% for people vs 98.52% for the system discussed in the article). Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model.
- sp332 12y agoI think the breakthrough with Facebook's system was the idea to transform the face before running the training or verification on it. And the new system seems to use the same idea: The new algorithm works by normalising each face into a 150 x 120 pixel image, by transforming it based on five image landmarks: the position of both eyes, the nose and the two corners of the mouth.
- apu 12y agoNo. Alignment is done by every face verification method, and even the "new" 3d alignment done here is on par with other alignment methods in practice today. At their talk at CVPR, they made clear that the biggest factor in getting their numbers was having tons of training data and a high-capacity learning model. The paper even shows that if you replace their 3d alignment with a standard 2d affine alignment (something people were doing 15 years ago), you lose only 1.5% on LFW (table 2).
- pessimizer 12y ago>Face verification gives an algorithm (or a person) two images of faces and asks, "Are these the same person?" >97.53% for people I'm surprised that people are that good - I think I'd be horrible at it. From googling, I can see that number came from Facebook - would you happen to know where to find the paper? >Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model. Does using this different model gain some sort of space or processing efficiency over DeepFace? Also: does anyone know the point of fighting to close this 3% gap? That seems good enough for any purpose. (Is there any other purpose to this technology other than mass surveillance?)
- apu 12y agoHere's the paper: https://www.facebook.com/publications/546316888800776/ https://www.facebook.com/publications/546316888800776/ The methods are quite different, but I think at root, it's just more being able to take advantage of much more training data. The 97.53% "human performance" number comes from my paper from a few years ago [0]. We initially thought that there's no way we would be able to do that well on this task (manually) either, but then we tried it and our scores were much higher (very close to 100%). So the verification task on LFW is actually not that hard, although in part it's because the dataset consists of public figures and therefore you can recognize many of the people outright if you've seen them before. (We tried to measure this in our human studies but found it to be inconclusive.) 3% is not the right way to look at error rates, btw. See [1] for why. But also, face verification is more of a low-level building block for recognition, because most real-world applications don't directly care about saying whether two faces are the same person or not ("verification"), but "who is this person?" ("recognition"). A simple way to build recognition from verification is to compare a test face against all faces in the database, and then take the one that gets ranked highest. In that (simplistic) scenario, your recognition rate scales (very roughly) as acc^lg(N), where N is the number of people in your database and acc is the verification accuracy. So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you might have a recognition accuracy of ~66% (very roughly!), which is not great. Finally, there are already lots of applications to face recognition other than surveillance, and many more waiting to be discovered. [0] http://homes.cs.washington.edu/~neeraj/projects/faceverification/ http://homes.cs.washington.edu/~neeraj/projects/faceverifica... [1] https://news.ycombinator.com/item?id=7393438 https://news.ycombinator.com/item?id=7393438
- dheera 12y agoI'm pretty sure in reality Facebook also uses your social network graph to restrict the candidate set and get higher recognition accuracy. This makes it hard to compare Facebook's results to a pure recognition algorithm. On that note, Facebook could do even better by restricting the candidate set based on time, precise location, and compass orientation, given that most mobile users have Facebook installed and are running it in their pockets when they get their picture taken by others. (They could do rough recognition purely based on position and orientation without even looking at the camera image, if they really wanted to, so with the camera image it could really be near 100% accurate, and even work if you take a picture of a friend's back.)
- apu 12y agoSure, but the numbers in the research paper are "pure" (i.e., not using all this additional information). In production, I'm sure they must be using all these additional cues as well.
- dheera 12y agoAh, I see, you mean the 97.35% is a "pure" algorithmic result. That's pretty impressive.
- dkhar 12y agoDeepFace's figures[1] also come from running it on the LFW dataset[2]. While Facebook's production tech will be more accurate because of what you say, the Deepface algorithm has a lot of raw power, too! [1] http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr... [2] http://vis-www.cs.umass.edu/lfw/index.html http://vis-www.cs.umass.edu/lfw/index.html
- hcarvalhoalves 12y ago> Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model. It's my understanding that deep learning is more interesting when you have a large dataset and need to generalize (e.g., recognize faces sideways, upside-down, etc), otherwise, when the requirements are more predictable, the hand-tuned algorithms tend to perform better. The task of finding whether two pictures are from the same person doesn't seem appropriate for DL (compared to e.g. find whether a picture contains a person). Would that be correct?
- apu 12y agoI'm not sure that's the distinction I'd use. To me, it seems more to be a matter of the kind and amount of training data you have. If you have a ton of labeled data (either for the exact task you're doing, or a related one), DL often wins because it's sufficiently high-capacity to learn sophisticated models from all that data. On the other hand, if you don't have enough data, then the deep network isn't going to be learnable, in which case hand-tuned methods might do better, depending on how much "domain expertise" you can bake into the system. Faces are a very constrained domain, and since there has been enormous work on them for decades now, it's one area that non-DL algorithms probably still have somewhat of a fighting chance (until people gather & label large enough databases, of course).
- hcarvalhoalves 12y agoThat makes more sense, thank you.
- jastanton 12y agoeven with Facebook's 97.35% they still confuse me with my toddlers :P
- PeterisP 12y agoActually, for humans this actually is a face recognition task - from the given examples, it seems that the dataset involves pictures of publicly known people, so it's comparing apples and oranges as humans are given an entirely different task. Computers are given 2 pictures and asked "Are these the same person?" However, if people are given the same 2 pictures, and one of them is a well-known actor; then the question is "Is the second picture the same person as the many pictures of this actor that I have seen during my life?" which is a rather different task. If you want a fair comparison, then either you have to use portraits of people that nobody knows, or the computer systems need to use databases of celebrity pictures for that.
- apu 12y agosigh no, it doesn't "outperform humans" on "face recognition". In particular, see my previous comments [1] and [2] for discussion on why this method might be doing well. As for "outperforming humans", a more accurate statement might be, "this algorithm outperforms (for this simplistic task) one experiment done with a limited set of humans on this one particular dataset which has been in the community for 10 years now and is thus highly gameable." But I realize that's a lot less pithy. In particular, this dataset is nearing saturation, and whenever that happens, differences in the accuracy numbers often don't mean much. So for example with Facebook's number at 97.53% and this paper's at 98.52%, you're talking about the difference between getting 148 pairs of faces wrong vs 89 pairs wrong. In practical terms, as a researcher working with a dataset like this, you very quickly learn to focus on just the ones your algorithm gets wrong, and it's impossible to not subconsciously try to optimize for getting those few cases correct, even if those techniques wouldn't actually help in the general case. [1] https://news.ycombinator.com/item?id=7637866 https://news.ycombinator.com/item?id=7637866 [2] https://news.ycombinator.com/item?id=7638269 https://news.ycombinator.com/item?id=7638269
- darkmighty 12y agohttps://class.coursera.org/ml-005/lecture/39 https://class.coursera.org/ml-005/lecture/39
- galapago 12y agohttps://www.youtube.com/watch?v=DQWI1kvmwRg https://www.youtube.com/watch?v=DQWI1kvmwRg
- yamaneko 12y agoI just love all the analysis you've done about these face recognition systems. They're so insightful!
- yummybear 12y agoYou seem to know something about this - how accurate are the best face detection algorithms on real-world datasets?
- gizmodo59 12y agoI had to say this, the articles from medium.com has more focus on PR than the quality of content. Today itself I saw two articles, Schrodinger's cat and this.
- shreyassaxena 12y agoNo it is wrong, the actual human performance on full images is close to 99%. [1] Ctrl+F human The performance of human it reports is on cropped images shown to human, which is not a fair comparison. [1] http://vis-www.cs.umass.edu/lfw/results.html http://vis-www.cs.umass.edu/lfw/results.html
- apu 12y agoNo, the 97.53 is more fair. See here: https://news.ycombinator.com/item?id=7638269 https://news.ycombinator.com/item?id=7638269
- drpgq 12y agoNIST's latest FRVT results for commercial face recognition (FRVT 2013) is available here: http://www.nist.gov/itl/iad/ig/frvt-2013.cfm http://www.nist.gov/itl/iad/ig/frvt-2013.cfm
- fenollp 12y ago> But when the algorithm is faced with images that are entirely different from the training set, it often fails. Over training it is.