4 ms·

This isn't really face recognition, this is face verification. In computer vision, face recognition usually means tell me who this person is (in psychology it m

This isn't really face recognition, this is face verification. In computer vision, face recognition usually means tell me who this person is (in psychology it means "have you seen this person before?"). Face verification gives an algorithm (or a person) two images of faces and asks, "Are these the same person?" Their result is impressive, and it improves a bit over Facebook's recent result on the same dataset with their DeepFace system (97.35% for DeepFace vs 97.53% for people vs 98.52% for the system discussed in the article). Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model.

sp332 12y ago
I think the breakthrough with Facebook's system was the idea to transform the face before running the training or verification on it. And the new system seems to use the same idea: The new algorithm works by normalising each face into a 150 x 120 pixel image, by transforming it based on five image landmarks: the position of both eyes, the nose and the two corners of the mouth.
- apu 12y ago
  No. Alignment is done by every face verification method, and even the "new" 3d alignment done here is on par with other alignment methods in practice today. At their talk at CVPR, they made clear that the biggest factor in getting their numbers was having tons of training data and a high-capacity learning model. The paper even shows that if you replace their 3d alignment with a standard 2d affine alignment (something people were doing 15 years ago), you lose only 1.5% on LFW (table 2).
pessimizer 12y ago
>Face verification gives an algorithm (or a person) two images of faces and asks, "Are these the same person?" >97.53% for people I'm surprised that people are that good - I think I'd be horrible at it. From googling, I can see that number came from Facebook - would you happen to know where to find the paper? >Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model. Does using this different model gain some sort of space or processing efficiency over DeepFace? Also: does anyone know the point of fighting to close this 3% gap? That seems good enough for any purpose. (Is there any other purpose to this technology other than mass surveillance?)
- apu 12y ago
  Here's the paper: https://www.facebook.com/publications/546316888800776/ https://www.facebook.com/publications/546316888800776/ The methods are quite different, but I think at root, it's just more being able to take advantage of much more training data. The 97.53% "human performance" number comes from my paper from a few years ago [0]. We initially thought that there's no way we would be able to do that well on this task (manually) either, but then we tried it and our scores were much higher (very close to 100%). So the verification task on LFW is actually not that hard, although in part it's because the dataset consists of public figures and therefore you can recognize many of the people outright if you've seen them before. (We tried to measure this in our human studies but found it to be inconclusive.) 3% is not the right way to look at error rates, btw. See [1] for why. But also, face verification is more of a low-level building block for recognition, because most real-world applications don't directly care about saying whether two faces are the same person or not ("verification"), but "who is this person?" ("recognition"). A simple way to build recognition from verification is to compare a test face against all faces in the database, and then take the one that gets ranked highest. In that (simplistic) scenario, your recognition rate scales (very roughly) as acc^lg(N), where N is the number of people in your database and acc is the verification accuracy. So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you might have a recognition accuracy of ~66% (very roughly!), which is not great. Finally, there are already lots of applications to face recognition other than surveillance, and many more waiting to be discovered. [0] http://homes.cs.washington.edu/~neeraj/projects/faceverification/ http://homes.cs.washington.edu/~neeraj/projects/faceverifica... [1] https://news.ycombinator.com/item?id=7393438 https://news.ycombinator.com/item?id=7393438
  - pessimizer 12y ago
    Found it about 10 secs ahead of your post - was coming back to edit it into my comment:) Thanks. >So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you'd have a recognition accuracy of ~66%, which is not great. Out of 100,000 people, I'd assume that there'd be many pairs that wouldn't be distinguishable by their parents, or even by each other. Wouldn't you eventually run into a quantum effect where the differences between people's faces from image-to-image would be larger than the difference between someone's face and everyone in a corpus of X million people? I'd think that a 66% would crush if you had 5 (for example) images of each person you were trying to identify, especially if they were intentionally taken from very different angles or in very different lighting, and you were trying to identify a person that you had 3 images of. >Finally, there are already lots of applications to face recognition other than surveillance I'm curious about those.
dheera 12y ago
I'm pretty sure in reality Facebook also uses your social network graph to restrict the candidate set and get higher recognition accuracy. This makes it hard to compare Facebook's results to a pure recognition algorithm. On that note, Facebook could do even better by restricting the candidate set based on time, precise location, and compass orientation, given that most mobile users have Facebook installed and are running it in their pockets when they get their picture taken by others. (They could do rough recognition purely based on position and orientation without even looking at the camera image, if they really wanted to, so with the camera image it could really be near 100% accurate, and even work if you take a picture of a friend's back.)
- apu 12y ago
  Sure, but the numbers in the research paper are "pure" (i.e., not using all this additional information). In production, I'm sure they must be using all these additional cues as well.
  - dheera 12y ago
    Ah, I see, you mean the 97.35% is a "pure" algorithmic result. That's pretty impressive.
- dkhar 12y ago
  DeepFace's figures[1] also come from running it on the LFW dataset[2]. While Facebook's production tech will be more accurate because of what you say, the Deepface algorithm has a lot of raw power, too! [1] http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr... [2] http://vis-www.cs.umass.edu/lfw/index.html http://vis-www.cs.umass.edu/lfw/index.html
hcarvalhoalves 12y ago
> Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model. It's my understanding that deep learning is more interesting when you have a large dataset and need to generalize (e.g., recognize faces sideways, upside-down, etc), otherwise, when the requirements are more predictable, the hand-tuned algorithms tend to perform better. The task of finding whether two pictures are from the same person doesn't seem appropriate for DL (compared to e.g. find whether a picture contains a person). Would that be correct?
- apu 12y ago
  I'm not sure that's the distinction I'd use. To me, it seems more to be a matter of the kind and amount of training data you have. If you have a ton of labeled data (either for the exact task you're doing, or a related one), DL often wins because it's sufficiently high-capacity to learn sophisticated models from all that data. On the other hand, if you don't have enough data, then the deep network isn't going to be learnable, in which case hand-tuned methods might do better, depending on how much "domain expertise" you can bake into the system. Faces are a very constrained domain, and since there has been enormous work on them for decades now, it's one area that non-DL algorithms probably still have somewhat of a fighting chance (until people gather & label large enough databases, of course).
  - hcarvalhoalves 12y ago
    That makes more sense, thank you.
jastanton 12y ago
even with Facebook's 97.35% they still confuse me with my toddlers :P
PeterisP 12y ago
Actually, for humans this actually is a face recognition task - from the given examples, it seems that the dataset involves pictures of publicly known people, so it's comparing apples and oranges as humans are given an entirely different task. Computers are given 2 pictures and asked "Are these the same person?" However, if people are given the same 2 pictures, and one of them is a well-known actor; then the question is "Is the second picture the same person as the many pictures of this actor that I have seen during my life?" which is a rather different task. If you want a fair comparison, then either you have to use portraits of people that nobody knows, or the computer systems need to use databases of celebrity pictures for that.

sp332 12y ago

I think the breakthrough with Facebook's system was the idea to transform the face before running the training or verification on it. And the new system seems to use the same idea: The new algorithm works by normalising each face into a 150 x 120 pixel image, by transforming it based on five image landmarks: the position of both eyes, the nose and the two corners of the mouth.

apu 12y ago
No. Alignment is done by every face verification method, and even the "new" 3d alignment done here is on par with other alignment methods in practice today. At their talk at CVPR, they made clear that the biggest factor in getting their numbers was having tons of training data and a high-capacity learning model. The paper even shows that if you replace their 3d alignment with a standard 2d affine alignment (something people were doing 15 years ago), you lose only 1.5% on LFW (table 2).

pessimizer 12y ago

>Face verification gives an algorithm (or a person) two images of faces and asks, "Are these the same person?" >97.53% for people I'm surprised that people are that good - I think I'd be horrible at it. From googling, I can see that number came from Facebook - would you happen to know where to find the paper? >Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model. Does using this different model gain some sort of space or processing efficiency over DeepFace? Also: does anyone know the point of fighting to close this 3% gap? That seems good enough for any purpose. (Is there any other purpose to this technology other than mass surveillance?)

apu 12y ago
Here's the paper: https://www.facebook.com/publications/546316888800776/ https://www.facebook.com/publications/546316888800776/ The methods are quite different, but I think at root, it's just more being able to take advantage of much more training data. The 97.53% "human performance" number comes from my paper from a few years ago [0]. We initially thought that there's no way we would be able to do that well on this task (manually) either, but then we tried it and our scores were much higher (very close to 100%). So the verification task on LFW is actually not that hard, although in part it's because the dataset consists of public figures and therefore you can recognize many of the people outright if you've seen them before. (We tried to measure this in our human studies but found it to be inconclusive.) 3% is not the right way to look at error rates, btw. See [1] for why. But also, face verification is more of a low-level building block for recognition, because most real-world applications don't directly care about saying whether two faces are the same person or not ("verification"), but "who is this person?" ("recognition"). A simple way to build recognition from verification is to compare a test face against all faces in the database, and then take the one that gets ranked highest. In that (simplistic) scenario, your recognition rate scales (very roughly) as acc^lg(N), where N is the number of people in your database and acc is the verification accuracy. So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you might have a recognition accuracy of ~66% (very roughly!), which is not great. Finally, there are already lots of applications to face recognition other than surveillance, and many more waiting to be discovered. [0] http://homes.cs.washington.edu/~neeraj/projects/faceverification/ http://homes.cs.washington.edu/~neeraj/projects/faceverifica... [1] https://news.ycombinator.com/item?id=7393438 https://news.ycombinator.com/item?id=7393438
- pessimizer 12y ago
  Found it about 10 secs ahead of your post - was coming back to edit it into my comment:) Thanks. >So if, e.g., you had 100,000 people to recognize from and your verification accuracy was 97.53%, you'd have a recognition accuracy of ~66%, which is not great. Out of 100,000 people, I'd assume that there'd be many pairs that wouldn't be distinguishable by their parents, or even by each other. Wouldn't you eventually run into a quantum effect where the differences between people's faces from image-to-image would be larger than the difference between someone's face and everyone in a corpus of X million people? I'd think that a 66% would crush if you had 5 (for example) images of each person you were trying to identify, especially if they were intentionally taken from very different angles or in very different lighting, and you were trying to identify a person that you had 3 images of. >Finally, there are already lots of applications to face recognition other than surveillance I'm curious about those.

dheera 12y ago

I'm pretty sure in reality Facebook also uses your social network graph to restrict the candidate set and get higher recognition accuracy. This makes it hard to compare Facebook's results to a pure recognition algorithm. On that note, Facebook could do even better by restricting the candidate set based on time, precise location, and compass orientation, given that most mobile users have Facebook installed and are running it in their pockets when they get their picture taken by others. (They could do rough recognition purely based on position and orientation without even looking at the camera image, if they really wanted to, so with the camera image it could really be near 100% accurate, and even work if you take a picture of a friend's back.)

apu 12y ago
Sure, but the numbers in the research paper are "pure" (i.e., not using all this additional information). In production, I'm sure they must be using all these additional cues as well.
- dheera 12y ago
  Ah, I see, you mean the 97.35% is a "pure" algorithmic result. That's pretty impressive.
dkhar 12y ago
DeepFace's figures[1] also come from running it on the LFW dataset[2]. While Facebook's production tech will be more accurate because of what you say, the Deepface algorithm has a lot of raw power, too! [1] http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr... [2] http://vis-www.cs.umass.edu/lfw/index.html http://vis-www.cs.umass.edu/lfw/index.html

hcarvalhoalves 12y ago

> Also, it is interesting that they are not using deep learning for this. They are using a Discriminative Gaussian Process Latent Variable Model. It's my understanding that deep learning is more interesting when you have a large dataset and need to generalize (e.g., recognize faces sideways, upside-down, etc), otherwise, when the requirements are more predictable, the hand-tuned algorithms tend to perform better. The task of finding whether two pictures are from the same person doesn't seem appropriate for DL (compared to e.g. find whether a picture contains a person). Would that be correct?

apu 12y ago
I'm not sure that's the distinction I'd use. To me, it seems more to be a matter of the kind and amount of training data you have. If you have a ton of labeled data (either for the exact task you're doing, or a related one), DL often wins because it's sufficiently high-capacity to learn sophisticated models from all that data. On the other hand, if you don't have enough data, then the deep network isn't going to be learnable, in which case hand-tuned methods might do better, depending on how much "domain expertise" you can bake into the system. Faces are a very constrained domain, and since there has been enormous work on them for decades now, it's one area that non-DL algorithms probably still have somewhat of a fighting chance (until people gather & label large enough databases, of course).
- hcarvalhoalves 12y ago
  That makes more sense, thank you.

jastanton 12y ago

even with Facebook's 97.35% they still confuse me with my toddlers :P

PeterisP 12y ago

Actually, for humans this actually is a face recognition task - from the given examples, it seems that the dataset involves pictures of publicly known people, so it's comparing apples and oranges as humans are given an entirely different task. Computers are given 2 pictures and asked "Are these the same person?" However, if people are given the same 2 pictures, and one of them is a well-known actor; then the question is "Is the second picture the same person as the many pictures of this actor that I have seen during my life?" which is a rather different task. If you want a fair comparison, then either you have to use portraits of people that nobody knows, or the computer systems need to use databases of celebrity pictures for that.