7 ms·

A Look Into Facebook's Potential to Recognize Anybody's Face

42 points by dkoch 13y ago

Irishsteve 13y ago
If each Facebook user has at least 100 photos they are tagged in under a variety of scenarios, quality etc. I'd of thought that would be a pretty reliable way of training some vision algo. Seems thats not the case.
- apu 13y ago
  It would, perhaps, but the problem is deciding between millions (or billions) of people -- if you recognize people by doing pair-wise comparisons, then even small error rates can quickly add up.
  - seiji 13y ago
    All you have to do is turn people into a feature vector then KNN them. Perhaps once you're in a smaller neighborhood, you can do more intensive analysis for high probability equality matching.
    - apu 13y ago
      This is not going to work (at least currently). Features for faces are not yet powerful enough to get you to a reasonable set of matches without also missing the right face in a large percentage of cases. Variations due to lighting, pose, and expression end up causing the same person to look very different -- often to the point where two different people in the same configuration can look more similar than the same person in different configurations. That being said, if the database is small enough (e.g., in a limited scenario or by applying other non-vision filters first), then the state of the art methods do use a similar approach. However, in practice people use methods with much stronger priors than KNN. Because faces fall into a low-dimensional manifold[1], you can take advantage of that to constrain queries much more than with generic KNN. [1] My PhD advisor has published several works about this topic, e.g., http://vision.ucsd.edu/kriegman-grp/papers/ijcv98.pdf http://vision.ucsd.edu/kriegman-grp/papers/ijcv98.pdf http://virtualhost.cs.columbia.edu/~belhumeur/journal/invariance_v2.pdf http://virtualhost.cs.columbia.edu/~belhumeur/journal/invari... http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.123.3344&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.123...
  - angersock 13y ago
    If only we had some social context information available to differentiate the three other faces that look like mine... ...oh wait. Fuck. :(
  - Irishsteve 13y ago
    Stick in some geo information as well (If you are trying to identify Mr X at location Y). That might work ?
    - apu 13y ago
      Yes, as the story points out, searching the entire database for a face probably won't work, but if you can prefilter based on other criteria such as location, employer, etc., it might become more feasible.
hnriot 13y ago
This, I suspect, is why In-Q-Tel (CIA) was an original investor in FB.
speedyrev 13y ago
Not only do they have face models. But they have immense details about those faces. Location, relationship connections, product endorsements, work and school connections... After the last few revelations about the NSA that have come out. I think you should assume that it is all being used.
phyalow 13y ago
I'm not surprised given the companies name...
jcutrell 13y ago
I heard from an unnamed Google employee once about very similar responsibility-laden abilities with the massive number of images Google has access too. In particular, a conversation about recognition algorithms for sorting pornography at the age-level was brought up. Now, I know this has been covered on here before, but what a massive amount of incriminating information must be kept in those servers. I really would like to be able to have access to facial model information for my hypothetical advertising company. Can you imagine if I could, for instance, find out what primary five face shapes people in Iowa find trustworthy, with a high (enough) level of accuracy? I'd pay a whole lot for that, hypothetically.
BIair 13y ago
I think it would be incredibly cool just to see your doppelgangers. How many people look like you, and how similar they are. Similar to searching for your name, only to discover other people have the same name, and the curiosity of who they are. Imagine searching for people who have the same face?
amirmc 13y ago
This reminded me a of a TED talk I saw recently: https://www.youtube.com/watch?v=H_pqhMO3ZSY https://www.youtube.com/watch?v=H_pqhMO3ZSY It's only 15mins long but you can skip to 2:35 for one of the face experiments and to 8:00 for an experiment they're running now. That latter experiment is quite creepy but I'd love to know the results.

42 points by dkoch 13y ago

If each Facebook user has at least 100 photos they are tagged in under a variety of scenarios, quality etc. I'd of thought that would be a pretty reliable way of training some vision algo. Seems thats not the case.

apu 13y ago
It would, perhaps, but the problem is deciding between millions (or billions) of people -- if you recognize people by doing pair-wise comparisons, then even small error rates can quickly add up.
- seiji 13y ago
  All you have to do is turn people into a feature vector then KNN them. Perhaps once you're in a smaller neighborhood, you can do more intensive analysis for high probability equality matching.
  - apu 13y ago
    This is not going to work (at least currently). Features for faces are not yet powerful enough to get you to a reasonable set of matches without also missing the right face in a large percentage of cases. Variations due to lighting, pose, and expression end up causing the same person to look very different -- often to the point where two different people in the same configuration can look more similar than the same person in different configurations. That being said, if the database is small enough (e.g., in a limited scenario or by applying other non-vision filters first), then the state of the art methods do use a similar approach. However, in practice people use methods with much stronger priors than KNN. Because faces fall into a low-dimensional manifold[1], you can take advantage of that to constrain queries much more than with generic KNN. [1] My PhD advisor has published several works about this topic, e.g., http://vision.ucsd.edu/kriegman-grp/papers/ijcv98.pdf http://vision.ucsd.edu/kriegman-grp/papers/ijcv98.pdf http://virtualhost.cs.columbia.edu/~belhumeur/journal/invariance_v2.pdf http://virtualhost.cs.columbia.edu/~belhumeur/journal/invari... http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.123.3344&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.123...
- angersock 13y ago
  If only we had some social context information available to differentiate the three other faces that look like mine... ...oh wait. Fuck. :(
- Irishsteve 13y ago
  Stick in some geo information as well (If you are trying to identify Mr X at location Y). That might work ?
  - apu 13y ago
    Yes, as the story points out, searching the entire database for a face probably won't work, but if you can prefilter based on other criteria such as location, employer, etc., it might become more feasible.

hnriot 13y ago

This, I suspect, is why In-Q-Tel (CIA) was an original investor in FB.

speedyrev 13y ago

Not only do they have face models. But they have immense details about those faces. Location, relationship connections, product endorsements, work and school connections... After the last few revelations about the NSA that have come out. I think you should assume that it is all being used.

phyalow 13y ago

I'm not surprised given the companies name...

jcutrell 13y ago

I heard from an unnamed Google employee once about very similar responsibility-laden abilities with the massive number of images Google has access too. In particular, a conversation about recognition algorithms for sorting pornography at the age-level was brought up. Now, I know this has been covered on here before, but what a massive amount of incriminating information must be kept in those servers. I really would like to be able to have access to facial model information for my hypothetical advertising company. Can you imagine if I could, for instance, find out what primary five face shapes people in Iowa find trustworthy, with a high (enough) level of accuracy? I'd pay a whole lot for that, hypothetically.

BIair 13y ago

I think it would be incredibly cool just to see your doppelgangers. How many people look like you, and how similar they are. Similar to searching for your name, only to discover other people have the same name, and the curiosity of who they are. Imagine searching for people who have the same face?

amirmc 13y ago

This reminded me a of a TED talk I saw recently: https://www.youtube.com/watch?v=H_pqhMO3ZSY https://www.youtube.com/watch?v=H_pqhMO3ZSY It's only 15mins long but you can skip to 2:35 for one of the face experiments and to 8:00 for an experiment they're running now. That latter experiment is quite creepy but I'd love to know the results.