12 ms·

Russian photographer matches random people with social network photos

259 points by davesailer 11y ago

larrik 11y ago
The artist (or the article) suggests that the pictures he took on the metro are the 'real' person, whereas the pictures on the social site are the 'ideal' person. I'm not so sure. I think its very possible that the person in the social network photographs is actually far closer to the person they really are than the photo of them riding alone on the subway.
- blakesterz 11y ago
  >> whereas the pictures on the social site are the 'ideal' person. Maybe it's that the picture on the web was chosen. Someone put thought into that, they said "oh hey I look good here", it's kinda like an ideal, or at least an above average view of themselves.
  - anotheryou 11y ago
    Of course my profile pic is a chosen ideal of myself. But you know i chose it and it depicts how I want to be seen. This says much more about me, than me monday morning or after a long of work, bored, alone, with the worst lighting conditions (blueish light from my mobile screen from below and neon lights from too far above).
  - pbhjpbhj 11y ago
    Just like in other social situations, people idealise themselves at work or when out with friends rather than presenting an evenly balanced image of their inner self. It's more surprising when people "overshare" and detail intimate information about themselves IME. Who looks through photos to choose to preserve and doesn't pick the one's in which they look "better", doubly so when sharing those photos in public?
- JackFr 11y ago
  Further, the artist is clearly curating what he shows and what he doesn't and likely has his own message.
  - bigiain 11y ago
    To be fair, he's pitching this as an art project, not a piece of social media or privacy research. Of _course_ he's curating the output to maximise his art's impact. (I really like this sort of "subversive art", that has deeply important educational aspects showing people how the world really is in interesting ways...)
- fapjacks 11y ago
  I don't think we are very good barometers for what kind of people we are. This is why having a confidant/BFF is so important.
- sp332 11y ago
  I thought exactly the same - and the photos chosen for the article don't really back up the point either. The pairs look like the same "person" to me, not one real and one fake. Anyway I don't know anybody who busts out their truest expression of self to ride the subway.
  - tom_mellior 11y ago
    > The pairs look like the same "person" to me, not one real and one fake. I seem to be alone in this, but... they don't even look like the same person to me. Do we have any confirmation that the pairs show the same person? About 10 million Russian men look like the guys in the second pair of photos.
    - funkdobiest 11y ago
      I have a friend from Czech who looks just like that guy. I was wondering the same.
    - burkaman 11y ago
      I think they just picked the two most striking examples for that article. Look at the rest here: https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-big-data.html https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-... Most of them are clearly the same person.
      - aab0 11y ago
        I dunno. Some of them clearly are because you can see they have moles in the same place and stuff like that, but a bunch leave me dubious. For example, that woman with the dyed white hair: the hair looks much frizzier/curlier in one than the other. And some I wonder how the face algorithm could possibly have matched them up - the guy in the parka vs the soldier don't even have the same facial hair, never mind how the parka fuzz obscures most of his head. (And you can't see more than a quarter of the next woman's face!) I wonder how much of this matching was manual.
        11y ago
- morganvachon 11y ago
  > I'm not so sure. I think its very possible that the person in the social network photographs is actually far closer to the person they really are than the photo of them riding alone on the subway. That's how I see it too, but that may be tempered by my personality. When I'm on public transit or in a crowd, I do my best not to stand out (which is difficult being 6'4" and 240lbs) but when I'm around people I know, I will smile and have fun. When I go to a concert alone, I'm also a bit closed off and tend to dress blandly, but if I go with a friend or a group I'm more lively and interactive, and dressed to suit the music. It's all about being comfortable with those around you, at least for me. So yes, on the subway I'm the dull guy in the jeans and t-shirt, and on social media I'm almost always with friends in a photo so I'm smiling and having fun. Nothing out of the ordinary in these photos if you ask me.
- a3n 11y ago
  Art can make you think or feel, and this art certainly makes me imagine, regardless of the artist's intended message.
blakesterz 11y ago
I keep trying to figure out why this feels so wrong and creepy. You're in public, you can't avoid some random person from taking your picture. You have a public profile, you post pictures of yourself there, in public. But some how this feels... wrong, why is that? Something about the surprise of it? It feels like a surprise, it feels like something that shouldn't be so easy.
- deleted 11y ago
  [deleted]
- lumberjack 11y ago
  https://commons.wikimedia.org/wiki/Commons:Country_specific_consent_requirements https://commons.wikimedia.org/wiki/Commons:Country_specific_... And in most places it still wouldn't be socially acceptable even if it may be legal.
- mfoy_ 11y ago
  This is just a taste... Now imagine what it must feel like to be a celebrity and have this be an every day occurrence-- hounded by paparazzi.
- eCa 11y ago
  > But some how this feels... wrong, why is that? It is not voluntary.
- Kalium 11y ago
  There's a... presumption that this is difficult and that all these disparate services and fragments of identity are distinct and separated. This stunt demonstrates that this is not true. It violates some basic assumptions.
  - AstralStorm 11y ago
    Yes, it gets easier and easier to correlate enough metadata and real data to find out many interesting things about many a person.
- 11y ago
prohor 11y ago
So, you can no longer assume being anonymous when walking down the street. If your random photo is matched with your profile, you can be basically fully identified anywhere. With all those cameras everywhere, you could be fully tracked.
- Karlozkiller 11y ago
  Given that there are public images with your face in them and name connected to them, that can be searched by a neural net... Yes.
  - brohee 11y ago
    No need for the images to be public. A police state could match images taken by street CCTV to e.g. the drivers license database and track the citizenry in real time. You could easily reduce the search space to match a face taken by a CCTV camera by using cell phone location data to limit it to people in the general area...
    - Karlozkiller 11y ago
      A police state has always been able to do this. Of course it gets easier by automatic face recognition and the possibility of sharing databases over long distances etc. The difference from before I guess, is that it's less in your face, and can thereby happen without people noticing as easily.
      - chopin 11y ago
        >A police state has always been able to do this. Not for everybody all the time, though...
- TeMPOraL 11y ago
  You could never assume that. The only thing that gives you anonymity on a public street is the fact that you're not interesting. Since most people walk mostly the same paths all the time, it's pretty much guaranteed that if you suddenly started to do something weird, at least some people on the street would recognize you and could - in principle - figure out who you are. Also consider that your local grocery store clerk probably knows your eating and drinking habits better than you know them yourself.
justsaysmthng 11y ago
I don't see why this can't be done in real time. Point your phone towards a person in a cafe and immediately figure out her name, age, maybe relationship status, social class, augmented reality style. 5 minutes later accidentally bump into her .. and start a conversation, armed with lots of private info about the person. Or even more basic - thieves picking their targets - is this woman's purse likely to contain a lot of money and jewelry ? I guess you could even write a tool which picks all the faces from a photo, and detects who's probably richest, most popular, most followed, etc. But this is unavoidable, right ? We just have to accept it as a fact of life and adapt to it..
- xlm1717 11y ago
  This describes almost exactly the world of the video game Watch Dogs. Didn't necessarily use a photo though, just a connection to the other person's phone. In the world of the video game, people did adapt to it. However, there was a faction that actively tried to destroy that system.
- foltz 11y ago
  How long before anti-facial recognition makeup, hair, and accessories becomes fashionable? https://cvdazzle.com https://cvdazzle.com
  - cryptoz 11y ago
    That doesn't seem like it would work at all. The makeup and hair styles suggested would make you stand out even more! Sure, 1990s-style facial recognition won't work as reliably, but surely today's systems would flag you right away!
    - foltz 11y ago
      I haven't tested it myself but according to CVdazzle it works. So the next question is how long before facial recognition technology overcomes efforts to confound it. Or how long until legislators get involved. Yes the looks are ridiculous, but most fashion starts out that way. People often look for ways to stand out and being "retro" could enhance the cool factor. What does someone care if a mugger's facial recognition software "flags" them? And who exactly is being flagged? And if a security surveillance system flags you what is the consequence? Who does that consequence apply to?
mamon 11y ago
If this becomes more common then dating websites, like Badoo are doomed :)
naskwo 11y ago
And this is exactly why, 11 years ago, I set up Famipix (www.famipix.com).
- pbhjpbhj 11y ago
  Link is to a "private photo sharing" website. I'm curious naskwo, it says in the tour "We don’t synchronise any content to any external sites or social networks." - this implies you don't have off-site backups? Second, for the birthdays calendar do you do anything like attempting to recognise photos of birthdays and log them, or use image meta-data to do the same?
nkw 11y ago
This is one of the cool features people have discussed for the past 20 years for wearable computers. I was hoping it would come to fruition with Google glass until Google got all weird about facial recognition on glass (and subsequently glass as a whole). On one side it is creepy, on the other it would be nice to have your digital assistant suitably remind you the name of the person who is walking towards you and recognizes you but you cannot remember who they are. A quick summary of who they are and perhaps a summary of your last interaction whispered in your ear could be handy. Or even a couple of relevant recent headlines about their company, etc. Of course creepy side is pointing at random stranger on street and instantly having their digital dossier overlaid on their face.
- yolesaber 11y ago
  I personally would be pretty disgusted if someone used it on me. If someone doesn't remember my name or their last interaction with me, maybe it's for the best. The last thing I need is people faking nice and friendly after they just scanned me without my consent.
  - type0 11y ago
    But this could be useful for someone with Alzheimers, or maybe a teacher that needs to remember his pupils.
    - TeMPOraL 11y ago
      Or normal people who, like me, can't connect names to faces without at least several interactions with a person in a short interval of time.
      - yolesaber 11y ago
        So instead of working on your memory or coming up with a mnemonic or just asking their name again (everyone forgets a name), you think it'd be less intrusive to scan someone without their permission?
supremeanger 11y ago
Easy to mitigate: https://cvdazzle.com/ https://cvdazzle.com/
xentronium 11y ago
FindFace is a very interesting piece of technology. It's legit (the authors, N-Tech.Lab, have successfully competed against google and other strong opponents in MegaFace Benchmark). It's not real-time, but fast enough (0.5-1.0 minute time required to recognize faces and provide their profile links). Their business model is freemium (first 10 or 30 searches per month are free), and they're trying to make it a matching service, you can narrow search by age and relationship status, and then you can like photos of people you find to allow searching for "your type" of partners.
- hayksaakian 11y ago
  their website was pretty unhelpful is there a signup page somewhere i want to try it out
  - gk1 11y ago
    Here you go: http://findface.ru/ http://findface.ru/ Looks as though it's only available in Russian and only works for VKontakte (the social network). Warning: It highjacks your back button.
    - hayksaakian 11y ago
      thank you
gwern 11y ago
I am not sure this works or that success rates like claimed are possible: > In 70 percent of cases, he was quite easily able to identify the people he had photographed. VKontakte has 280m+ accounts and is the second most popular website in Russia (https://en.wikipedia.org/wiki/VK_%28social_networking%29); https://en.wikipedia.org/wiki/VK_%28social_networking%29); the Russian population is ~144m. I can't find any decent estimate of how many Russians use VK, but it seems to be at least as dominate in Russia as Facebook in the USA, which is to say, ~71% (http://www.pewinternet.org/fact-sheets/social-networking-fact-sheet/ http://www.pewinternet.org/fact-sheets/social-networking-fac...). So from non-VK usage alone, he should be able to identify no more than 70-80 percent since you can't identify those who aren't on VK in the first place. Facial recognition is a notoriously difficult machine learning task, and doing facial recognition on the VK population of St Petersburg (>249688 * 0.7 = >174781) will be challenging even with extremely good facial recognition. I don't know what this 'FindFace' application is, but I'll be charitable and assume it matches the 2015 state of the art, "Facenet: A unified embedding for face recognition and clustering" http://arxiv.org/abs/1503.03832 http://arxiv.org/abs/1503.03832 AFAIK, with a best-case binary classification accuracy of 99.63% (https://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification https://en.wikipedia.org/wiki/Accuracy_and_precision#In_bina...): 0.9963 = true positives + true negatives / true positives + true negatives + false negatives + true negatives If each person = 1 profile photo, then to classify a photo of someone there would be 1 true positive, 174781-1 true negatives, and unknown number of false negatives. 0.9963 = (1 + (174781-1)) / 1 + (174781-1) + x + (174781-1) 0.9963 = 174781 / 349561 + x 0.9963349561(1/174781) = x 1.9925943 = x There would be ~2 false negatives for every true positive. So there would be a <1/3 chance the photographed person is not on VK at all, and when one searched for them when they are in fact on VK, they would only show up <1/3 of the time (the other two-thirds their subway photograph would not 'match' when tested in the dataset and it would be a false negative, and it gets much worse if the face detection has one of the lower state of the art accuracies on the other datasets). So on net you would expect to turn up a person <07 * 0.3 = <0.21 of the time. (It gets worse if we take into account the false positives as well but that particular paper doesn't report precision.) There is no mention of any attempt to validate the findings, so I think for many of his results, he may have merely found a similar-looking person (possibly in some cases, look-alikes http://www.nytimes.com/2014/08/26/science/looking-at-twin-personality-through-look-alikes.html http://www.nytimes.com/2014/08/26/science/looking-at-twin-pe... https://en.wikipedia.org/wiki/Look-alike https://en.wikipedia.org/wiki/Look-alike ). Look-alikes are far from identical twins, so it's not necessarily surprising that > Often, there was a striking difference between a person’s real look and the image they projected on their social network profile: a shy and grim young man might appear to be the life and soul of the party and a lover of extreme sports on his VKontakte page. This could be due to care in social presentation. Or it could just be that they're not the same person in the first place. (The photos are not that persuasive. I'm sure there's more than one grim young men with widow's peaks in St Petersburg, or young women with curly black hair.)
alexc05 11y ago
The first thing that struck me (After I got over the whole, creepy/horror/revulsion thing) was how much more fun people seemed in their social media lives. Rather than seeing this as an exposition of "people on social media are so much more bland than they appear" I found myself feeling "people on the subway are so much more beautiful and interesting in their 'real life'" I feel like that might be backwards from how it gets presented elsewhere. But, if you look for example at the blonde woman snarling a piece of cake https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-big-data.html https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-... (second last) I thought, that was charming and funny... But you'd never know that from under they grey fluorescent lights of the subway car. It's really interesting to be confronted with the realization that every single person filling up space on the subway around you has a rich and lustrous story that brought them to that place and time. Still, super creepy.
- tcdent 11y ago
  In a word: empathy. Nothing beats the visceral explanation of a demo, though.
- deleted 11y ago
  [deleted]
- djhn 11y ago
  Don't ignore the advantage of having been taken by a professional photographer, who maybe even specializes in portraits of people, with a high-quality lens attached to a high-quality body.
- xerophyte12932 11y ago
  You are right, I saw this as that as well. Someone who looks bland may actually have a very fun and colorful life. That's an important lesson here.
- sangnoir 11y ago
  > It's really interesting to be confronted with the realization that every single person filling up space on the subway around you has a rich and lustrous story that brought them to that place and time. The dictionary of obscure sorrows[1] names this feeling "Sonder" The realization that each random passerby is living a life as vivid and complex as your own—populated with their own ambitions, friends, routines, worries and inherited craziness 1. it's a made-up word, not a formal English one. http://www.dictionaryofobscuresorrows.com/post/23536922667/sonder http://www.dictionaryofobscuresorrows.com/post/23536922667/s...
zmitri 11y ago
Another corollary of this is that if you take a good image of someone else, you can build a model of it (sometimes automatically), and you can map it on to your face and appear as them digitally. I've built tools like this that even let you do it live in the browser using this library https://github.com/auduno/clmtrackr/ https://github.com/auduno/clmtrackr/
bcheung 11y ago
I've seen this happen in real time with augmented reality googles. It had a bunch of the Facebook profiles in advance though so it was a much smaller search space.
brokencog 11y ago
Completely missing the point. "Social Media Site" aka, a place where we put (mostly) public imagery. By definition this is curated, and as the prof says is a self-selected view of ourselves. But, the point being missed is that OF COURSE people don't look the same "in day to day life" because those social media pics are just a moment of one day. If the prof happened to meet the person moments after the social media picture was taken then ... he would have concluded that "people present themselves truly as they are in reality" ... My point, the point ignored in the article, is that Social Media is a self-reflective, self-curated snapshot. Not a continuous, objective representation.
jpatokal 11y ago
Direct link to the photographer's site: https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-big-data.html https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-...
imaginenore 11y ago
I don't believe these claims. These photos are matched too well, and I suspect he did it manually. Some of the train photos are so bad, there's no way they got automatically recognized. Not to mention the resources you would need just to get the data on 280 million users, let alone train a neural net on it, and make it work with a 70% success rate. I think it's a viral way to get app installs. The app reviews are really bad btw, it doesn't work.
- egjerlow 11y ago
  I agree that this seems very far-fetched. From what I know about facial recognition (admittedly very little) for an algorithm to be able to recognize these profiles in such different conditions as displayed in the article seems very unlikely. Incidentally, is there anyone who could ELI5 how picture data like this is treated in order to run these algorithms on it?
eggy 11y ago
And friends bash me for always using a caricature avatar of myself on my social media accounts! I'll have to run it, and see if it matches a photo. It's a greyscale line drawing sketch.
known 11y ago
If drones have Facial recognition software they'll become ultimate killing machines;
flashman 11y ago
Minority Report's personal advertising is getting closer and closer: https://www.youtube.com/watch?v=7bXJ_obaiYQ https://www.youtube.com/watch?v=7bXJ_obaiYQ And they won't even need retinal scanners. Facial recognition will probably be good enough.
stefanix 11y ago
This is very much a study of bad vs good lighting.
prawn 11y ago
We are not far off AR-type scenarios where someone in public is overlaid with info about them and their interests. Keen on that girl? It instantly tells you she likes a particular band and movie genre, giving you quick access to small talk cues. Or that she never mentions a boyfriend online. Or that the guy about to get off the subway earlier thanked his friend for giving him that money he owed him.
huuu 11y ago
Reminds me of "Stranger Visions" from Heather Dewey-Hagborg. She created 3D faces by using DNA found in public places: http://deweyhagborg.com/projects/stranger-visions http://deweyhagborg.com/projects/stranger-visions
jcoffland 11y ago
I found it interesting that with out exception in the photos on the artist's Website the people on the subway are activity using their smart phones and/or have headphones on. The transformation of the human race into cyborgs, of sorts, over the last two decades is uncanny.
xerophyte12932 11y ago
It's interesting how almost everyone he photographed completely ignored him by either closing their eyes, diving into their cellphone or just pretending to be interested in something else
bflesch 11y ago
I have registered http://wikiface.org/ http://wikiface.org/ some time ago to create an open facial recognition database as a proof of concept project.
dsfyu404ed 11y ago
Maybe it's not a popular look over there but I couldn't help but notice there were no photos of guys with a full beard, ball cap and aviator style sunglasses...

259 points by davesailer 11y ago

The artist (or the article) suggests that the pictures he took on the metro are the 'real' person, whereas the pictures on the social site are the 'ideal' person. I'm not so sure. I think its very possible that the person in the social network photographs is actually far closer to the person they really are than the photo of them riding alone on the subway.

blakesterz 11y ago
>> whereas the pictures on the social site are the 'ideal' person. Maybe it's that the picture on the web was chosen. Someone put thought into that, they said "oh hey I look good here", it's kinda like an ideal, or at least an above average view of themselves.
- anotheryou 11y ago
  Of course my profile pic is a chosen ideal of myself. But you know i chose it and it depicts how I want to be seen. This says much more about me, than me monday morning or after a long of work, bored, alone, with the worst lighting conditions (blueish light from my mobile screen from below and neon lights from too far above).
- pbhjpbhj 11y ago
  Just like in other social situations, people idealise themselves at work or when out with friends rather than presenting an evenly balanced image of their inner self. It's more surprising when people "overshare" and detail intimate information about themselves IME. Who looks through photos to choose to preserve and doesn't pick the one's in which they look "better", doubly so when sharing those photos in public?
JackFr 11y ago
Further, the artist is clearly curating what he shows and what he doesn't and likely has his own message.
- bigiain 11y ago
  To be fair, he's pitching this as an art project, not a piece of social media or privacy research. Of _course_ he's curating the output to maximise his art's impact. (I really like this sort of "subversive art", that has deeply important educational aspects showing people how the world really is in interesting ways...)
fapjacks 11y ago
I don't think we are very good barometers for what kind of people we are. This is why having a confidant/BFF is so important.
sp332 11y ago
I thought exactly the same - and the photos chosen for the article don't really back up the point either. The pairs look like the same "person" to me, not one real and one fake. Anyway I don't know anybody who busts out their truest expression of self to ride the subway.
- tom_mellior 11y ago
  > The pairs look like the same "person" to me, not one real and one fake. I seem to be alone in this, but... they don't even look like the same person to me. Do we have any confirmation that the pairs show the same person? About 10 million Russian men look like the guys in the second pair of photos.
  - funkdobiest 11y ago
    I have a friend from Czech who looks just like that guy. I was wondering the same.
  - burkaman 11y ago
    I think they just picked the two most striking examples for that article. Look at the rest here: https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-big-data.html https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-... Most of them are clearly the same person.
    - aab0 11y ago
      I dunno. Some of them clearly are because you can see they have moles in the same place and stuff like that, but a bunch leave me dubious. For example, that woman with the dyed white hair: the hair looks much frizzier/curlier in one than the other. And some I wonder how the face algorithm could possibly have matched them up - the guy in the parka vs the soldier don't even have the same facial hair, never mind how the parka fuzz obscures most of his head. (And you can't see more than a quarter of the next woman's face!) I wonder how much of this matching was manual.
      - 11y ago
morganvachon 11y ago
> I'm not so sure. I think its very possible that the person in the social network photographs is actually far closer to the person they really are than the photo of them riding alone on the subway. That's how I see it too, but that may be tempered by my personality. When I'm on public transit or in a crowd, I do my best not to stand out (which is difficult being 6'4" and 240lbs) but when I'm around people I know, I will smile and have fun. When I go to a concert alone, I'm also a bit closed off and tend to dress blandly, but if I go with a friend or a group I'm more lively and interactive, and dressed to suit the music. It's all about being comfortable with those around you, at least for me. So yes, on the subway I'm the dull guy in the jeans and t-shirt, and on social media I'm almost always with friends in a photo so I'm smiling and having fun. Nothing out of the ordinary in these photos if you ask me.
a3n 11y ago
Art can make you think or feel, and this art certainly makes me imagine, regardless of the artist's intended message.

blakesterz 11y ago

I keep trying to figure out why this feels so wrong and creepy. You're in public, you can't avoid some random person from taking your picture. You have a public profile, you post pictures of yourself there, in public. But some how this feels... wrong, why is that? Something about the surprise of it? It feels like a surprise, it feels like something that shouldn't be so easy.

deleted 11y ago
[deleted]
lumberjack 11y ago
https://commons.wikimedia.org/wiki/Commons:Country_specific_consent_requirements https://commons.wikimedia.org/wiki/Commons:Country_specific_... And in most places it still wouldn't be socially acceptable even if it may be legal.
mfoy_ 11y ago
This is just a taste... Now imagine what it must feel like to be a celebrity and have this be an every day occurrence-- hounded by paparazzi.
eCa 11y ago
> But some how this feels... wrong, why is that? It is not voluntary.
Kalium 11y ago
There's a... presumption that this is difficult and that all these disparate services and fragments of identity are distinct and separated. This stunt demonstrates that this is not true. It violates some basic assumptions.
- AstralStorm 11y ago
  Yes, it gets easier and easier to correlate enough metadata and real data to find out many interesting things about many a person.
11y ago

prohor 11y ago

So, you can no longer assume being anonymous when walking down the street. If your random photo is matched with your profile, you can be basically fully identified anywhere. With all those cameras everywhere, you could be fully tracked.

Karlozkiller 11y ago
Given that there are public images with your face in them and name connected to them, that can be searched by a neural net... Yes.
- brohee 11y ago
  No need for the images to be public. A police state could match images taken by street CCTV to e.g. the drivers license database and track the citizenry in real time. You could easily reduce the search space to match a face taken by a CCTV camera by using cell phone location data to limit it to people in the general area...
  - Karlozkiller 11y ago
    A police state has always been able to do this. Of course it gets easier by automatic face recognition and the possibility of sharing databases over long distances etc. The difference from before I guess, is that it's less in your face, and can thereby happen without people noticing as easily.
    - chopin 11y ago
      >A police state has always been able to do this. Not for everybody all the time, though...
TeMPOraL 11y ago
You could never assume that. The only thing that gives you anonymity on a public street is the fact that you're not interesting. Since most people walk mostly the same paths all the time, it's pretty much guaranteed that if you suddenly started to do something weird, at least some people on the street would recognize you and could - in principle - figure out who you are. Also consider that your local grocery store clerk probably knows your eating and drinking habits better than you know them yourself.

justsaysmthng 11y ago

I don't see why this can't be done in real time. Point your phone towards a person in a cafe and immediately figure out her name, age, maybe relationship status, social class, augmented reality style. 5 minutes later accidentally bump into her .. and start a conversation, armed with lots of private info about the person. Or even more basic - thieves picking their targets - is this woman's purse likely to contain a lot of money and jewelry ? I guess you could even write a tool which picks all the faces from a photo, and detects who's probably richest, most popular, most followed, etc. But this is unavoidable, right ? We just have to accept it as a fact of life and adapt to it..

xlm1717 11y ago
This describes almost exactly the world of the video game Watch Dogs. Didn't necessarily use a photo though, just a connection to the other person's phone. In the world of the video game, people did adapt to it. However, there was a faction that actively tried to destroy that system.
foltz 11y ago
How long before anti-facial recognition makeup, hair, and accessories becomes fashionable? https://cvdazzle.com https://cvdazzle.com
- cryptoz 11y ago
  That doesn't seem like it would work at all. The makeup and hair styles suggested would make you stand out even more! Sure, 1990s-style facial recognition won't work as reliably, but surely today's systems would flag you right away!
  - foltz 11y ago
    I haven't tested it myself but according to CVdazzle it works. So the next question is how long before facial recognition technology overcomes efforts to confound it. Or how long until legislators get involved. Yes the looks are ridiculous, but most fashion starts out that way. People often look for ways to stand out and being "retro" could enhance the cool factor. What does someone care if a mugger's facial recognition software "flags" them? And who exactly is being flagged? And if a security surveillance system flags you what is the consequence? Who does that consequence apply to?

mamon 11y ago

If this becomes more common then dating websites, like Badoo are doomed :)

naskwo 11y ago

And this is exactly why, 11 years ago, I set up Famipix (www.famipix.com).

pbhjpbhj 11y ago
Link is to a "private photo sharing" website. I'm curious naskwo, it says in the tour "We don’t synchronise any content to any external sites or social networks." - this implies you don't have off-site backups? Second, for the birthdays calendar do you do anything like attempting to recognise photos of birthdays and log them, or use image meta-data to do the same?

nkw 11y ago

This is one of the cool features people have discussed for the past 20 years for wearable computers. I was hoping it would come to fruition with Google glass until Google got all weird about facial recognition on glass (and subsequently glass as a whole). On one side it is creepy, on the other it would be nice to have your digital assistant suitably remind you the name of the person who is walking towards you and recognizes you but you cannot remember who they are. A quick summary of who they are and perhaps a summary of your last interaction whispered in your ear could be handy. Or even a couple of relevant recent headlines about their company, etc. Of course creepy side is pointing at random stranger on street and instantly having their digital dossier overlaid on their face.

yolesaber 11y ago
I personally would be pretty disgusted if someone used it on me. If someone doesn't remember my name or their last interaction with me, maybe it's for the best. The last thing I need is people faking nice and friendly after they just scanned me without my consent.
- type0 11y ago
  But this could be useful for someone with Alzheimers, or maybe a teacher that needs to remember his pupils.
  - TeMPOraL 11y ago
    Or normal people who, like me, can't connect names to faces without at least several interactions with a person in a short interval of time.
    - yolesaber 11y ago
      So instead of working on your memory or coming up with a mnemonic or just asking their name again (everyone forgets a name), you think it'd be less intrusive to scan someone without their permission?

supremeanger 11y ago

Easy to mitigate: https://cvdazzle.com/ https://cvdazzle.com/

xentronium 11y ago

FindFace is a very interesting piece of technology. It's legit (the authors, N-Tech.Lab, have successfully competed against google and other strong opponents in MegaFace Benchmark). It's not real-time, but fast enough (0.5-1.0 minute time required to recognize faces and provide their profile links). Their business model is freemium (first 10 or 30 searches per month are free), and they're trying to make it a matching service, you can narrow search by age and relationship status, and then you can like photos of people you find to allow searching for "your type" of partners.

hayksaakian 11y ago
their website was pretty unhelpful is there a signup page somewhere i want to try it out
- gk1 11y ago
  Here you go: http://findface.ru/ http://findface.ru/ Looks as though it's only available in Russian and only works for VKontakte (the social network). Warning: It highjacks your back button.
  - hayksaakian 11y ago
    thank you

gwern 11y ago

I am not sure this works or that success rates like claimed are possible: > In 70 percent of cases, he was quite easily able to identify the people he had photographed. VKontakte has 280m+ accounts and is the second most popular website in Russia (https://en.wikipedia.org/wiki/VK_%28social_networking%29); https://en.wikipedia.org/wiki/VK_%28social_networking%29); the Russian population is ~144m. I can't find any decent estimate of how many Russians use VK, but it seems to be at least as dominate in Russia as Facebook in the USA, which is to say, ~71% (http://www.pewinternet.org/fact-sheets/social-networking-fact-sheet/ http://www.pewinternet.org/fact-sheets/social-networking-fac...). So from non-VK usage alone, he should be able to identify no more than 70-80 percent since you can't identify those who aren't on VK in the first place. Facial recognition is a notoriously difficult machine learning task, and doing facial recognition on the VK population of St Petersburg (>249688 * 0.7 = >174781) will be challenging even with extremely good facial recognition. I don't know what this 'FindFace' application is, but I'll be charitable and assume it matches the 2015 state of the art, "Facenet: A unified embedding for face recognition and clustering" http://arxiv.org/abs/1503.03832 http://arxiv.org/abs/1503.03832 AFAIK, with a best-case binary classification accuracy of 99.63% (https://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification https://en.wikipedia.org/wiki/Accuracy_and_precision#In_bina...): 0.9963 = true positives + true negatives / true positives + true negatives + false negatives + true negatives If each person = 1 profile photo, then to classify a photo of someone there would be 1 true positive, 174781-1 true negatives, and unknown number of false negatives. 0.9963 = (1 + (174781-1)) / 1 + (174781-1) + x + (174781-1) 0.9963 = 174781 / 349561 + x 0.9963349561(1/174781) = x 1.9925943 = x There would be ~2 false negatives for every true positive. So there would be a <1/3 chance the photographed person is not on VK at all, and when one searched for them when they are in fact on VK, they would only show up <1/3 of the time (the other two-thirds their subway photograph would not 'match' when tested in the dataset and it would be a false negative, and it gets much worse if the face detection has one of the lower state of the art accuracies on the other datasets). So on net you would expect to turn up a person <07 * 0.3 = <0.21 of the time. (It gets worse if we take into account the false positives as well but that particular paper doesn't report precision.) There is no mention of any attempt to validate the findings, so I think for many of his results, he may have merely found a similar-looking person (possibly in some cases, look-alikes http://www.nytimes.com/2014/08/26/science/looking-at-twin-personality-through-look-alikes.html http://www.nytimes.com/2014/08/26/science/looking-at-twin-pe... https://en.wikipedia.org/wiki/Look-alike https://en.wikipedia.org/wiki/Look-alike ). Look-alikes are far from identical twins, so it's not necessarily surprising that > Often, there was a striking difference between a person’s real look and the image they projected on their social network profile: a shy and grim young man might appear to be the life and soul of the party and a lover of extreme sports on his VKontakte page. This could be due to care in social presentation. Or it could just be that they're not the same person in the first place. (The photos are not that persuasive. I'm sure there's more than one grim young men with widow's peaks in St Petersburg, or young women with curly black hair.)

alexc05 11y ago

The first thing that struck me (After I got over the whole, creepy/horror/revulsion thing) was how much more fun people seemed in their social media lives. Rather than seeing this as an exposition of "people on social media are so much more bland than they appear" I found myself feeling "people on the subway are so much more beautiful and interesting in their 'real life'" I feel like that might be backwards from how it gets presented elsewhere. But, if you look for example at the blonde woman snarling a piece of cake https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-big-data.html https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-... (second last) I thought, that was charming and funny... But you'd never know that from under they grey fluorescent lights of the subway car. It's really interesting to be confronted with the realization that every single person filling up space on the subway around you has a rich and lustrous story that brought them to that place and time. Still, super creepy.

tcdent 11y ago
In a word: empathy. Nothing beats the visceral explanation of a demo, though.
deleted 11y ago
[deleted]
djhn 11y ago
Don't ignore the advantage of having been taken by a professional photographer, who maybe even specializes in portraits of people, with a high-quality lens attached to a high-quality body.
xerophyte12932 11y ago
You are right, I saw this as that as well. Someone who looks bland may actually have a very fun and colorful life. That's an important lesson here.
sangnoir 11y ago
> It's really interesting to be confronted with the realization that every single person filling up space on the subway around you has a rich and lustrous story that brought them to that place and time. The dictionary of obscure sorrows[1] names this feeling "Sonder" The realization that each random passerby is living a life as vivid and complex as your own—populated with their own ambitions, friends, routines, worries and inherited craziness 1. it's a made-up word, not a formal English one. http://www.dictionaryofobscuresorrows.com/post/23536922667/sonder http://www.dictionaryofobscuresorrows.com/post/23536922667/s...

zmitri 11y ago

Another corollary of this is that if you take a good image of someone else, you can build a model of it (sometimes automatically), and you can map it on to your face and appear as them digitally. I've built tools like this that even let you do it live in the browser using this library https://github.com/auduno/clmtrackr/ https://github.com/auduno/clmtrackr/

bcheung 11y ago

I've seen this happen in real time with augmented reality googles. It had a bunch of the Facebook profiles in advance though so it was a much smaller search space.

brokencog 11y ago

Completely missing the point. "Social Media Site" aka, a place where we put (mostly) public imagery. By definition this is curated, and as the prof says is a self-selected view of ourselves. But, the point being missed is that OF COURSE people don't look the same "in day to day life" because those social media pics are just a moment of one day. If the prof happened to meet the person moments after the social media picture was taken then ... he would have concluded that "people present themselves truly as they are in reality" ... My point, the point ignored in the article, is that Social Media is a self-reflective, self-curated snapshot. Not a continuous, objective representation.

jpatokal 11y ago

Direct link to the photographer's site: https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-big-data.html https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-...

imaginenore 11y ago

I don't believe these claims. These photos are matched too well, and I suspect he did it manually. Some of the train photos are so bad, there's no way they got automatically recognized. Not to mention the resources you would need just to get the data on 280 million users, let alone train a neural net on it, and make it work with a 70% success rate. I think it's a viral way to get app installs. The app reviews are really bad btw, it doesn't work.

egjerlow 11y ago
I agree that this seems very far-fetched. From what I know about facial recognition (admittedly very little) for an algorithm to be able to recognize these profiles in such different conditions as displayed in the article seems very unlikely. Incidentally, is there anyone who could ELI5 how picture data like this is treated in order to run these algorithms on it?

eggy 11y ago

And friends bash me for always using a caricature avatar of myself on my social media accounts! I'll have to run it, and see if it matches a photo. It's a greyscale line drawing sketch.

known 11y ago

If drones have Facial recognition software they'll become ultimate killing machines;

flashman 11y ago

Minority Report's personal advertising is getting closer and closer: https://www.youtube.com/watch?v=7bXJ_obaiYQ https://www.youtube.com/watch?v=7bXJ_obaiYQ And they won't even need retinal scanners. Facial recognition will probably be good enough.

stefanix 11y ago

This is very much a study of bad vs good lighting.

prawn 11y ago

We are not far off AR-type scenarios where someone in public is overlaid with info about them and their interests. Keen on that girl? It instantly tells you she likes a particular band and movie genre, giving you quick access to small talk cues. Or that she never mentions a boyfriend online. Or that the guy about to get off the subway earlier thanked his friend for giving him that money he owed him.

huuu 11y ago

Reminds me of "Stranger Visions" from Heather Dewey-Hagborg. She created 3D faces by using DNA found in public places: http://deweyhagborg.com/projects/stranger-visions http://deweyhagborg.com/projects/stranger-visions

jcoffland 11y ago

I found it interesting that with out exception in the photos on the artist's Website the people on the subway are activity using their smart phones and/or have headphones on. The transformation of the human race into cyborgs, of sorts, over the last two decades is uncanny.

xerophyte12932 11y ago

It's interesting how almost everyone he photographed completely ignored him by either closing their eyes, diving into their cellphone or just pretending to be interested in something else

bflesch 11y ago

I have registered http://wikiface.org/ http://wikiface.org/ some time ago to create an open facial recognition database as a proof of concept project.

dsfyu404ed 11y ago

Maybe it's not a popular look over there but I couldn't help but notice there were no photos of guys with a full beard, ball cap and aviator style sunglasses...