5 ms·

Reverse-engineering facial recognition

drpgq 12y ago
As a researcher at a face recognition company, this is interesting, although the fact that algorithms change pretty frequently makes me think the work is quite ephemeral. Also I'm wondering if he's heard of liveness detection.
Houshalter 12y ago
Reminds me of this: https://www.youtube.com/watch?v=qS5HWBNvf9U https://www.youtube.com/watch?v=qS5HWBNvf9U
geekrax 12y ago
Sounds interesting. But the link received "HN hug of death".
lucb1e 12y ago
Cache: http://webcache.googleusercontent.com/search?q=cache:http://sterlingcrispin.com/data-masks.html http://webcache.googleusercontent.com/search?q=cache:http://... Edit: Original seems up again
coleca 12y ago
Since the site seems to be down, I'll mention that there was an article on Medium covering this artists' work a few days ago: https://medium.com/matter/this-is-what-your-face-looks-like-to-facebook-f771b3e11ed https://medium.com/matter/this-is-what-your-face-looks-like-... Seems like what he did was take Umass' Labeled Faces in the Wild (LFW) dataset, combine all the faces together and generate a composite rendering. Hardly a representation of "What your face looks like to Facebook". Articles like this do little to advance the discussion of the real concerns around facial recognition other than scare folks into an Orwellian vision of the future a la Hollywood and Minority Report. In addition to the work that the NTIA is doing around setting privacy standards for facial recognition applications (http://www.ntia.doc.gov/other-publication/2014/privacy-multistakeholder-process-facial-recognition-technology http://www.ntia.doc.gov/other-publication/2014/privacy-multi...) there are a lot of companies using the technology for good, but it is very difficult to get as much press and attention for those types of positive stories. For example, my own company, (http://www.kairos.com http://www.kairos.com) is working with some very brave and passionate guys at HelpingFaceless.com in India who are using facial recognition to help combat the growing problem of child trafficking and enslavement in India. See more about their story here: http://social.yourstory.com/2014/07/helping-faceless/ http://social.yourstory.com/2014/07/helping-faceless/
- lbenes 12y ago
  "Think of the children!" Whether it's for backdoors to your data in the cloud or facial recognition, protecting the children is ALWAYS what governments claim they need the tech for. Their track record is terrible, almost always using it for control and surveillance. I want to see cameras on every police officer in every city, before even considering more giving them more surveillance technology.
- sterlingcrispin 12y ago
  That's not what I did. I used a genetic algorithm with facial detection/recognition algorithms as fitness functions Recognition algorithms like eigenfaces, fischerfaces will reproduce one individual. More general detection algorithms like YEF realtime object detection as a fitness function will result in more general faces which represent some of the learned features from the LFW database. I'm not just combining images, there's lots of machine learning involved. Recheck my site in two weeks or so I'll be posting my MS thesis about the subject with more technical details. (Wrote this on my phone excuse the brevity)
  - michael_storm 12y ago
    You might want to summarize that at the top of your post, then, if you're going to submit it to HN. When it contains such nuggets as: These masks are shadows of human beings as seen by the minds-eye of the machine-organism. These DATA-MASKS are animistic deities brought out of the algorithmic-spirit-world of the machine and into our material world, ready to tell us their secrets, or warn us of what’s to come. It's not surprising that HN viewers are confused about what you actually did. In any case, seems like a cool project, now that I know what it is :-).
    - sterlingcrispin 12y ago
      Thanks, but btw someone else submitted this to HN I hadn't planned on fully publishing the project until a few weeks from now when my thesis is due but it was posted to medium.com so I quickly put some documentation together There are more technical details at the bottom of that page in the form of diagrams
MisterNegative 12y ago
This does pose the question whether these mask work in real life. I bet their software uses the fact that they can control the lightning and picture quality. Creating a direct copy of a face seems easier for real life purposes anyway. Other than demonstrating the flaws of current facial recognition algorithms there does not seem to be any use for this. Anyway, pretty cool use of evolutionary algorithms and nice pictures.
lbotos 12y ago
Reminds me of this: http://www.zachblas.info/projects/facial-weaponization-suite/ http://www.zachblas.info/projects/facial-weaponization-suite... Which is also pretty interesting.
- sterlingcrispin 12y ago
  His project is pure poetics/politics and has no direct relationship to computer vision Its as much anti facial recognition as a paper bag
- Houshalter 12y ago
  That video is incredibly grating and slow to the point. And the mask just seems to be random faces mixed together, but they did it in such a poor way it just produced random garbage. This guy seems to be optimizing his mask to fool facial recognition.
drcomputer 12y ago
> Computer systems built to represent human identities have contained with them many ontological assumptions about what it is to be an individual and what personal identity is. These systems define the human as a “what” ie: that which can be measured, not as a “who” ie: our inner self. > If the state of the art in computer science can produce a unique feature that describes an individual as such, what good does that do the individual if this knowledge is only leveraged against them? > If private citizens personal information, social graphs, and communications are being analyzed then the results should be made available to said persons to empower rather than enslave them. This attitude has become popular in personal fitness but not in communications, biometric identity, or social networks. > “We see the world, not as it is, but as we are” - Talmud I love this. Really fantastic work.
- sterlingcrispin 12y ago
  Thank you please recheck my site in a few weeks for my full thesis
deleted 12y ago
[deleted]
mattxxx 12y ago
Cool tangential topic: http://www.onformative.com/lab/googlefaces/ http://www.onformative.com/lab/googlefaces/ ...Reminds of being up at night as a kid, and seeing faces in shadows, patterns on the wall, vvhatever
userbinator 12y ago
I was hoping that an article about "reverse engineering facial recognition, facial detection, and image correlation techniques in order to reveal how they represent human identity" would have at least one mention of eigenfaces: http://en.wikipedia.org/wiki/Eigenface http://en.wikipedia.org/wiki/Eigenface
- sterlingcrispin 12y ago
  I know its not in plain sight but if you RTFA there's a diagram at the bottom that mentions them. http://sterlingcrispin.com/data-masks/map_of_attempts.png http://sterlingcrispin.com/data-masks/map_of_attempts.png
bane 12y ago
This is really cool. This work, along with the previous work of reverse engineering handwritting recognizers is important for us to better understand what our tools are actually doing. A lot of what happens inside these ML algorithms is not terribly well understood, even by the creators, this work gives us a really good insight into what the mind of a machine is actually thinking. When I was a teacher, I used to try to make sure students understood things going both ways. This is very important work in this respect. For people struggling to understand, it's a little like having a tool that uses automatically generated regular expressions to find things in text, except you don't know what regex the computer is actually using. For my name it could be as simply as "bane" or something crazy that a human would never think up, say b[a-z]+e. It matches my name, but it's interesting to explore what else it matches: "barserphursnatche" is also perfectly acceptable. Remember, you don't know what the computer is using to recognize things, so you have to simply probe: randomly generate strings, use some kind of genetic algorithm etc. In the end, you probably won't find b[a-z]+e, but you'll find some expression with a sizable intersection like b[a-y][b-z]*e. Once you have that you can use the expression as a generator and build a list of all the possible names it will match with something like Microsoft's Rex [1] What we find is that lots of the stuff the computer likely matches are not my name at all, or even look like a name (e.g. "baghuadsoadshasdguoasdughasdgaodsguafdghaufogafuhgafduoe") and it gives us insight into what it's doing. I think why I like this work is that it makes me think of a sci-fi short story: - A boy encounters an alien civilization - The boy befriends a robot from the civilization - They go off on some adventures together - During a scene of doubt in their relationship, during a stressful final adventure, the boy asks the robot what he thinks of him. The robot 3d-prints off one of these heads and gives it to the boy. 1 - http://research.microsoft.com/en-us/downloads/7f1d87be-f6d9-495d-a699-f12599cea030/ http://research.microsoft.com/en-us/downloads/7f1d87be-f6d9-...

drpgq 12y ago

As a researcher at a face recognition company, this is interesting, although the fact that algorithms change pretty frequently makes me think the work is quite ephemeral. Also I'm wondering if he's heard of liveness detection.

Houshalter 12y ago

Reminds me of this: https://www.youtube.com/watch?v=qS5HWBNvf9U https://www.youtube.com/watch?v=qS5HWBNvf9U

geekrax 12y ago

Sounds interesting. But the link received "HN hug of death".

lucb1e 12y ago

Cache: http://webcache.googleusercontent.com/search?q=cache:http://sterlingcrispin.com/data-masks.html http://webcache.googleusercontent.com/search?q=cache:http://... Edit: Original seems up again

coleca 12y ago

Since the site seems to be down, I'll mention that there was an article on Medium covering this artists' work a few days ago: https://medium.com/matter/this-is-what-your-face-looks-like-to-facebook-f771b3e11ed https://medium.com/matter/this-is-what-your-face-looks-like-... Seems like what he did was take Umass' Labeled Faces in the Wild (LFW) dataset, combine all the faces together and generate a composite rendering. Hardly a representation of "What your face looks like to Facebook". Articles like this do little to advance the discussion of the real concerns around facial recognition other than scare folks into an Orwellian vision of the future a la Hollywood and Minority Report. In addition to the work that the NTIA is doing around setting privacy standards for facial recognition applications (http://www.ntia.doc.gov/other-publication/2014/privacy-multistakeholder-process-facial-recognition-technology http://www.ntia.doc.gov/other-publication/2014/privacy-multi...) there are a lot of companies using the technology for good, but it is very difficult to get as much press and attention for those types of positive stories. For example, my own company, (http://www.kairos.com http://www.kairos.com) is working with some very brave and passionate guys at HelpingFaceless.com in India who are using facial recognition to help combat the growing problem of child trafficking and enslavement in India. See more about their story here: http://social.yourstory.com/2014/07/helping-faceless/ http://social.yourstory.com/2014/07/helping-faceless/

lbenes 12y ago
"Think of the children!" Whether it's for backdoors to your data in the cloud or facial recognition, protecting the children is ALWAYS what governments claim they need the tech for. Their track record is terrible, almost always using it for control and surveillance. I want to see cameras on every police officer in every city, before even considering more giving them more surveillance technology.
sterlingcrispin 12y ago
That's not what I did. I used a genetic algorithm with facial detection/recognition algorithms as fitness functions Recognition algorithms like eigenfaces, fischerfaces will reproduce one individual. More general detection algorithms like YEF realtime object detection as a fitness function will result in more general faces which represent some of the learned features from the LFW database. I'm not just combining images, there's lots of machine learning involved. Recheck my site in two weeks or so I'll be posting my MS thesis about the subject with more technical details. (Wrote this on my phone excuse the brevity)
- michael_storm 12y ago
  You might want to summarize that at the top of your post, then, if you're going to submit it to HN. When it contains such nuggets as: These masks are shadows of human beings as seen by the minds-eye of the machine-organism. These DATA-MASKS are animistic deities brought out of the algorithmic-spirit-world of the machine and into our material world, ready to tell us their secrets, or warn us of what’s to come. It's not surprising that HN viewers are confused about what you actually did. In any case, seems like a cool project, now that I know what it is :-).
  - sterlingcrispin 12y ago
    Thanks, but btw someone else submitted this to HN I hadn't planned on fully publishing the project until a few weeks from now when my thesis is due but it was posted to medium.com so I quickly put some documentation together There are more technical details at the bottom of that page in the form of diagrams

MisterNegative 12y ago

This does pose the question whether these mask work in real life. I bet their software uses the fact that they can control the lightning and picture quality. Creating a direct copy of a face seems easier for real life purposes anyway. Other than demonstrating the flaws of current facial recognition algorithms there does not seem to be any use for this. Anyway, pretty cool use of evolutionary algorithms and nice pictures.

lbotos 12y ago

Reminds me of this: http://www.zachblas.info/projects/facial-weaponization-suite/ http://www.zachblas.info/projects/facial-weaponization-suite... Which is also pretty interesting.

sterlingcrispin 12y ago
His project is pure poetics/politics and has no direct relationship to computer vision Its as much anti facial recognition as a paper bag
Houshalter 12y ago
That video is incredibly grating and slow to the point. And the mask just seems to be random faces mixed together, but they did it in such a poor way it just produced random garbage. This guy seems to be optimizing his mask to fool facial recognition.

drcomputer 12y ago

> Computer systems built to represent human identities have contained with them many ontological assumptions about what it is to be an individual and what personal identity is. These systems define the human as a “what” ie: that which can be measured, not as a “who” ie: our inner self. > If the state of the art in computer science can produce a unique feature that describes an individual as such, what good does that do the individual if this knowledge is only leveraged against them? > If private citizens personal information, social graphs, and communications are being analyzed then the results should be made available to said persons to empower rather than enslave them. This attitude has become popular in personal fitness but not in communications, biometric identity, or social networks. > “We see the world, not as it is, but as we are” - Talmud I love this. Really fantastic work.

sterlingcrispin 12y ago
Thank you please recheck my site in a few weeks for my full thesis

deleted 12y ago

[deleted]

mattxxx 12y ago

Cool tangential topic: http://www.onformative.com/lab/googlefaces/ http://www.onformative.com/lab/googlefaces/ ...Reminds of being up at night as a kid, and seeing faces in shadows, patterns on the wall, vvhatever

userbinator 12y ago

I was hoping that an article about "reverse engineering facial recognition, facial detection, and image correlation techniques in order to reveal how they represent human identity" would have at least one mention of eigenfaces: http://en.wikipedia.org/wiki/Eigenface http://en.wikipedia.org/wiki/Eigenface

sterlingcrispin 12y ago
I know its not in plain sight but if you RTFA there's a diagram at the bottom that mentions them. http://sterlingcrispin.com/data-masks/map_of_attempts.png http://sterlingcrispin.com/data-masks/map_of_attempts.png

bane 12y ago

This is really cool. This work, along with the previous work of reverse engineering handwritting recognizers is important for us to better understand what our tools are actually doing. A lot of what happens inside these ML algorithms is not terribly well understood, even by the creators, this work gives us a really good insight into what the mind of a machine is actually thinking. When I was a teacher, I used to try to make sure students understood things going both ways. This is very important work in this respect. For people struggling to understand, it's a little like having a tool that uses automatically generated regular expressions to find things in text, except you don't know what regex the computer is actually using. For my name it could be as simply as "bane" or something crazy that a human would never think up, say b[a-z]+e. It matches my name, but it's interesting to explore what else it matches: "barserphursnatche" is also perfectly acceptable. Remember, you don't know what the computer is using to recognize things, so you have to simply probe: randomly generate strings, use some kind of genetic algorithm etc. In the end, you probably won't find b[a-z]+e, but you'll find some expression with a sizable intersection like b[a-y][b-z]*e. Once you have that you can use the expression as a generator and build a list of all the possible names it will match with something like Microsoft's Rex [1] What we find is that lots of the stuff the computer likely matches are not my name at all, or even look like a name (e.g. "baghuadsoadshasdguoasdughasdgaodsguafdghaufogafuhgafduoe") and it gives us insight into what it's doing. I think why I like this work is that it makes me think of a sci-fi short story: - A boy encounters an alien civilization - The boy befriends a robot from the civilization - They go off on some adventures together - During a scene of doubt in their relationship, during a stressful final adventure, the boy asks the robot what he thinks of him. The robot 3d-prints off one of these heads and gives it to the boy. 1 - http://research.microsoft.com/en-us/downloads/7f1d87be-f6d9-495d-a699-f12599cea030/ http://research.microsoft.com/en-us/downloads/7f1d87be-f6d9-...