13 ms·
What a Deep Neural Network thinks about selfies
- danblick 11y agoThis is neat. I bet Facebook or OkCupid are sitting on all sorts of click data that could be used to develop tools for helping people make their photos look better. (Even if, personally, I can't wait for a cultural backlash against internet narcissism...) [Edit: Even better, he didn't use click data to train the model, just public likes.]
- visarga 11y agoThe idea to use a convnet to reframe the selfie is neat. Makes it 5% better. Also, if it can be run on the phone, it could possible warn people they are about to post a shitty selfie before they do.
- trhway 11y agolooking at the top 100 one can only wonder how Hollywood has figured it out well before mighty power of computer :)
- goodJobWalrus 11y agoFor me, this thing about having the top of your head cut from the picture is new. Who would have thought..
- falcolas 11y agoMakes a bit of sense, in combination with the "be female" advice, cutting off the forehead puts the center of the photograph closer to her cleavage, and typically shows off her entire chest.
- goodJobWalrus 11y agoCleavage does not feature a lot in the top 100 actually, but I'm half way there, in a sense that I'm a female. I'll definitely try the half-forehead thing next time!
- visarga 11y agoWe could try and see if the activation for good selfies comes from the cleavage or the eyes.
- lqdc13 11y agoI thought cutting off forehead happens when the target is closer to the camera, so it is more personal.
- mirimir 11y agoIt seems that eyes and mouth, and their alignment, matter most for female attractiveness.[0,1] [0] http://www.nbcnews.com/id/34482178/ns/health-skin_and_beauty/t/ideal-beauty-matter-millimeters-study-says/ http://www.nbcnews.com/id/34482178/ns/health-skin_and_beauty... [1] http://www.ncbi.nlm.nih.gov/pubmed/25836007 http://www.ncbi.nlm.nih.gov/pubmed/25836007
- lqdc13 11y agoA guide on how to take a good selfie that others will like: be female be blonde be attractive Incidentally, Christian Rudder did a really good "study" on the dating site pictures a few years ago: http://blog.okcupid.com/index.php/dont-be-ugly-by-accident/ http://blog.okcupid.com/index.php/dont-be-ugly-by-accident/
- steve_taylor 11y agoA better guide on how to take a selfie: Don't take a selfie.
- deleted 11y ago[deleted]
- tdaltonc 11y agoAnd, if you are female, chop off you forehead.
- bakhy 11y agoapparently, she should be white too.
- gus_massa 11y agoAlso, long hair in front of your shoulders (no ponytail).
- deleted 11y ago[deleted]
- spikels 11y agoDNN is a key technology of the future. I highly recommend the education program Professor Karpathy mentions at the end of this post. All are excellent and free.
- anunderachiever 11y agoI would like to see a deep dream selfie ... Feed it an initial picture (noise, clouds, a selfie) and then backwards manipulate the input to maximize the assessed quality of the "selfie". I guess that would look pretty funny.
- Tyr42 11y agoHe did run something like that for cropping. He showed his favourite two "rude" ones at the bottom, where the 'Net cropped out the face of the person taking the selfie.
- yoha 11y agoActually, he used random crops and selected the highest rated. A "deep dream selfie" would actually run the neural network in reverse so as to generate a completely different image.
- misiti3780 11y agoOne thing I always found interesting is Lecun is credited with developing covnets, but Hinton is apparently credited with scaling them and showing the world how great they are in the paper from 2012 - why was Hinton's group (Toronto) able to publish these ground breaking results before Lecun's group (NYU)
- pramodliv1 11y agoGeoff Hinton answers this question in episode 6 of the Talking Machines podcast. http://www.thetalkingmachines.com/blog/2015/3/13/how-machine-learning-got-where-it-is-and-the-future-of-the-field http://www.thetalkingmachines.com/blog/2015/3/13/how-machine... Geoff Hinton had grad students who wanted to work on the problem, but Yann LeCun didn't. "In about 2012, it should have been Yann's group, but Yann was unlucky, he didn't have a student who really wanted to do it. But we had a couple of students who wanted to do it and we took all of Yann's techniques and added some of our own."
- misiti3780 11y agointeresting - i took the course but did not notice that - thanks!
- Houshalter 11y agoIIRC the deep learning revolution started with pretraining and RBMs, which I believe Hinton invented.
- vonnik 11y agoI think it's less about the head getting chopped than about having "the head take up about 1/3 of the image," as Karpathy says. So what the net is learning is composition, or balance in an image, which is really cool. The rule of thirds is actually pretty well know to people in photography: https://en.wikipedia.org/wiki/Rule_of_thirds https://en.wikipedia.org/wiki/Rule_of_thirds (Our deep-learning framework http://deeplearning4j.org http://deeplearning4j.org missed his list, but it's got working convnets, too.)
- netheril96 11y agoOne caveat with these machine inspired knowledge: they are prone to error, probably more than humans, at least for now. For example, if you train a CNN directly with human faces, its recognition rate comes way below what a human is capable of. Only after you apply tons of handcrafted optimizations, which are mostly black art, will you get close to or surpass a human's capability. Without much domain specific tuning, an AI's insight is far from reliable.
- eivarv 11y agoWhat type of handcrafted optimizations are you talking about here? The state of the art I've read about* (deep CNNs) in later years rely more on generalized tricks like augmenting the training data (artificially inflating the data set), pre-training and fine-tuning, ReLU, regularization methods like dropout, etc. For anyone interested, here [1] are some benchmarks. * Late night here, but often in the vein of this [0] work. [0]: https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf https://www.cs.toronto.edu/~ranzato/publications/taigman_cvp... [1]: http://vis-www.cs.umass.edu/lfw/results.html http://vis-www.cs.umass.edu/lfw/results.html
- nl 11y agoThis is more wrong than right. The example is correct, but not for the reasons stated. Humans are very, very good at face recognition. However, CNNs are pretty close to human performance for face detection. Only after you apply tons of handcrafted optimizations, which are mostly black art, will you get close to or surpass a human's capability. Without much domain specific tuning, an AI's insight is far from reliable. This just isn't the case. Take the GoogLeNet or VGGNet papers, build the CNN as described using Caffe/whatever, train as described in the paper and you'll end up with something that is pretty much on par with human performance for categorizing ImageNet images. Take that same CNN architecture, and retrain it for another domain and it will perform roughly as well there too, for the task of categorizing into ~1K-10K image classes. This isn't domain specific tuning. It's domain specific training, which is very different (although collecting the data is a big job). Only after you apply tons of handcrafted optimizations, which are mostly black art, will you get close to or surpass a human's capability. For CNNs, this is pretty much entirely false.
- nightpool 11y ago>Be female. Women are consistently ranked higher than men. In particular, notice that there is not a single guy in the top 100. This sounds true, but it can't be the real reason—selfies are ranked relative to the other images by the same user. So unless users are taking a lot of #selfies of people of different genders, we can assume the dataset is already controlled for the gender of the person in the image, no? Unless there's some confounding factor at play, such as some demographic segment being more likely to optimize for good selfies occasionally but have boring feeds the rest of the time. would be super interesting, if the data is available, to normalize this by exposure. Of the people that saw an image, how many clicked "like"?
- lqdc13 11y agoYeah, female users probably post more pictures and also probably have more friends.
- nightpool 11y agoThis also would be controlled for by the tools the blog author used though—if a women has more friends, then they would also probably get more likes on all of the rest of their images. Not sure if posting more photos would drive the average up or down, but it would probably drive the "above the baseline" selfies in the same way.
- lqdc13 11y agoMore friends, but then the likes are not uniformly distributed with the increase of friends. Also more pictures means the "best picture" could be more of an outlier. So best pictures might rise further above baseline for that person. That is, top picture gets 1000 likes, but most pictures get zero. Sort of like Zipfian distribution of words. Anyway, these things are actually really hard to control for particularly because different types of friends/people have different effects on the likes. Now add to this cultural differences between countries/states/universities/rural-urban, etc. I think the best method that is actually practical was the one okcupid did at some point with "my best face" where you rate a bunch of people's pictures and they rate yours. Then you figure out what pictures are good from the data. If they kept the data for all these contests, it would be much easier to interpret in aggregate.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- JoachimS 11y agoA really good read. Good intro to ConvNets, a well designed and implemented test. Ad funny.
- JabavuAdams 11y agoHow to take a good selfie: don't be black or dark-skinned, unless you're a celebrity. How do we prevent our AIs from learning racism? EDIT> Informative article, BTW. A good read.
- apu 11y agoThis is an important point. People are thinking about it, and a lot of it will have to do with how the input data is gathered and curated.
- Lawtonfogle 11y agoIf a given question has an answer that is due to racism, the answer is still the answer. For example, if society has some underlying racism that factors into what it considers attractive, that doesn't change what it considers attractive. I don't think these algorithms are learning racism. They are only being blunt in revealing what already exists.
- JabavuAdams 11y ago> If a given question has an answer that is due to racism, the answer is still the answer. That's why it's important to be clear about the question. This ConvNet doesn't really answer the question "What makes a good selfie". It answers a much narrower and more complicated to state question. The absence of reflection in the system means that if it's used to answer a question that's superficially similar to the designer's intent, there's no way to reason around the bias in the training data. Imagine I'm a Canadian who trains an automated turret to classify friend / foe based on data from Afghanistan and Iraq. I've not trained the system to answer "Is this group of pixels a friend / foe", in the general sense. If the system is used outside the narrow context of its validity, say in Northern Ireland, or in a civilian Muslim neighbourhood in Paris, we should expect bad results. So you're right to point out that the racism is in the social context. But I'm arguing that we don't actually want a classifier to learn that if there's a good chance it'll be used in a way that discards or ignores that social context. Same as using an expert system outside its domain.
- thewhitetulip 11y agoWell, you don't need to ask a deep neural network to say that selfies are getting stupid daily with teens sticking their tongues out
- visarga 11y agoBEEP BEEP. Bad selfie detected. You run the risk of making a fool of yourself! BEEP BEEP
- RealityVoid 11y agoIt seems this neural network has a sense of humor if you look at the "Finding the Optimal Crop for a selfie" area. You can see it optimized the last selfie by cropping the face fully out of the picture.. :))
- amai 11y agoI have seen similar results before: https://medium.com/the-physics-arxiv-blog/the-algorithm-that-sees-beauty-in-photographic-portraits-435ab8064646 https://medium.com/the-physics-arxiv-blog/the-algorithm-that...