9 ms·
Open Sourcing a Deep Learning Solution for Detecting NSFW Images
- deleted 10y ago[deleted]
- BinaryIdiot 10y agoThe Yahoo blog[1] post is far more interesting than this techcrunch "article". Suggest changing URL to the Yahoo Blog please. [1] https://yahooeng.tumblr.com/post/151148689421/open-sourcing-a-deep-learning-solution-for https://yahooeng.tumblr.com/post/151148689421/open-sourcing-...
- sctb 10y agoOK, we've updated the link from https://techcrunch.com/2016/09/30/yahoo-open-sources-its-porn-detecting-neural-network/ https://techcrunch.com/2016/09/30/yahoo-open-sources-its-por....
- yk 10y agoI would suggest, that the link should go to Yahoo's blog post https://yahooeng.tumblr.com/post/151148689421/open-sourcing-a-deep-learning-solution-for https://yahooeng.tumblr.com/post/151148689421/open-sourcing-... which contains some technical details. (And furthermore, I guess the HN crowd has enough Internet experience to come up with stupid jokes of their own design.)
- brianwawok 10y agoSo can it be reversed to become the ultimate porn-finding neural network?
- empath75 10y agoYou can use the nsfw subreddits as a massive dataset for categorizing porn.
- Forge36 10y agoNot all of them. gonwild[1] is a just a massive polygon pun subreddit [1] https://www.reddit.com/r/gonwild https://www.reddit.com/r/gonwild
- crobertsbmw 10y agoI was a little nervous at first because it asked me to conform my age and everything but it looks like you are right. Just a bunch of math geeks, haha.
- pbhjpbhj 10y agoVery edgy stuff!
- BinaryIdiot 10y agoMy first thought as well. Integrating with this some web crawlers that test images as it goes so it keeps going down the path of maximum NSFW websites. Though at the same time I fear the type of pornography it may come across.
- nerfhammer 10y agoUsing it as an adversarial trainer for another net it could generate infinite amounts of porn.
- ryandrake 10y agoThe Internet seems to be accomplishing this on its own just fine already.
- deleted 10y ago[deleted]
- XCSme 10y agoWhen you said "reversed" I thought about it being a porn-generative neural network. Just enter your favorite keywords and an unique, tailored to your needs scene will be generated just for you!
- zardo 10y agoAs long as what you need is a nightmarish version of the requested scene. What happens here with generating illegal content? If you put a public text->image gan up, and someone uses it to generate child porn, are you responsible? How could you make sure it couldn't?
- BinaryIdiot 10y agoI would imagine if you're generating then no real people were part of its creation therefore it would be legal. If I remember correctly cartoons of children having sex is not illegal (in the United States as far as I know). Though that then raises the question: when happens when it can be generated so realistically that it looks indistinguishable from the real thing? Would it still be treated like cartoons? How could you prove one way or the other? Lots of questions here.
- voxic11 10y agoIn the United States under the 2003 PROTECT act they are actually illegal but only if they are "obscene" as obscene speech is not protected by the first Amendment. >Prohibits drawings, sculptures, and pictures of such drawings and sculptures depicting minors in actions or situations that meet the Miller test of being obscene, OR are engaged in sex acts that are deemed to meet the same obscene condition. The law does not explicitly state that images of fictional beings who appear to be under 18 engaged in sexual acts that are not deemed to be obscene are rendered illegal in and of their own condition (illustration of sex of fictional minors). Maximum sentence of 5 years for possession, 10 years for distribution. Interestingly the same act does make all computer generated child pornography which is "virtually indistinguishable from that of a minor engaging in sexually explicit conduct" with no requirement that it be obscene, I don't think that has been tested in court yet. https://en.m.wikipedia.org/wiki/PROTECT_Act_of_2003 https://en.m.wikipedia.org/wiki/PROTECT_Act_of_2003
- Something1234 10y agohttp://www.scp-wiki.net/scp-1004 http://www.scp-wiki.net/scp-1004
- mhurron 10y agoThat's what Bing is for.
- Grishnakh 10y agoAnd in fact, that's the only thing Bing is good for. It is really good at it, though.
- jomamaxx 10y ago"So can it be reversed to become the ultimate porn-finding neural network?" They have that, it's called 'Google' :)
- nostrademons 10y agoMy first thought as well, but it turns out that finding porn on the Internet is just not that much of a problem.
- Esau 10y agoNow I can replace Bing!
- lifeisstillgood 10y agoMy first thought was from years ago, when I was pitching open source forensic services to London police (did not get far, bad Salesman that I am) Cataloging, categorising pornography seized is a nasty job and one that cops across the planet might do better with good common OSS tools. Hopefully this will help
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- inlined 10y agoForgive my ignorance of ML but the last bit: "you'll need your own porn to train on" confused me. Does this mean that they're just exposing the rough topology of their neutral net (eg depth) and not the actual weights between nodes? I'm curious to learn from an ML expert how much this actually offers.
- zardo 10y agoIt looks like they are sharing the trained network, but they aren't sharing the training data set.
- daturkel 10y agoThe training set is almost certainly composed of copyrighted material.
- Asooka 10y agoInteresting thought - doesn't every single porn producer now have a valid copyright claim on the trained network? I don't see how you can argue this isn't a derivative work based on the movies they produced.
- x1798DE 10y agoI don't think it's a derivative work just because one of the inputs is copyrighted. I think it's more descriptive than derivative. Content producers don't generally own copyright in critics' descriptions of their movies or of plot summaries, even though their copyrighted material is a necessary input to the description's creation.
- onli 10y agoAlso, good luck trying to prove that your film was used to train this network.
- 10y ago
- bahro 10y agoI should update my sexy map finder: http://exclav.es/2016/05/20/sexy-maps/ http://exclav.es/2016/05/20/sexy-maps/
- the_af 10y agoThose are some hot map pictures! I don't understand how Google's algorithm can be misled into finding sexiness in those. I imagine it has something to do with skin tones or flesh colors, but then what about the high-contrast patchwork of green and brown fields Google finds "likely to contain adult content"? That's totally puzzling. The confusion with medical images is way more understandable. If you squint, you can almost imagine those are pics of skin cancer or lesions. Oddly enough, I even see violence in the "violent" picture. In an abstract, Rorschach Test sort of way. Well done, Google!
- re 10y ago> I don't understand how Google's algorithm can be misled into finding sexiness in those. I'm reminded of a paper for which the authors generated different pictures of static that fooled neural network image classifiers into confidently identifying them as different objects: https://arxiv.org/abs/1412.1897 https://arxiv.org/abs/1412.1897 Wired summary: https://www.wired.com/2015/01/simple-pictures-state-art-ai-still-cant-recognize/ https://www.wired.com/2015/01/simple-pictures-state-art-ai-s... > Computer vision and human vision are nothing alike. And yet, since it increasingly relies on neural networks that teach themselves to see, we’re not sure precisely how computer vision differs from our own. As Jeff Clune, one of the researchers who conducted the study, puts it, when it comes to AI, “we can get the results without knowing how we’re getting those results.”
- andydoan 10y agoTangent: is your blog pure HTML/CSS? I really like the look of it and have been wanting to start something just like yours.
- Secretmapper 10y agoNot OP, but looks like it's built on Jekyll. If you're not familiar, it's a Static Site Generator. It generates static html files from your content (generally markdown files). There are other SSGs, I use Hugo myself (http://gohugo.io http://gohugo.io, sample of my blog: http://arianv.com/ http://arianv.com/). I like Hugo cause it's probably the fastest SSG (every time you create a post/change content, you're remaking your entire site from scratch. If you have lots of posts - this adds up!) but Jekyll is the most popular and has great tooling. Hope that helps!
- eggoa 10y agoI wonder if anyone at Yahoo tried using this to "deconvolute" noise into Cronenberg nightmare porn?
- zfedoran 10y agoHas anyone tried taking the features that are learned at the various layers of a neural net and feeding them into something like this: https://news.ycombinator.com/item?id=12612246 https://news.ycombinator.com/item?id=12612246? I imagine we would get some really interesting images back...
- NicoJuicy 10y agoWe are not releasing the training images or other details due to the nature of the data, but instead we open source the output model which can be used for classification by a developer. I'm guessing the one who had to input the data/images had a fun time at work :p
- bazzargh 10y agoI used to work in a company which had a division doing manual image classification next door. Not a fun time at all, the people who worked there regularly burned out on relentlessly seeing terrible things.
- davegauer 10y agoI've often thought that it would be even more helpful to automatically filter violent images. Particularly to spare humans from having to be the filters (and "relentlessly see terrible things"). However, I imagine that's far more difficult to accomplish. How do you detect graphic violence? Looking for blood isn't going to cut it. Also, I can't imagine how you'd separate the fictional from the real - I can watch horror movies with realistic special effects all day, but real violence/mutilation/death bothers me deeply.
- packetslave 10y agolikewise the folks at YouTube that have to evaluate potential CP / gore / etc. all day long.
- Joof 10y agoOr they just turned safe search off and collected everything that doesn't also turn up on moderate search.
- wildpeaks 10y agoDirect link to Github: https://github.com/yahoo/open_nsfw https://github.com/yahoo/open_nsfw
- TrueDuality 10y agoSit back grab some popcorn. Lets see how long it takes people to start running data backwards to get new original porn.
- pluma 10y agoI see you never have seen what kind of images that tends to generate. Or you're really into R'lyehian porn.
- krapp 10y ago>Or you're really into R'lyehian porn. It's called tentacle hentai...
- venomsnake 10y agoCthulhu has nice tentacles. So I guess it will be run of the mill hentai.
- pluma 10y agoYou have no idea. Try googling for "deep dream porn".
- devonkim 10y agoThis was done before and posted to HN a while ago somewhat (albeit not in the same sense like DeepDream) NSFW http://blog.clarifai.com/what-convolutional-neural-networks-see-at-when-they-see-nudity/ http://blog.clarifai.com/what-convolutional-neural-networks-...
- echelon 10y agoCan you run deep dream on this? That would be quite fascinating.
- RandomInteger4 10y agoI think you misspelled "horrifying". I can only imagine that producing something akin to the human centipede ... "Infinite Girls, One Cup" ...
- deleted 10y ago[deleted]
- zuzun 10y agoWith access to Flickr and Tumblr it must have been very easy to create a huge training set for such a task.
- Happpy 10y agoCould this work on mobile to detect 18+ content in images or video? Or would it be a trained library of 50mb+?
- c3534l 10y agoMy first thought: would probably be very useful for sites to crack down on inappropriate content. My second thought: I could probably use this to find porn in unexpected places via a webscraping Python program.
- Bud 10y agoSo this is what Yahoo was up to for the last 10 years, instead of building any sort of security, keeping Yahoo Messenger working properly, or anything else of value? Heckuva job, Yahoo.
- m-i-l 10y agoGood to see they've automated this (beyond the initial classification of training data). In the early days of the web, such filters were typically based on manually maintained lists of sites. I actually met someone at a party once whose full-time job was to surf for porn, to maintain the filter for a provider of IT services to schools (he worked for a company now called RM Education). He said it was his ideal job for the first few days, but soon grew tiresome (note that back in those days there wasn't really any extremely objectionable material on the web).
- SloopJon 10y agoAnyone else see the irony in acknowledging that NFSW is subjective and contextual, but assuming that pornographic images are not?
- CodeMage 10y agoThere's no definition of "ironic" that I can think of applying to that. It's like saying that "beautiful painting" is subjective and contextual, but "still life painting" isn't. It just happens that pornography is almost[1] always considered NFSW, but again, I can't see how that is ironic. [1] Almost, because porn is SFW when your work involves porn.
- SloopJon 10y agoI think it's easier to agree that a picture of a naked person is NSFW than that it's pornographic.
- robbrown451 10y agoI didn't see where they said that pornographic images are not subjective and contextual. In fact it seemed like they were using "NSFW" and "pornographic" almost interchangeably, while acknowledging that the current implementation doesn't deal with violent images etc. For instance, if you read it like this, it still makes sense (I replaced NSFW with pornographic): Disclaimer: The definition of pornographic is subjective and contextual. This model is a general purpose reference model, which can be used for the preliminary filtering of pornographic images. We do not provide guarantees of accuracy of output, rather we make this available for developers to explore and enhance as an open source project.
- CompanionCuube 10y agoHas anyone run this NN on the censored Facebook image?
- matheweis 10y agois this just the network or is it a fully trained model? The TechCrunch article suggests the former but the yahoo post the latter...
- tedd4u 10y agoIt's the trained model. You can use it out of the box or refine it to tailor to your environment.
- matheweis 10y agoNice, was on phone only at the time and not able to dig deeper. Took a couple hours to get it all up and running but indeed it works, and not half badly at that! This is obviously a way cheaper alternative to https://sightengine.com https://sightengine.com or http://imagevision.com http://imagevision.com Kudos to Yahoo for releasing this!
- joshmn 10y agoI'm not a deep learning person whatsoever, but I do have an interesting use case that I won't disclose publicly: Is there a way to build this, and output detections based on the, ugh, object it has detected? e.g. penis 0.94 vagina 0.01
- pjreddie 10y agoYes. You would have to have a large training set with these labels but it would be pretty straightforward to train. You would probably want a tagging model not a classifier because there could be multiple objects of interest in the same image. If you get me the training data I could train a model for you pretty quickly.
- bbctol 10y agoAnd it would be an... interesting job to tag the training set. Although for higher level content, I suppose lots of porn videos have very specific category tags that could be an interesting data set to play with. Uh, to analyze.
- inlined 10y agoIirc you can use ML to learn the tags as well. They tend to be in text surrounding the media
- joshmn 10y ago> large training set Is the training set itself large, or are you meaning a... large training set? (sorry I couldn't help myself)
- joshvm 10y agoWhat's the current ML pet method for multi-label image classification? It seems like you could string together a bunch of individual classifiers e.g. "Scene contains dog", "Scene contains cat", but is there an efficient (and effective) way of doing it in one go? Does it significantly increase the complexity of the network? I would imagine a cat detector would be far simpler than a cat and/or dog detector.
- johnnyo 10y agoI'll bet this would be a good tool for sysadmins or network administrators to run against their network and see what it finds.
- deleted 10y ago[deleted]
- ganwar 10y agoTo be precise they are only releasing the already trained model. The associated dataset is not being made public. Thus, it is meant to be for off the shelf use rather than being able to tinker with the network to produce nuanced results.
- mattthebaker 10y agoOr they just don't want to be distributing gigabytes of porn... most of which is probably under copyright. Making the data set available and whether you can tinker with or retrain it are very different things.
- Dim25 10y agoanother work in this field: "Adult video content detection using Machine Learning Techniques" PDF: http://colorlab.no/content/download/37238/470343/file/VictorTorres_MasterThesis.pdf http://colorlab.no/content/download/37238/470343/file/Victor...
- KennyCason 10y agoI literally just started working on this problem 2 hours ago >_<
- darklajid 10y agoThey acknowledge that NSFW (or pornographic) is hard to define, a la 'I recognize it if I see it'. But looking at the meager 3 sample images I'm confused about the scoring already. Why is the one in the middle scoring the highest? The question is an honest one. The two rightmost images seem to be interchangeable to me and are ~boring~: People at the beach. Is this network therefor already trained to include the biases of the creators?
- plingamp 10y agoAll ML networks are inherently biased towards its creators. My colleague recently described this issue to me as the "Old, white, male" problem. This is why most voice recognition services drastically fail when they are shown foreign accents.
- vidarh 10y ago> This is why most voice recognition services drastically fail when they are shown foreign accents As someone with a broad Norwegian accent: This has gotten massively better over the last few years. Not that long ago, my local cinema chain started using voice recognition to discriminate between a list of city names, and it would consistently think I said "Birmingham" when I said "London" (!). These days, both my Amazon Fire and the Youtube app will correctly recognise most things I throw at it, including e.g. names of random Youtube channels that bear no relation to real English words. It's by no means perfect, but it's getting there. In relation to the "old, white, male" problem (well, I do somewhat fit that), presumably because these systems are now finally trained on huge and varied data sets.
- cvwright 10y agoReminds me of this post from hackerfactor where he describes his own porn filter based on pHash. http://www.hackerfactor.com/blog/index.php?/archives/529-Kind-of-Like-That.html http://www.hackerfactor.com/blog/index.php?/archives/529-Kin... It'd be interesting to see a direct comparison of the two. Off the cuff, I'd expect the deep neural network to be more accurate and better at generalizing, but much more expensive to train.
- cft 10y agoI hope this is ported to TensorFlow soon!
- chadscira 10y agoAwesome! I have been using nude.js to do this ( http://s.codepen.io/icodeforlove/debug/gMrEKV http://s.codepen.io/icodeforlove/debug/gMrEKV ), which is hit or miss.
- Joof 10y agoAre there any other fairly basic image recognition problems that people want? I'd be happy to provide as long as a dataset is easy to collect.
- slowmovintarget 10y agoI think I'll pass on browsing the the deep dream visualizations for this.
- patrickaljord 10y agoI wonder what would happen if we stopped firing people for watching NSFW images. I mean bosses look at NSFW images all the time and it sounds like a shallow reason to fire someone.
- davidgerard 10y agoCreates a hostile work environment, however.
- zwindl 10y agoThat's it! That's what I'm looking for.
- askew 10y agoInteresting that the photo of two women on the beach is given a higher NSFW rating than the photo of a man on the beach.
- rasz_pl 10y agooh silly Americans, its just tits
- prirun 10y agoAren't there more important problems to work on than worrying about someone looking at naked people? This is just what we need: more effort spent on censoring and controlling people.
- myth_buster 10y agoWon't you say that preventing a kid from accidentally viewing porn while searching images with `Safe Search: On` an important problem?
- prirun 10y agoWell, considering the other shit they look at and participate in, like video games with people killing each other and blood spattering everywhere, I'd rather they were viewing naked, non-violent people.