10 ms·

What is the best way to programatically detect porn images? (2009)

142 points by romain_g 13y ago

quarterto 13y ago
Google reverse image search can come up with a search likely to return the given image. Perhaps this can be used for porn classification.
eksith 13y ago
To this day, I believe the best method for picking out these images is a human censor (with appropriate, company provided, counseling afterward). Edit: No shortage of stock image reviewer jobs https://google.com/search?hl=en&q=%22image%20reviewer%22 https://google.com/search?hl=en&q=%22image%20reviewer%22 I'm trying to find an interview of one of these people describing what it's like on the other end. It wasn't a pleasant story. These folks are employed by the likes of Facebook, Photobucket etc... Most are outsourced, obviously, and they all have very high turnover.
- RossM 13y ago
  I seem to remember reading an article about people doing this at Google(?) in a pretty poor state. Edit: I think it was this one: http://www.buzzfeed.com/reyhan/tech-confessional-the-googler-who-looks-at-the-wo http://www.buzzfeed.com/reyhan/tech-confessional-the-googler...
  - eksith 13y ago
    Yep! That's the one. Thanks. A pretty unpleasant job no matter what angle you look at it. I remember this bit the most : "Google covered one session with a government-appointed therapist — and encouraged me to go out and get my own therapy after I left." This is the down side to 'abstracting away' the dirty end of filtering. I'm looking forward to a day when this can be properly automated, but, considering the ever-changing nature of erotica to begin with, I don't see that happening any time soon.
    - XorNot 13y ago
      Developing a strong AI which can do this without going insane itself is going to be the robo-pysche challenge of the future.
mixmax 13y ago
detecting all porn seems to be an almost impossible problem. Many kinds of advanced porn (BDSM, etc.) don't have much skin - often the actors are in latex, tied up, or whatever. It's obviously porn when you see it, but detecting it seems incredibly hard. Detecting smurf-porn(1) (yes that's a thing...) is even harder since all the actors are blue. http://pinporngifs.blogspot.dk/2012/09/smurfs-porn.html?zx=7ac31f5871d5a788 http://pinporngifs.blogspot.dk/2012/09/smurfs-porn.html?zx=7... - obviously very NSFW, but quite funny.
- ma2rten 13y ago
  It is possible high accuracy if you use machine learning and a sufficiently large training set. That said said even humans sometimes don't agree is something is porn or not.
  - rfergie 13y ago
    Just saying "machine learning" is not very useful here. What machine learning techniques work well in this case and what are the major pitfalls? Then you can convince me that a "sufficiently large training set" exists and is smaller than "all the images on the internet".
    - ma2rten 13y ago
      See my other post on this page. I would argue that this less difficult than you average image classification problem. Just have a look at what kind of challenges image classification can tackle, picking the correct class out of 1000s of classes. Porn is normally well-lit, the subject is at the center of the image, etc. The main difficulty is to define what is porn and what is not ... It's easy see the difference between porn and pictures of bicycles. But how about porn and artistic nudity? You see it's actually a scale, but you are trying to make a binary decision. Another problem (at least with the method I explained below) is that portraits sometimes get misclassified. Maybe it could help to integrate face detection. I'd suspect that more recent models would not have this problem (e.g. ones that take not only local features into account). Other times it makes mistakes, where you think "why on earth would think this is porn". Again combing different method should help to eliminate those outliers. Outliers are also a problem, e.g. black and white pictures. Again an ensemble of different models (e.g. one color independent one) might help. Niches are not really a big problem. BSDM porn is as far as I have seen the only niche of porn that really different visually.
bachback 13y ago
machine learning? because you also want to filter cats.
- rspeer 13y ago
  Machine learning is a field of research, not a magical incantation that you say to solve everything.
asolove 13y ago
Amazon Mechanical Turk has an adult-content marker specifically for this purpose. Lots of people have done the paperwork to qualify for adult-content jobs and the cost of having humans do it at scale is very low: https://requester.mturk.com/help/faq#can_explicit_offensive https://requester.mturk.com/help/faq#can_explicit_offensive Source: I helped implement a MT job to filter adult content for a large hosting company.
- phorese 13y ago
  So i can literally get paid for looking at porn. Huh, who knew... :)
  - selmnoo 13y ago
    At the cost of your mental health, sure. I recall reading an article about the human workers who had this job at Google... they had crap benefits, crap pay, and no mental healthcare. As a result a lot of the people in the field had depression and other mental issues.
    - _mulder_ 13y ago
      Source?.... or any hints as to what the article was called. Sounds interesting.
    - romain_g 13y ago
      Would it be this one? http://www.buzzfeed.com/reyhan/tech-confessional-the-googler-who-looks-at-the-wo http://www.buzzfeed.com/reyhan/tech-confessional-the-googler...
    - gadders 13y ago
      And RSI as well I would imagine.
    - sharkweek 13y ago
      Friend was a contractor on the YouTube filter team -- they lasted about 9 months before it became too much and had to leave. There was absolutely no reward in showing up to work, and some of the things they saw have likely scarred their memories forever
jmngomes 13y ago
I came across nude.js (http://www.patrick-wied.at/static/nudejs/ http://www.patrick-wied.at/static/nudejs/) when researching for a social network project, seems quite nice and is Javascript based.
hugofirth 13y ago
Whilst I agree that programmatically eliminating porn images is a very hard problem. Programmatically filtering porn websites might be easier, beyond just a simple key word search and whitelist. If you assume that porn tends to cluster, rather than exist in isolation, then a crawl of other images on the source pages , applying computer vision techniques, should allow you to block pages that score above a threshold number of positive results (thus accounting for inaccuracy and false positives).
- VLM 13y ago
  "score above a threshold number of positive results" How about social scoring? A normal (or even a weirdo) teenage boy would spend less than a second examining my ugly old profile pix, but after ten or so of your known teen male users are detected to spend 5 minutes at a time, a couple times a day, closely studying a suspected profile pix, I think you can safely conclude that pix is not a pix of me and then flag / censor / whatever it for the next 10K users.
  - hugofirth 13y ago
    The graph theorist in me is rubbing my hands in glee at the thought of seeing if you could extrapolate out that approach to catch a broader range of offensive imagery through relationships and usage patterns.
    - XorNot 13y ago
      Key problem: profile pictures are everywhere on a website by definition. People look at them for over an hour at a time depending how your site is laid out. If you wanted to spot shock imagery it's easier - study navigation aways or rapid scrolling.
VLM 13y ago
This is probably going to get downvoted, but if lots of people are not overzealous puritans and want some skin, the best overall system design that maximizes happiness and profit is probably sharding into puritanweirdos.example.com with no skin showing between toes and top of turtleneck (edited to add no pokies either) and normalpeople.example.com with 99% of the human race The best solution to a problem involving computers is sometimes computer related, but sometimes is social. The puritans are never going to get along with the normal people anyway, so its not like sharding them is going to hurt. Another way to hack the system is not to hire or accept holier than thou puritans. Personality doesn't mesh with the team, doesn't fit culture, etc. You have to draw the line somewhere, and weirdos on either end should get cut, so no CP or animals at one extreme, and no holy rollers on the other extreme. The final social hack is its kind of like dealing with bullies via appeasement. So they're blocking reasonable stuff today, tomorrow they want to block all women not wearing burkhas or depictions of women damaging their ovaries by driving. Appeasing bullies never really works in the long run, so why bother starting. "If you claim not to like it, or at least enjoy telling everyone else repeatedly how you claim not to like it, stop looking at it so much, case closed"
- Shivetya 13y ago
  Harassment laws/lawyers would find you their wet dream. The rule with pictures in any work environment is to err on the side of extreme caution. More than likely your not going to find out they don't mesh with the team till they quit or are fired. If your lucky its not followed by a lawsuit. See, its not their job to not be offended, it is your job to offer a harassment free work environment. Whom you are catering too depends on what is PC or not at the time, fortunately its pretty easy to determine whose whims to cater to or not
  - jiggy2011 13y ago
    I'm not sure it is that easy, even HN discussions on the topic don't come to any particular conclusion and that is within a fairly narrow range of people. In an old job I had the task of cleaning spam off a forum, this meant looking at rather a lot of porn but it would never have occurred to me to sue on that basis, yet that is still a job that needs to be done.
cvshepherd 13y ago
i've read about these skin tone filters more than once now, and i'm really curious to find out if these take all possible skin tones into consideration (i can't really imagine that they do. escpecially dark skin tones would get way too many false positives, would they not?).
wehadfun 13y ago
Probably the easiest way is with motion and sound. Checking for skin would be hard depending on the type of content as mixmax pointed out
VLM 13y ago
Nobody has discussed i18n and l10n issues? What passes for pr0n in SF is a bit different than tx.us and thats different from .eu and from .sa (sa is saudi arabia not south africa, although they've probably got some interesting cultural norms too) If you're trying for "must not offend any human being on the planet" then you've got an AI problem that exceeds even my own human intelligence problem to figure out. Especially when it extends past pr0n and into stuff like satire, is that just some dudes weird self portrait, or a satire of the prophet, and are you qualified to figure it out?
deleted 13y ago
[deleted]
betterunix 13y ago
How about a picture of a woman's breasts? What about an erect penis? Sounds like porn, but you might also see these things in the context of health-related pictures or some other educational material. The classic problem of trying to filter pornography is trying to separate it from information about human bodies. I suspect that doing this with images will be even harder than doing it with text.
- Zimahl 13y ago
  Definitely true. Facebook had a dust-up when a woman posted a topless photo of herself after she had had a double mastectomy. That said, not all sites are like Facebook and we aren't talking about filtering all the images on the internet, just ones on specific sites. One example I can think of is that a forum for a sports team might not want NSFW pictures posted as it would be irrelevant.
Theodores 13y ago
In The Olden Pre-Digital Days porn was either in print or on a television screen. Back then (we are talking two whole decades ago) experienced broadcast engineers could instantly spot porn just by catching a look at an oscilloscope (of which there were usually many in a machine room). Notionally the oscilloscope would be there to show that the luminance and chroma was okay in the signal (i.e. it could be broadcast over the airwaves to look as intended at the other end - PAL/NTSC), however, porn and anything likely to be porn had a distinctive pattern on the oscilloscope screen. Should porn be suspected then the source material would obviously be patched through to a monitor 'just in case'. Note that the oscilloscope was analog and that the image would be changing 25/30 times a second. Also, back then there were not so many false positives on broadcast TV, e.g. pop videos etc. where today's audience deems them artful rather than porn. If I had to solve the problem programatically I would find a retired broadcast engineer and start from there, with what can be learned from a 'scope.
- daurnimator 13y ago
  It's probably related to the type of cameras available in the day rather than anything else...
racbart 13y ago
Wouldn't testing for skin colors produce far too many false positives to be useful? All these beach photos, fashion lingerie photos, even close portraits. And how about half of music stars these days who seem to try to never get caught more clothed than half naked? Nudity != porn and certainly half-nudity != porn. I'd rather go for pattern recognition. There's lot of image recognition software these days that can distinguish the Eiffel Tower from the Statue of Liberty and it might be useful to detect certain body parts and certain body configurations (for these shots that don't contain any private body part but there are two bodies in an unambiguous configuration).
- betterunix 13y ago
  "detect certain body parts" When I was a kid, we had a firewall at school that tried to filter pornography by doing something similar with text. Doing research on breast cancer turned out to be rather tricky. So let's say you try to detect certain body parts. Now you have someone who wants to know more about their body, but you are classifying images from medical / health articles as pornography. "certain body configurations" So now instead of having trouble reading about my own body, I will have trouble looking at certain martial arts photos: https://upload.wikimedia.org/wikipedia/commons/1/14/BostonKetsugoJujutsu.jpg https://upload.wikimedia.org/wikipedia/commons/1/14/BostonKe... I am not saying these are unsolvable problems, but they are certainly hard problems. Even using humans to filter images tends to result in outrageous false positives sometimes: http://abcnews.go.com/Health/breastfeeding-advocates-hold-facebook-protest/story?id=15530012 http://abcnews.go.com/Health/breastfeeding-advocates-hold-fa...
  - racbart 13y ago
    You are correct and I'm not saying that I described the Holy Grail of detecting porn in a single paragraph. I'm just pointing to another direction. No solution to a very complex problem could be one-dimensional. Combining several different tests might lead to a solution. I.e. these jujitsu photos should not even be detected as “certain body configurations” as people there are fully clothed and there's not much actual bodies seen in the picture (so mentioned skin color definitely should come to play when detecting wether you see a body or not). At the end of the day I doubt there could be a fully bulletproof and always correct solution using current state of tech. But you need to factor much more than just skin color if you try to build an automated solution to this problem.
_mulder_ 13y ago
Here's an idea... Develop a bot to trawl NSFW sites and hash each image (combined with the 'skin detecting' algorithms detailed previously). Then compare the user uploaded image hash with those in the NSFW database. This technique relies on the assumption that NSFW images that are spammed onto social media sites will use images that already exist on NSFW sites (or are very similar to). Then it simply becomes a case of pattern recognition, much like SoundHound for audio, or Google Image search. It wouldn't reliably detect 'original' NSFW material, but given enough cock shots as source material, it could probably find a common pattern over time. edit: I've just noticed rfusca in the OP suggests a similar method
- pa5tabear 13y ago
  How do you program that sort of thing? Do you have to tell it what shapes/colors to look for? Or do a combination of overall image similar combined with localized image similarity and portion by portion image comparison?
  - viraptor 13y ago
    Maybe recognising the furniture in the background would work too ;) I remember there was a website/catalog of IKEA furniture somewhere made using NSFW photos.
  - _mulder_ 13y ago
    Well Image Hashing is distinct from normal MD5 hashing as the hash does consider similarity of colour, etc. so it's not purely binary. A Google search produced a library called pHash.org that might do something similar.
- Fomite 13y ago
  "This multi-TB disk array labeled 'Porn' has a legitimate business use!"
- lifeformed 13y ago
  Is it possible to hash an image so that you can partially match it with subsets of that image (like cropped regions or resizes)? Or a slight modification of that image (colors shifted, image flipped, etc).
dschiptsov 13y ago
by filename ,)
primaryobjects 13y ago
This is a classic categorical problem for machine learning. I'm surprised so many suggestions have involved formulating some sort of clever algorithm like skin detection, colors, etc. You could certainly use one of those for a baseline, but I'd bet machine learning would out-score most human-derived algorithms. Take a look at the scores for classifying dogs vs cats with 97% accuracy http://www.kaggle.com/c/dogs-vs-cats/leaderboard http://www.kaggle.com/c/dogs-vs-cats/leaderboard. You could use a technique of digitizing the image pixels and feeding to a learning algorithm, similar to http://www.primaryobjects.com/CMS/Article154.aspx http://www.primaryobjects.com/CMS/Article154.aspx.
adorable 13y ago
I have developed an algorithm to detect such images, based on several articles published by research teams all over the world (it's incredible to see how many teams have tried to solve this problem!). I found out that no single technique works great. If you want an efficient algorithm, you probably have to blend different ideas and compute a "nudity score" for each image. That's at least what I do. I'd be happy to discuss how it works. Here are a few techniques used: - color recognition (as discussed in other comments) - haar-wavelets to detect specific shapes (that's what Facebook and others use to detect faces for example) - texture recognition (skin and wood may have the same colors but not the same texture) - shape/contour recognition (machine learning of course) - matching with a growing database of NSFW images The algorithm is open for test here: http://sightengine.com http://sightengine.com It works OK right now but once version 2 is out it should really be great.
nailer 13y ago
I used pifilter (now WeSEE:Filter , http://corp.wesee.com/products/filter/ http://corp.wesee.com/products/filter/) for a production, realtime, anonymous confession site (imeveryone.com) in 2010. It cost, IIRC, a tenth of a cent per image URL. Rather than being based on skin tone, it was created based on algos to specifically identify labia, anuses, penises, etc. REST API: send a URL, get back a yes/no/maybe. You decided what to do with the maybes. My experience: - Before launch, I tested it with 4chan b as a feed, and was able to produce a mostly clean version of b with the exception of cartoon imagery. - It could catch most of the stuff people tried to post to the site. Small breasted women (being that breasts are considered 'adult' in the US) was the only thing that would get through and wasn't a huge concern. Completely unmaintained public hair (as revealing as a black bikini) would also get through. - Since people didn't know what I was testing with they didn't work around it (so nobody tried posting drawings or cartoons), but I imagine eg a photo of a prolapse might not trigger the anus detection as the shape would be too different. - pifilter erred on the side of false negative, but one notable false positive: a pastrami sandwich.
- cobrausn 13y ago
  I don't remember where I read this, but someone once recommended having a bot that posts the image to 4chan b feed and monitor the views and replies. Since porn images on b usually had somewhat predictable replies, it would be a good way to augment another technique and prevent false positives. Funny idea, but not really sure how viable.
  - t2d2 13y ago
    Using /b/ as an unpaid mechanical Turk alternative sounds intriguing yet profoundly disturbing. Imagine being able to fire off a dozen /b/tards bots at the website or target of your choice.
    - deleted 13y ago
      [deleted]
- twic 13y ago
bedhead 13y ago
Maybe we can channel Potter Stweart into an algorithm somehow?
- talleyrand 13y ago
  That's what I was thinking, counselor.
nathanb 13y ago
Seems like we were having this same problem with email spam, and Bayesian-based learning filters revolutionized the spam filtering landscape. Has anyone tried throwing computer learning at this problem? We as humans can readily classify images into three vague categories: clean, questionable, and pornographic. The problem of classification is not only one of determining which bucket an image falls into but also one of determining where the boundaries between buckets are. Is a topless woman pornographic? A topless man? A painting of a topless woman created centuries ago by a well-recognized artist? A painting of a topless woman done yesterday by a relatively unknown artist? An infant being bathed? A woman breastfeeding her baby? Reasonable people may disagree on which bucket these examples fall in. So what if I create three filter sets: restrictive, moderate, and permissive, and then categorize 1,000 sample images as one of those three categories for each filter set (restrictive could be equal to moderate but filter questionable images as well as pornographic ones). Assuming that the learning algorithm was programmed to look at a sufficiently large number of image attributes, this approach should easily be capable of creating the most robust (and learning!) filter to date. Has anyone done this?
- clienthunter 13y ago
  This was my first thought. With a good training set and a savvy algo I believe machine learning can be good with images, and theres an unprecedented amount of training sets out there to be scraped...
ismaelc 13y ago
You can use APIs like these to do nude detection - https://www.mashape.com/search?query=nude https://www.mashape.com/search?query=nude
jcfiala 13y ago
It seems to me that if you could somehow solicit comments on the picture, you then could do text analysis on the comments to see if someone thought they were porn or not. (Well, I'm being a little silly, but there's a germ of an idea there.)
ma2rten 13y ago
I did this for my bachelor thesis for a company that shall remain unnamed. I am pretty confident that my approach works better than any of the answer posted on stackoverflow. I used the so called Bag of Visual Words approach. At that time the state of the art in image recognition (now it's neural networks). You can read about on Wikipeida. The only main change from the standard approach (SHIFT + k-means + histograms + SVM + chi2 kernel) was that I used a version of SHIFT that uses color features. In addition to this I used a second machine learning classifier based on the context of the picture. Who posted it? Is it a new user? What are the words in the title? How many view does the picture have.... In combination the two classifiers worked nearly flawless. Shortly after that, chat roulette has having it's porn problem and it was in the media that the founder was working on a porn filter. I send an email to offer my help, but didn't get an reaction.
- clebio 13y ago
  This sounds quite interesting. Is there any of the research of code base that you can share? Or otherwise any references about the standard approach which you would recommend?
  - ma2rten 13y ago
    This software is free for non-commerical use: http://koen.me/research/colordescriptors/ http://koen.me/research/colordescriptors/ You can find other implementation of varying quality if you Google for Bag of Visual Words. For the final classification, I would recommend scikit-learn.
- anjc 13y ago
  SHIFT or SIFT? What's SHIFT?
  - mturmon 13y ago
    I think he means SIFT. I'm not aware of SHIFT either, and a look at his PAMI paper shows use of color features as well as various SIFT features.
- 13y ago
singlow 13y ago
So, who's going to write the ROT13 algorithm for images. Just call it ROT128 and rotate the color value of the bits and use a ROT128 image viewer to view the original image.
- kalleboo 13y ago
  That function is typically called "Invert".
digitalsushi 13y ago
Invent an algorithm that can calculate humanity's creative thoughts.
level09 13y ago
would any one be interested in purchasing an API subscription for this kind of service ? IMO, a pipeline of AI filters can be efficient to some extent.
- ismaelc 13y ago
  Put it in Mashape :) (Disclaimer: I work for Mashape)
hcarvalhoalves 13y ago
Pornography is so creative that I find it hard to have one algorithm that can detect it all. Looking for features certainly wouldn't catch the more weird stuff. Maybe a good approach is an image lookup, trying to find the image on the web and seeing if it appears on a porn site, or a pornographic context.
maddddddddddddd 13y ago
this is what i did my final image processing project on in college. it was fun to sit in front of a room of people with loads of porn on the projector. bottom line: you can't do this with any acceptable level of accuracy.
djent 13y ago
It wouldn't solve the entire problem, but you could look for the watermarks that major porn networks stamp on their images.
bicknergseng 13y ago
Use CrowdFlower.
unoti 13y ago
If you're interested in Machine Learning, the outstanding Coursera course on machine learning just started a couple of days ago. It covers a variety of machine learning topics, including image recognition. The first assignment isn't due for a couple of weeks, so it's a perfect time to jump in and take the machine learning course! https://www.coursera.org/course/ml https://www.coursera.org/course/ml
npatten 13y ago
"You can programatically detect skin tones - and porn images tend to have a lot of skin. This will create false positives but if this is a problem you can pass images so detected through actual moderation. This not only greatly reduces the the work for moderators but also gives you lots of free porn. It's win-win." hilarious!
lectrick 13y ago
Relevant: http://en.wikipedia.org/wiki/I_know_it_when_I_see_it http://en.wikipedia.org/wiki/I_know_it_when_I_see_it Basically, it's impossible to completely accurately identify pornography without a human actor in the mix, due to the subjectivity... and especially considering that not all nudity is pornographic.
Houshalter 13y ago
Everyone is focusing on the machine vision problem but the OP had a good idea: >There are already a few image based search engines as well as face recognition stuff available so I am assuming it wouldn't be rocket science and it could be done. Just do a reverse image search for the image, see if it comes up on any porn sites or is associated with porn words.
denzil_correa 13y ago
I am aware of some nice scholarly work in this space. You may find Shih et al. approach of particular interest [0]. Their approach is very straight forward and based on image retrieval. They have also reported an accuracy of 99.54% for Adult image detection in their dataset. [0] Shih, J. L., Lee, C. H., & Yang, C. S. (2007). An adult image identification system employing image retrieval technique. Pattern Recognition Letters, 28(16), 2367-2374. Chicago http://sjl.csie.chu.edu.tw/sjl/albums/userpics/10001/An_adult_image_0identification_system_employing_image_retrieval_technique.pdf http://sjl.csie.chu.edu.tw/sjl/albums/userpics/10001/An_adul...
nate510 13y ago
A corollary of Rule 34 is that an algorithm to classify porn is NP-Hard. Um, so to speak.
umphetico 13y ago
why?
beat 13y ago
Algorithmic solutions will always be hard. "I know it when I see it" is hard to program. Depending on the site, I'd go to a trust-based solution. New users get their images approved by a human censor (pr0n == spambot in most cases). Established users can add images without approval. If you're going to try software, try something that errs on the side of caution, and send everything to a human for final decision-making, just like spam filters.

142 points by romain_g 13y ago

Google reverse image search can come up with a search likely to return the given image. Perhaps this can be used for porn classification.

eksith 13y ago

To this day, I believe the best method for picking out these images is a human censor (with appropriate, company provided, counseling afterward). Edit: No shortage of stock image reviewer jobs https://google.com/search?hl=en&q=%22image%20reviewer%22 https://google.com/search?hl=en&q=%22image%20reviewer%22 I'm trying to find an interview of one of these people describing what it's like on the other end. It wasn't a pleasant story. These folks are employed by the likes of Facebook, Photobucket etc... Most are outsourced, obviously, and they all have very high turnover.

RossM 13y ago
I seem to remember reading an article about people doing this at Google(?) in a pretty poor state. Edit: I think it was this one: http://www.buzzfeed.com/reyhan/tech-confessional-the-googler-who-looks-at-the-wo http://www.buzzfeed.com/reyhan/tech-confessional-the-googler...
- eksith 13y ago
  Yep! That's the one. Thanks. A pretty unpleasant job no matter what angle you look at it. I remember this bit the most : "Google covered one session with a government-appointed therapist — and encouraged me to go out and get my own therapy after I left." This is the down side to 'abstracting away' the dirty end of filtering. I'm looking forward to a day when this can be properly automated, but, considering the ever-changing nature of erotica to begin with, I don't see that happening any time soon.
  - XorNot 13y ago
    Developing a strong AI which can do this without going insane itself is going to be the robo-pysche challenge of the future.

mixmax 13y ago

detecting all porn seems to be an almost impossible problem. Many kinds of advanced porn (BDSM, etc.) don't have much skin - often the actors are in latex, tied up, or whatever. It's obviously porn when you see it, but detecting it seems incredibly hard. Detecting smurf-porn(1) (yes that's a thing...) is even harder since all the actors are blue. http://pinporngifs.blogspot.dk/2012/09/smurfs-porn.html?zx=7ac31f5871d5a788 http://pinporngifs.blogspot.dk/2012/09/smurfs-porn.html?zx=7... - obviously very NSFW, but quite funny.

ma2rten 13y ago
It is possible high accuracy if you use machine learning and a sufficiently large training set. That said said even humans sometimes don't agree is something is porn or not.
- rfergie 13y ago
  Just saying "machine learning" is not very useful here. What machine learning techniques work well in this case and what are the major pitfalls? Then you can convince me that a "sufficiently large training set" exists and is smaller than "all the images on the internet".
  - ma2rten 13y ago
    See my other post on this page. I would argue that this less difficult than you average image classification problem. Just have a look at what kind of challenges image classification can tackle, picking the correct class out of 1000s of classes. Porn is normally well-lit, the subject is at the center of the image, etc. The main difficulty is to define what is porn and what is not ... It's easy see the difference between porn and pictures of bicycles. But how about porn and artistic nudity? You see it's actually a scale, but you are trying to make a binary decision. Another problem (at least with the method I explained below) is that portraits sometimes get misclassified. Maybe it could help to integrate face detection. I'd suspect that more recent models would not have this problem (e.g. ones that take not only local features into account). Other times it makes mistakes, where you think "why on earth would think this is porn". Again combing different method should help to eliminate those outliers. Outliers are also a problem, e.g. black and white pictures. Again an ensemble of different models (e.g. one color independent one) might help. Niches are not really a big problem. BSDM porn is as far as I have seen the only niche of porn that really different visually.

bachback 13y ago

machine learning? because you also want to filter cats.

rspeer 13y ago
Machine learning is a field of research, not a magical incantation that you say to solve everything.

asolove 13y ago

Amazon Mechanical Turk has an adult-content marker specifically for this purpose. Lots of people have done the paperwork to qualify for adult-content jobs and the cost of having humans do it at scale is very low: https://requester.mturk.com/help/faq#can_explicit_offensive https://requester.mturk.com/help/faq#can_explicit_offensive Source: I helped implement a MT job to filter adult content for a large hosting company.

phorese 13y ago
So i can literally get paid for looking at porn. Huh, who knew... :)
- selmnoo 13y ago
  At the cost of your mental health, sure. I recall reading an article about the human workers who had this job at Google... they had crap benefits, crap pay, and no mental healthcare. As a result a lot of the people in the field had depression and other mental issues.
  - _mulder_ 13y ago
    Source?.... or any hints as to what the article was called. Sounds interesting.
  - romain_g 13y ago
    Would it be this one? http://www.buzzfeed.com/reyhan/tech-confessional-the-googler-who-looks-at-the-wo http://www.buzzfeed.com/reyhan/tech-confessional-the-googler...
  - gadders 13y ago
    And RSI as well I would imagine.
  - sharkweek 13y ago
    Friend was a contractor on the YouTube filter team -- they lasted about 9 months before it became too much and had to leave. There was absolutely no reward in showing up to work, and some of the things they saw have likely scarred their memories forever

jmngomes 13y ago

I came across nude.js (http://www.patrick-wied.at/static/nudejs/ http://www.patrick-wied.at/static/nudejs/) when researching for a social network project, seems quite nice and is Javascript based.

hugofirth 13y ago

Whilst I agree that programmatically eliminating porn images is a very hard problem. Programmatically filtering porn websites might be easier, beyond just a simple key word search and whitelist. If you assume that porn tends to cluster, rather than exist in isolation, then a crawl of other images on the source pages , applying computer vision techniques, should allow you to block pages that score above a threshold number of positive results (thus accounting for inaccuracy and false positives).

VLM 13y ago
"score above a threshold number of positive results" How about social scoring? A normal (or even a weirdo) teenage boy would spend less than a second examining my ugly old profile pix, but after ten or so of your known teen male users are detected to spend 5 minutes at a time, a couple times a day, closely studying a suspected profile pix, I think you can safely conclude that pix is not a pix of me and then flag / censor / whatever it for the next 10K users.
- hugofirth 13y ago
  The graph theorist in me is rubbing my hands in glee at the thought of seeing if you could extrapolate out that approach to catch a broader range of offensive imagery through relationships and usage patterns.
  - XorNot 13y ago
    Key problem: profile pictures are everywhere on a website by definition. People look at them for over an hour at a time depending how your site is laid out. If you wanted to spot shock imagery it's easier - study navigation aways or rapid scrolling.

VLM 13y ago

This is probably going to get downvoted, but if lots of people are not overzealous puritans and want some skin, the best overall system design that maximizes happiness and profit is probably sharding into puritanweirdos.example.com with no skin showing between toes and top of turtleneck (edited to add no pokies either) and normalpeople.example.com with 99% of the human race The best solution to a problem involving computers is sometimes computer related, but sometimes is social. The puritans are never going to get along with the normal people anyway, so its not like sharding them is going to hurt. Another way to hack the system is not to hire or accept holier than thou puritans. Personality doesn't mesh with the team, doesn't fit culture, etc. You have to draw the line somewhere, and weirdos on either end should get cut, so no CP or animals at one extreme, and no holy rollers on the other extreme. The final social hack is its kind of like dealing with bullies via appeasement. So they're blocking reasonable stuff today, tomorrow they want to block all women not wearing burkhas or depictions of women damaging their ovaries by driving. Appeasing bullies never really works in the long run, so why bother starting. "If you claim not to like it, or at least enjoy telling everyone else repeatedly how you claim not to like it, stop looking at it so much, case closed"

Shivetya 13y ago
Harassment laws/lawyers would find you their wet dream. The rule with pictures in any work environment is to err on the side of extreme caution. More than likely your not going to find out they don't mesh with the team till they quit or are fired. If your lucky its not followed by a lawsuit. See, its not their job to not be offended, it is your job to offer a harassment free work environment. Whom you are catering too depends on what is PC or not at the time, fortunately its pretty easy to determine whose whims to cater to or not
- jiggy2011 13y ago
  I'm not sure it is that easy, even HN discussions on the topic don't come to any particular conclusion and that is within a fairly narrow range of people. In an old job I had the task of cleaning spam off a forum, this meant looking at rather a lot of porn but it would never have occurred to me to sue on that basis, yet that is still a job that needs to be done.

cvshepherd 13y ago

i've read about these skin tone filters more than once now, and i'm really curious to find out if these take all possible skin tones into consideration (i can't really imagine that they do. escpecially dark skin tones would get way too many false positives, would they not?).

wehadfun 13y ago

Probably the easiest way is with motion and sound. Checking for skin would be hard depending on the type of content as mixmax pointed out

VLM 13y ago

Nobody has discussed i18n and l10n issues? What passes for pr0n in SF is a bit different than tx.us and thats different from .eu and from .sa (sa is saudi arabia not south africa, although they've probably got some interesting cultural norms too) If you're trying for "must not offend any human being on the planet" then you've got an AI problem that exceeds even my own human intelligence problem to figure out. Especially when it extends past pr0n and into stuff like satire, is that just some dudes weird self portrait, or a satire of the prophet, and are you qualified to figure it out?

deleted 13y ago

[deleted]

betterunix 13y ago

How about a picture of a woman's breasts? What about an erect penis? Sounds like porn, but you might also see these things in the context of health-related pictures or some other educational material. The classic problem of trying to filter pornography is trying to separate it from information about human bodies. I suspect that doing this with images will be even harder than doing it with text.

Zimahl 13y ago
Definitely true. Facebook had a dust-up when a woman posted a topless photo of herself after she had had a double mastectomy. That said, not all sites are like Facebook and we aren't talking about filtering all the images on the internet, just ones on specific sites. One example I can think of is that a forum for a sports team might not want NSFW pictures posted as it would be irrelevant.

Theodores 13y ago

In The Olden Pre-Digital Days porn was either in print or on a television screen. Back then (we are talking two whole decades ago) experienced broadcast engineers could instantly spot porn just by catching a look at an oscilloscope (of which there were usually many in a machine room). Notionally the oscilloscope would be there to show that the luminance and chroma was okay in the signal (i.e. it could be broadcast over the airwaves to look as intended at the other end - PAL/NTSC), however, porn and anything likely to be porn had a distinctive pattern on the oscilloscope screen. Should porn be suspected then the source material would obviously be patched through to a monitor 'just in case'. Note that the oscilloscope was analog and that the image would be changing 25/30 times a second. Also, back then there were not so many false positives on broadcast TV, e.g. pop videos etc. where today's audience deems them artful rather than porn. If I had to solve the problem programatically I would find a retired broadcast engineer and start from there, with what can be learned from a 'scope.

daurnimator 13y ago
It's probably related to the type of cameras available in the day rather than anything else...

racbart 13y ago

Wouldn't testing for skin colors produce far too many false positives to be useful? All these beach photos, fashion lingerie photos, even close portraits. And how about half of music stars these days who seem to try to never get caught more clothed than half naked? Nudity != porn and certainly half-nudity != porn. I'd rather go for pattern recognition. There's lot of image recognition software these days that can distinguish the Eiffel Tower from the Statue of Liberty and it might be useful to detect certain body parts and certain body configurations (for these shots that don't contain any private body part but there are two bodies in an unambiguous configuration).

betterunix 13y ago
"detect certain body parts" When I was a kid, we had a firewall at school that tried to filter pornography by doing something similar with text. Doing research on breast cancer turned out to be rather tricky. So let's say you try to detect certain body parts. Now you have someone who wants to know more about their body, but you are classifying images from medical / health articles as pornography. "certain body configurations" So now instead of having trouble reading about my own body, I will have trouble looking at certain martial arts photos: https://upload.wikimedia.org/wikipedia/commons/1/14/BostonKetsugoJujutsu.jpg https://upload.wikimedia.org/wikipedia/commons/1/14/BostonKe... I am not saying these are unsolvable problems, but they are certainly hard problems. Even using humans to filter images tends to result in outrageous false positives sometimes: http://abcnews.go.com/Health/breastfeeding-advocates-hold-facebook-protest/story?id=15530012 http://abcnews.go.com/Health/breastfeeding-advocates-hold-fa...
- racbart 13y ago
  You are correct and I'm not saying that I described the Holy Grail of detecting porn in a single paragraph. I'm just pointing to another direction. No solution to a very complex problem could be one-dimensional. Combining several different tests might lead to a solution. I.e. these jujitsu photos should not even be detected as “certain body configurations” as people there are fully clothed and there's not much actual bodies seen in the picture (so mentioned skin color definitely should come to play when detecting wether you see a body or not). At the end of the day I doubt there could be a fully bulletproof and always correct solution using current state of tech. But you need to factor much more than just skin color if you try to build an automated solution to this problem.

_mulder_ 13y ago

Here's an idea... Develop a bot to trawl NSFW sites and hash each image (combined with the 'skin detecting' algorithms detailed previously). Then compare the user uploaded image hash with those in the NSFW database. This technique relies on the assumption that NSFW images that are spammed onto social media sites will use images that already exist on NSFW sites (or are very similar to). Then it simply becomes a case of pattern recognition, much like SoundHound for audio, or Google Image search. It wouldn't reliably detect 'original' NSFW material, but given enough cock shots as source material, it could probably find a common pattern over time. edit: I've just noticed rfusca in the OP suggests a similar method

pa5tabear 13y ago
How do you program that sort of thing? Do you have to tell it what shapes/colors to look for? Or do a combination of overall image similar combined with localized image similarity and portion by portion image comparison?
- viraptor 13y ago
  Maybe recognising the furniture in the background would work too ;) I remember there was a website/catalog of IKEA furniture somewhere made using NSFW photos.
- _mulder_ 13y ago
  Well Image Hashing is distinct from normal MD5 hashing as the hash does consider similarity of colour, etc. so it's not purely binary. A Google search produced a library called pHash.org that might do something similar.
Fomite 13y ago
"This multi-TB disk array labeled 'Porn' has a legitimate business use!"
lifeformed 13y ago
Is it possible to hash an image so that you can partially match it with subsets of that image (like cropped regions or resizes)? Or a slight modification of that image (colors shifted, image flipped, etc).

dschiptsov 13y ago

by filename ,)

primaryobjects 13y ago

This is a classic categorical problem for machine learning. I'm surprised so many suggestions have involved formulating some sort of clever algorithm like skin detection, colors, etc. You could certainly use one of those for a baseline, but I'd bet machine learning would out-score most human-derived algorithms. Take a look at the scores for classifying dogs vs cats with 97% accuracy http://www.kaggle.com/c/dogs-vs-cats/leaderboard http://www.kaggle.com/c/dogs-vs-cats/leaderboard. You could use a technique of digitizing the image pixels and feeding to a learning algorithm, similar to http://www.primaryobjects.com/CMS/Article154.aspx http://www.primaryobjects.com/CMS/Article154.aspx.

adorable 13y ago

I have developed an algorithm to detect such images, based on several articles published by research teams all over the world (it's incredible to see how many teams have tried to solve this problem!). I found out that no single technique works great. If you want an efficient algorithm, you probably have to blend different ideas and compute a "nudity score" for each image. That's at least what I do. I'd be happy to discuss how it works. Here are a few techniques used: - color recognition (as discussed in other comments) - haar-wavelets to detect specific shapes (that's what Facebook and others use to detect faces for example) - texture recognition (skin and wood may have the same colors but not the same texture) - shape/contour recognition (machine learning of course) - matching with a growing database of NSFW images The algorithm is open for test here: http://sightengine.com http://sightengine.com It works OK right now but once version 2 is out it should really be great.

nailer 13y ago

I used pifilter (now WeSEE:Filter , http://corp.wesee.com/products/filter/ http://corp.wesee.com/products/filter/) for a production, realtime, anonymous confession site (imeveryone.com) in 2010. It cost, IIRC, a tenth of a cent per image URL. Rather than being based on skin tone, it was created based on algos to specifically identify labia, anuses, penises, etc. REST API: send a URL, get back a yes/no/maybe. You decided what to do with the maybes. My experience: - Before launch, I tested it with 4chan b as a feed, and was able to produce a mostly clean version of b with the exception of cartoon imagery. - It could catch most of the stuff people tried to post to the site. Small breasted women (being that breasts are considered 'adult' in the US) was the only thing that would get through and wasn't a huge concern. Completely unmaintained public hair (as revealing as a black bikini) would also get through. - Since people didn't know what I was testing with they didn't work around it (so nobody tried posting drawings or cartoons), but I imagine eg a photo of a prolapse might not trigger the anus detection as the shape would be too different. - pifilter erred on the side of false negative, but one notable false positive: a pastrami sandwich.

cobrausn 13y ago
I don't remember where I read this, but someone once recommended having a bot that posts the image to 4chan b feed and monitor the views and replies. Since porn images on b usually had somewhat predictable replies, it would be a good way to augment another technique and prevent false positives. Funny idea, but not really sure how viable.
- t2d2 13y ago
  Using /b/ as an unpaid mechanical Turk alternative sounds intriguing yet profoundly disturbing. Imagine being able to fire off a dozen /b/tards bots at the website or target of your choice.
  - deleted 13y ago
    [deleted]
twic 13y ago

bedhead 13y ago

Maybe we can channel Potter Stweart into an algorithm somehow?

talleyrand 13y ago
That's what I was thinking, counselor.

nathanb 13y ago

Seems like we were having this same problem with email spam, and Bayesian-based learning filters revolutionized the spam filtering landscape. Has anyone tried throwing computer learning at this problem? We as humans can readily classify images into three vague categories: clean, questionable, and pornographic. The problem of classification is not only one of determining which bucket an image falls into but also one of determining where the boundaries between buckets are. Is a topless woman pornographic? A topless man? A painting of a topless woman created centuries ago by a well-recognized artist? A painting of a topless woman done yesterday by a relatively unknown artist? An infant being bathed? A woman breastfeeding her baby? Reasonable people may disagree on which bucket these examples fall in. So what if I create three filter sets: restrictive, moderate, and permissive, and then categorize 1,000 sample images as one of those three categories for each filter set (restrictive could be equal to moderate but filter questionable images as well as pornographic ones). Assuming that the learning algorithm was programmed to look at a sufficiently large number of image attributes, this approach should easily be capable of creating the most robust (and learning!) filter to date. Has anyone done this?

clienthunter 13y ago
This was my first thought. With a good training set and a savvy algo I believe machine learning can be good with images, and theres an unprecedented amount of training sets out there to be scraped...

ismaelc 13y ago

You can use APIs like these to do nude detection - https://www.mashape.com/search?query=nude https://www.mashape.com/search?query=nude

jcfiala 13y ago

It seems to me that if you could somehow solicit comments on the picture, you then could do text analysis on the comments to see if someone thought they were porn or not. (Well, I'm being a little silly, but there's a germ of an idea there.)

ma2rten 13y ago

I did this for my bachelor thesis for a company that shall remain unnamed. I am pretty confident that my approach works better than any of the answer posted on stackoverflow. I used the so called Bag of Visual Words approach. At that time the state of the art in image recognition (now it's neural networks). You can read about on Wikipeida. The only main change from the standard approach (SHIFT + k-means + histograms + SVM + chi2 kernel) was that I used a version of SHIFT that uses color features. In addition to this I used a second machine learning classifier based on the context of the picture. Who posted it? Is it a new user? What are the words in the title? How many view does the picture have.... In combination the two classifiers worked nearly flawless. Shortly after that, chat roulette has having it's porn problem and it was in the media that the founder was working on a porn filter. I send an email to offer my help, but didn't get an reaction.

clebio 13y ago
This sounds quite interesting. Is there any of the research of code base that you can share? Or otherwise any references about the standard approach which you would recommend?
- ma2rten 13y ago
  This software is free for non-commerical use: http://koen.me/research/colordescriptors/ http://koen.me/research/colordescriptors/ You can find other implementation of varying quality if you Google for Bag of Visual Words. For the final classification, I would recommend scikit-learn.
anjc 13y ago
SHIFT or SIFT? What's SHIFT?
- mturmon 13y ago
  I think he means SIFT. I'm not aware of SHIFT either, and a look at his PAMI paper shows use of color features as well as various SIFT features.
13y ago

singlow 13y ago

So, who's going to write the ROT13 algorithm for images. Just call it ROT128 and rotate the color value of the bits and use a ROT128 image viewer to view the original image.

kalleboo 13y ago
That function is typically called "Invert".

digitalsushi 13y ago

Invent an algorithm that can calculate humanity's creative thoughts.

level09 13y ago

would any one be interested in purchasing an API subscription for this kind of service ? IMO, a pipeline of AI filters can be efficient to some extent.

ismaelc 13y ago
Put it in Mashape :) (Disclaimer: I work for Mashape)

hcarvalhoalves 13y ago

Pornography is so creative that I find it hard to have one algorithm that can detect it all. Looking for features certainly wouldn't catch the more weird stuff. Maybe a good approach is an image lookup, trying to find the image on the web and seeing if it appears on a porn site, or a pornographic context.

maddddddddddddd 13y ago

this is what i did my final image processing project on in college. it was fun to sit in front of a room of people with loads of porn on the projector. bottom line: you can't do this with any acceptable level of accuracy.

djent 13y ago

It wouldn't solve the entire problem, but you could look for the watermarks that major porn networks stamp on their images.

bicknergseng 13y ago

Use CrowdFlower.

unoti 13y ago

If you're interested in Machine Learning, the outstanding Coursera course on machine learning just started a couple of days ago. It covers a variety of machine learning topics, including image recognition. The first assignment isn't due for a couple of weeks, so it's a perfect time to jump in and take the machine learning course! https://www.coursera.org/course/ml https://www.coursera.org/course/ml

npatten 13y ago

"You can programatically detect skin tones - and porn images tend to have a lot of skin. This will create false positives but if this is a problem you can pass images so detected through actual moderation. This not only greatly reduces the the work for moderators but also gives you lots of free porn. It's win-win." hilarious!

lectrick 13y ago

Relevant: http://en.wikipedia.org/wiki/I_know_it_when_I_see_it http://en.wikipedia.org/wiki/I_know_it_when_I_see_it Basically, it's impossible to completely accurately identify pornography without a human actor in the mix, due to the subjectivity... and especially considering that not all nudity is pornographic.

Houshalter 13y ago

Everyone is focusing on the machine vision problem but the OP had a good idea: >There are already a few image based search engines as well as face recognition stuff available so I am assuming it wouldn't be rocket science and it could be done. Just do a reverse image search for the image, see if it comes up on any porn sites or is associated with porn words.

denzil_correa 13y ago

I am aware of some nice scholarly work in this space. You may find Shih et al. approach of particular interest [0]. Their approach is very straight forward and based on image retrieval. They have also reported an accuracy of 99.54% for Adult image detection in their dataset. [0] Shih, J. L., Lee, C. H., & Yang, C. S. (2007). An adult image identification system employing image retrieval technique. Pattern Recognition Letters, 28(16), 2367-2374. Chicago http://sjl.csie.chu.edu.tw/sjl/albums/userpics/10001/An_adult_image_0identification_system_employing_image_retrieval_technique.pdf http://sjl.csie.chu.edu.tw/sjl/albums/userpics/10001/An_adul...

nate510 13y ago

A corollary of Rule 34 is that an algorithm to classify porn is NP-Hard. Um, so to speak.

umphetico 13y ago

why?

beat 13y ago

Algorithmic solutions will always be hard. "I know it when I see it" is hard to program. Depending on the site, I'd go to a trust-based solution. New users get their images approved by a human censor (pr0n == spambot in most cases). Established users can add images without approval. If you're going to try software, try something that errs on the side of caution, and send everything to a human for final decision-making, just like spam filters.