9 ms·

Show HN: An AI program to check videos for NSFW content

alexb_ 5y ago
I wonder if it would be possible to not only single scenes in a movie out, but also use imdb data to help detect which actor/actress is in said scenes for categorization. Could be an extremely useful tool.
- dynamite-ready 5y ago
  That's an interesting idea. There are off the shelf tools that can recognise famous people already (Amazon Rekognition, for example). The problem in connection with this hack, is in not wanting to call a third party web service.
- hbn 5y ago
  Wouldn't it be easier to use facial recognition for that? In fact I recall years ago Google Play Movies already did that, where you could pause the movie and it would tell you the name of the faces on screen at that moment. Or do you mean if you're looking for NSFW scenes of a particular actor?
capableweb 5y ago
It seems to acknowledge itself as not being accurate, at least from what I gather in this passage: > it determines the Baywatch intro to be somewhat NSFW at a glance As nothing in the Baywatch intro is NSFW (https://www.youtube.com/watch?v=O0nqwgu_Us4 https://www.youtube.com/watch?v=O0nqwgu_Us4), it seems the model is failing at large here. Interesting subject and lots of praise to be had if the model can be made accurate, but seems it's not really there yet. I wish you luck in getting there!
- globular-toast 5y ago
  I would consider the Baywatch intro NSFW. Perhaps you don't consider it so because it's Baywatch, but if it was a random video of closeups of people in bathing costumes I bet you would. One thing I find really interesting with AI/Machine Learning is how it can cause you to re-evaluate your own biases. I say the AI is right here and you are wrong.
- dynamite-ready 5y ago
  Here's a blog post where I explain my motivation for creating the program - https://raskie.com/post/practical-ai-autodetecting-nsfw https://raskie.com/post/practical-ai-autodetecting-nsfw. There are some hyperparameters to tune, but you're right, it's extremely flaky. Part of that might be down to my work (I personally wanted the app to be prudish enough to flag the Baywatch intro), but the off the shelf model I used for this, does produce some strange results. Kind of wanted to use this app as a convo piece to discuss AI with people who aren't entirely technical, or with technical people who wonder what AI can and can't do.
- heeen2 5y ago
  > nothing in the Baywatch intro is NSFW a debatable claim. There are several scenes in the intro I would not want to have paused fullscreen in a work environment.
  - wongarsu 5y ago
    NSFW is definitely a sliding (multi-dimensional) scale, and different cultures and work places place themselves differently on it. At a previous workplace (software department in a manufacturing company in Western Europe) a women with only the lower half of her bikini laying on a beach was fine as a desktop background as long as you couldn't see her nipples. At a different company in field service exposed frontal breasts were still SFW. Meanwhile in many other jobs you would get in trouble for far less.
    - meepmorp 5y ago
      Many years ago, I worked a printing company, and several of our accounts were local porn mags and an adult newspaper that was basically escort ads. The account reps loved those jobs.
  - BeefWellington 5y ago
    Are there shows you'd be comfortable having paused fullscreen in a work environment? Just doing so would suggest you aren't actually working (on its own a problem in many cultures).
    - Charlie_26 5y ago
      How can you call yourself Beef Wellington when you aren't even familiar with the concept of lunch
    - macksd 5y ago
      I probably wouldn't do this in a shared office, but I'm pretty open about the fact that I often like to have a show in the background while I work. Keeps my mind from wandering too far if I have a momentary pause for a big download or a long compile or something. And a lot of people find moderate background noise helps them focus. I've been through every Star Trek multiple times and am plenty productive at work. But Bay Watch isn't something I would want coworkers to see. Nipples or no nipples, it's trying to sexualize / objectify about as much as they could get away with in prime time TV. I can't go join a meeting about inclusive culture with my female coworkers if they just saw me staring at Pamela Anderson's bouncing tits in the middle of the workday.
- hutzlibu 5y ago
  "As nothing in the Baywatch intro is NSFW " I would suppose that depends on the workplace and area(and time) where you live. It is definitely "somewhat" sexualised content in my opinion (close up of a womens body stripping down), even though there is no nipple to see. But sure, lots of normal advertisement sells more sex. So is the red line, when a nipple is shown? That would classify biological and medical content as nsfw. My point is, human morals are very different. Humans fight about, what content is NSFW. A computer model can never be "acurate" in that sense.
  - hotpotamus 5y ago
    Indeed, I worked in a photo lab back in the day when NSFW images were still on film and we had to think about these kinds of rules, but basically anything was permitted for medical purposes - I once scanned slides for a doctor doing a lecture on pelvic inflammatory disease. She did not say what they were when dropping them off and amusedly asked about it when she picked them up. But it was the very opposite of sexy and quite clearly for genuine medical purposes. This reminds me of a question I've long had about this sort of thing you reminded me of - in the US at least, we seem to draw the line for showing off breasts at the nipple, but only female nipples. Setting aside any merit or logic to that, can AI actually determine female vs male nipples? I assume not through the nipple itself, but the breast it's attached to or the face of the person?
    - hutzlibu 5y ago
      "can AI actually determine female vs male nipples?" Not with 100% accuracy, but on average yes, as a female breast is indeed quite different from a male breast. Shape, hair, size, ... But there are of course exeptions. Muscular women, with small breast who did not breast feed yet, are probably quite hard to detect.
    - amelius 5y ago
      > can AI actually determine female vs male nipples? Yes, through context. A bearded woman holding a chainsaw would probably fail the test.
- unfocussed_mike 5y ago
  > As nothing in the Baywatch intro is NSFW Clearly incorrect by any reasonable definition unless you work at a swimwear company or a few bits of the media. I mean, you or I might think Alexandra Paul in a one-piece is an image approaching the beauty of classical sculpture, but quite a lot of classical sculpture is NSFW in most of the world, and the brow is a bit lower where Baywatch is concerned.
  - capableweb 5y ago
    One (dare I say?) "reasonable" definition of NSFW is nudity, which in most contexts I'm guilty of associating it with. I know it's an acronym (Not Safe For Work) but I never took it that literally, as otherwise most content would be NSFW unless you work in media. I wouldn't consider Seinfeld NSFW, but the companies I work for wouldn't want to see me watching it at work time, even if it's not normally considered NSFW.
    - unfocussed_mike 5y ago
      Nudity is not even the beginning of NSFW. If you're looking at nudity in most places you're way beyond NSFW and into gross misconduct. Very many workplaces have a definition of NSFW that is a bit more restrictive than Facebook's content rules. And rightly so. Go to work, do your work, go home. It's not adult daycare. Watching video footage of people running around in swimwear is not really appropriate content unless you work in media or swimwear. And the title sequence to Baywatch is hardly an advert for sporting endeavour; Baywatch is titillation. It would be entirely reasonable for almost anyone, male or female, to complain about that content. It is far more sensible to define NSFW as "things I shouldn't be looking at just in case someone has an unjustifiable complaint to make about me that they are looking to bolster with additional offences".
- binarymax 5y ago
  There's no such thing as a perfect model. The acknowledgement is welcome. Per the Baywatch example - there's clearly a grey area here, and in my opinion the model is working as intended. False positives for NSFW are better than false negatives.
dsr_ 5y ago
If it doesn't understand meaning and context, it can't work. It will over-report on video of baby's first bath, beach volleyball, and high school wrestling; it will under-report on many of the kinkier or more subtle forms of porn. Humans get this stuff wrong all the time simply by having different cultural contexts; there's no way the 2022 state of the art is up to the challenge.
- lijogdfljk 5y ago
  Why does everything have to be perfect? Humans get this wrong all the time as you say, and we still do it - right? Seems like we already acknowledge good enough practices and are okay with them. Lets not demand perfection in the same way we shouldn't assume perfection of ~~AI~~ ML.
- nonameiguess 5y ago
  It could potentially revolutionize the opposite application, those sites where someone manually finds and catalogs all the time points at which movies contain nudity so people can see their favorite celebs in the buff. False positives are probably worth the automation there.
- FrostKiwi 5y ago
  Fully agree - This seems like a project with rather limited in scope. And that's ok. The authors even acknowledge it. It's not an absolute judge of SFW'ness, neither does it claim to be. I can see it as a tool to skim a large set of videos for further human review. Besides, beach volleyball very much is NSFW depending on the situation.
  - rovr138 5y ago
    >depending on the situation. That's part of the issue.
  - Chris2048 5y ago
    > And that's ok. The authors even acknowledge it That doesn't always stop corps productizing it, and making it their official NSFW filter. Good enough is good enough, just so long as they can pay less salaries.
bojangleslover 5y ago
Hot dog or not hot dog?
- snek_case 5y ago
  Melon or not melon.
dncornholio 5y ago
What this program seems to do is create some screenshots from an mp4 and feed them through 2 .exe files, then display the results in a React app
- dynamite-ready 5y ago
  Yep. It's classic hack. If this was for a product, the first thing I would have done, was drop half the tools in the stack for their inefficiency! For example, the GUI shell is using Node Webkit (which I prefer to Electron, but is essentially the same thing). That in itself is quite bad, but it's not the worst approach to building a desktop app, as Microsoft and Slack have already proven. But the .exe's you mentioned are also quite large. The code is very transparent though. I doubt you'd have been the only one to note how clunky this all is!
- MayeulC 5y ago
  Yep, two random exe, and no license is just what I saw before closing the page.
  - dynamite-ready 5y ago
    Actually, there's a GNU license in both the CLI and GUI project. And the source code for the two 'random exe' can be found here - https://github.com/dynamite-ready/movie-parser-cli https://github.com/dynamite-ready/movie-parser-cli
    - maleldil 5y ago
      There isn't a LICENSE file in the GUI project. The LICENSE file in the CLI project seems to be the GPLv3, but the text is non-standard.
      - dynamite-ready 5y ago
        It's in a subfolder for the GUI project. Do you have any advice on the non standard text in the license? This is a concern I've never had before. Tbh, I'm surprised anyone has read it!
scyzoryk_xyz 5y ago
Ok now can we make this work in the opposite direction - make sfw video nsfw please?
symisc_devel 5y ago
Shameless plug: At PixLab, we offer similar model available as a REST API endpoint: https://pixlab.io/cmd?id=nsfw https://pixlab.io/cmd?id=nsfw. The NSFW API endpoint which let you detect bloody & adult content. This can help the developer automate things such as filtering users' image uploads. A tutorial on using such API is available on https://dev.to/unqlite_db/filter-image-uploads-according-to-their-nsfw-score-15be https://dev.to/unqlite_db/filter-image-uploads-according-to-.... The solution can also be deployed on-promises for real-time, local video analysis without leaving the deployment environment: https://pixlab.io/on-premises https://pixlab.io/on-premises.
- SomaticPirate 5y ago
  Seems like this might be the use case for OPs project. A overly sensitive filter that allows you to send only positive alerts to an external API for further validation. Surprised there is no free tier here but I’m not experienced in this space.
dustymcp 5y ago
Reminds me of hotdog not hotdog from silicon valley,
- soylentcola 5y ago
  Hey, it still could be.
nottorp 5y ago
NSFW for which culture?
sschueller 5y ago
How do you determine if a video shows death or torture? The fact that our modern society thinks nudity and sex is bad but violence, torture and death are OK I just don't understand. I would much rather have my kid see the Pamela Anderson sex tape than someone being murdered or even water boarded.

I wonder if it would be possible to not only single scenes in a movie out, but also use imdb data to help detect which actor/actress is in said scenes for categorization. Could be an extremely useful tool.

dynamite-ready 5y ago
That's an interesting idea. There are off the shelf tools that can recognise famous people already (Amazon Rekognition, for example). The problem in connection with this hack, is in not wanting to call a third party web service.
hbn 5y ago
Wouldn't it be easier to use facial recognition for that? In fact I recall years ago Google Play Movies already did that, where you could pause the movie and it would tell you the name of the faces on screen at that moment. Or do you mean if you're looking for NSFW scenes of a particular actor?

capableweb 5y ago

It seems to acknowledge itself as not being accurate, at least from what I gather in this passage: > it determines the Baywatch intro to be somewhat NSFW at a glance As nothing in the Baywatch intro is NSFW (https://www.youtube.com/watch?v=O0nqwgu_Us4 https://www.youtube.com/watch?v=O0nqwgu_Us4), it seems the model is failing at large here. Interesting subject and lots of praise to be had if the model can be made accurate, but seems it's not really there yet. I wish you luck in getting there!

globular-toast 5y ago
I would consider the Baywatch intro NSFW. Perhaps you don't consider it so because it's Baywatch, but if it was a random video of closeups of people in bathing costumes I bet you would. One thing I find really interesting with AI/Machine Learning is how it can cause you to re-evaluate your own biases. I say the AI is right here and you are wrong.
dynamite-ready 5y ago
Here's a blog post where I explain my motivation for creating the program - https://raskie.com/post/practical-ai-autodetecting-nsfw https://raskie.com/post/practical-ai-autodetecting-nsfw. There are some hyperparameters to tune, but you're right, it's extremely flaky. Part of that might be down to my work (I personally wanted the app to be prudish enough to flag the Baywatch intro), but the off the shelf model I used for this, does produce some strange results. Kind of wanted to use this app as a convo piece to discuss AI with people who aren't entirely technical, or with technical people who wonder what AI can and can't do.
heeen2 5y ago
> nothing in the Baywatch intro is NSFW a debatable claim. There are several scenes in the intro I would not want to have paused fullscreen in a work environment.
- wongarsu 5y ago
  NSFW is definitely a sliding (multi-dimensional) scale, and different cultures and work places place themselves differently on it. At a previous workplace (software department in a manufacturing company in Western Europe) a women with only the lower half of her bikini laying on a beach was fine as a desktop background as long as you couldn't see her nipples. At a different company in field service exposed frontal breasts were still SFW. Meanwhile in many other jobs you would get in trouble for far less.
  - meepmorp 5y ago
    Many years ago, I worked a printing company, and several of our accounts were local porn mags and an adult newspaper that was basically escort ads. The account reps loved those jobs.
- BeefWellington 5y ago
  Are there shows you'd be comfortable having paused fullscreen in a work environment? Just doing so would suggest you aren't actually working (on its own a problem in many cultures).
  - Charlie_26 5y ago
    How can you call yourself Beef Wellington when you aren't even familiar with the concept of lunch
  - macksd 5y ago
    I probably wouldn't do this in a shared office, but I'm pretty open about the fact that I often like to have a show in the background while I work. Keeps my mind from wandering too far if I have a momentary pause for a big download or a long compile or something. And a lot of people find moderate background noise helps them focus. I've been through every Star Trek multiple times and am plenty productive at work. But Bay Watch isn't something I would want coworkers to see. Nipples or no nipples, it's trying to sexualize / objectify about as much as they could get away with in prime time TV. I can't go join a meeting about inclusive culture with my female coworkers if they just saw me staring at Pamela Anderson's bouncing tits in the middle of the workday.
hutzlibu 5y ago
"As nothing in the Baywatch intro is NSFW " I would suppose that depends on the workplace and area(and time) where you live. It is definitely "somewhat" sexualised content in my opinion (close up of a womens body stripping down), even though there is no nipple to see. But sure, lots of normal advertisement sells more sex. So is the red line, when a nipple is shown? That would classify biological and medical content as nsfw. My point is, human morals are very different. Humans fight about, what content is NSFW. A computer model can never be "acurate" in that sense.
- hotpotamus 5y ago
  Indeed, I worked in a photo lab back in the day when NSFW images were still on film and we had to think about these kinds of rules, but basically anything was permitted for medical purposes - I once scanned slides for a doctor doing a lecture on pelvic inflammatory disease. She did not say what they were when dropping them off and amusedly asked about it when she picked them up. But it was the very opposite of sexy and quite clearly for genuine medical purposes. This reminds me of a question I've long had about this sort of thing you reminded me of - in the US at least, we seem to draw the line for showing off breasts at the nipple, but only female nipples. Setting aside any merit or logic to that, can AI actually determine female vs male nipples? I assume not through the nipple itself, but the breast it's attached to or the face of the person?
  - hutzlibu 5y ago
    "can AI actually determine female vs male nipples?" Not with 100% accuracy, but on average yes, as a female breast is indeed quite different from a male breast. Shape, hair, size, ... But there are of course exeptions. Muscular women, with small breast who did not breast feed yet, are probably quite hard to detect.
  - amelius 5y ago
    > can AI actually determine female vs male nipples? Yes, through context. A bearded woman holding a chainsaw would probably fail the test.
unfocussed_mike 5y ago
> As nothing in the Baywatch intro is NSFW Clearly incorrect by any reasonable definition unless you work at a swimwear company or a few bits of the media. I mean, you or I might think Alexandra Paul in a one-piece is an image approaching the beauty of classical sculpture, but quite a lot of classical sculpture is NSFW in most of the world, and the brow is a bit lower where Baywatch is concerned.
- capableweb 5y ago
  One (dare I say?) "reasonable" definition of NSFW is nudity, which in most contexts I'm guilty of associating it with. I know it's an acronym (Not Safe For Work) but I never took it that literally, as otherwise most content would be NSFW unless you work in media. I wouldn't consider Seinfeld NSFW, but the companies I work for wouldn't want to see me watching it at work time, even if it's not normally considered NSFW.
  - unfocussed_mike 5y ago
    Nudity is not even the beginning of NSFW. If you're looking at nudity in most places you're way beyond NSFW and into gross misconduct. Very many workplaces have a definition of NSFW that is a bit more restrictive than Facebook's content rules. And rightly so. Go to work, do your work, go home. It's not adult daycare. Watching video footage of people running around in swimwear is not really appropriate content unless you work in media or swimwear. And the title sequence to Baywatch is hardly an advert for sporting endeavour; Baywatch is titillation. It would be entirely reasonable for almost anyone, male or female, to complain about that content. It is far more sensible to define NSFW as "things I shouldn't be looking at just in case someone has an unjustifiable complaint to make about me that they are looking to bolster with additional offences".
binarymax 5y ago
There's no such thing as a perfect model. The acknowledgement is welcome. Per the Baywatch example - there's clearly a grey area here, and in my opinion the model is working as intended. False positives for NSFW are better than false negatives.

dsr_ 5y ago

If it doesn't understand meaning and context, it can't work. It will over-report on video of baby's first bath, beach volleyball, and high school wrestling; it will under-report on many of the kinkier or more subtle forms of porn. Humans get this stuff wrong all the time simply by having different cultural contexts; there's no way the 2022 state of the art is up to the challenge.

lijogdfljk 5y ago
Why does everything have to be perfect? Humans get this wrong all the time as you say, and we still do it - right? Seems like we already acknowledge good enough practices and are okay with them. Lets not demand perfection in the same way we shouldn't assume perfection of ~~AI~~ ML.
nonameiguess 5y ago
It could potentially revolutionize the opposite application, those sites where someone manually finds and catalogs all the time points at which movies contain nudity so people can see their favorite celebs in the buff. False positives are probably worth the automation there.
FrostKiwi 5y ago
Fully agree - This seems like a project with rather limited in scope. And that's ok. The authors even acknowledge it. It's not an absolute judge of SFW'ness, neither does it claim to be. I can see it as a tool to skim a large set of videos for further human review. Besides, beach volleyball very much is NSFW depending on the situation.
- rovr138 5y ago
  >depending on the situation. That's part of the issue.
- Chris2048 5y ago
  > And that's ok. The authors even acknowledge it That doesn't always stop corps productizing it, and making it their official NSFW filter. Good enough is good enough, just so long as they can pay less salaries.

bojangleslover 5y ago

Hot dog or not hot dog?

snek_case 5y ago
Melon or not melon.

dncornholio 5y ago

What this program seems to do is create some screenshots from an mp4 and feed them through 2 .exe files, then display the results in a React app

dynamite-ready 5y ago
Yep. It's classic hack. If this was for a product, the first thing I would have done, was drop half the tools in the stack for their inefficiency! For example, the GUI shell is using Node Webkit (which I prefer to Electron, but is essentially the same thing). That in itself is quite bad, but it's not the worst approach to building a desktop app, as Microsoft and Slack have already proven. But the .exe's you mentioned are also quite large. The code is very transparent though. I doubt you'd have been the only one to note how clunky this all is!
MayeulC 5y ago
Yep, two random exe, and no license is just what I saw before closing the page.
- dynamite-ready 5y ago
  Actually, there's a GNU license in both the CLI and GUI project. And the source code for the two 'random exe' can be found here - https://github.com/dynamite-ready/movie-parser-cli https://github.com/dynamite-ready/movie-parser-cli
  - maleldil 5y ago
    There isn't a LICENSE file in the GUI project. The LICENSE file in the CLI project seems to be the GPLv3, but the text is non-standard.
    - dynamite-ready 5y ago
      It's in a subfolder for the GUI project. Do you have any advice on the non standard text in the license? This is a concern I've never had before. Tbh, I'm surprised anyone has read it!

scyzoryk_xyz 5y ago

Ok now can we make this work in the opposite direction - make sfw video nsfw please?

symisc_devel 5y ago

Shameless plug: At PixLab, we offer similar model available as a REST API endpoint: https://pixlab.io/cmd?id=nsfw https://pixlab.io/cmd?id=nsfw. The NSFW API endpoint which let you detect bloody & adult content. This can help the developer automate things such as filtering users' image uploads. A tutorial on using such API is available on https://dev.to/unqlite_db/filter-image-uploads-according-to-their-nsfw-score-15be https://dev.to/unqlite_db/filter-image-uploads-according-to-.... The solution can also be deployed on-promises for real-time, local video analysis without leaving the deployment environment: https://pixlab.io/on-premises https://pixlab.io/on-premises.

SomaticPirate 5y ago
Seems like this might be the use case for OPs project. A overly sensitive filter that allows you to send only positive alerts to an external API for further validation. Surprised there is no free tier here but I’m not experienced in this space.

dustymcp 5y ago

Reminds me of hotdog not hotdog from silicon valley,

soylentcola 5y ago
Hey, it still could be.

nottorp 5y ago

NSFW for which culture?

sschueller 5y ago

How do you determine if a video shows death or torture? The fact that our modern society thinks nudity and sex is bad but violence, torture and death are OK I just don't understand. I would much rather have my kid see the Pamela Anderson sex tape than someone being murdered or even water boarded.