6 ms·

If it doesn't understand meaning and context, it can't work. It will over-report on video of baby's first bath, beach volleyball, and high school wrestling; it

by dsr_ 5y ago

If it doesn't understand meaning and context, it can't work. It will over-report on video of baby's first bath, beach volleyball, and high school wrestling; it will under-report on many of the kinkier or more subtle forms of porn. Humans get this stuff wrong all the time simply by having different cultural contexts; there's no way the 2022 state of the art is up to the challenge.

lijogdfljk 5y ago
Why does everything have to be perfect? Humans get this wrong all the time as you say, and we still do it - right? Seems like we already acknowledge good enough practices and are okay with them. Lets not demand perfection in the same way we shouldn't assume perfection of ~~AI~~ ML.
nonameiguess 5y ago
It could potentially revolutionize the opposite application, those sites where someone manually finds and catalogs all the time points at which movies contain nudity so people can see their favorite celebs in the buff. False positives are probably worth the automation there.
FrostKiwi 5y ago
Fully agree - This seems like a project with rather limited in scope. And that's ok. The authors even acknowledge it. It's not an absolute judge of SFW'ness, neither does it claim to be. I can see it as a tool to skim a large set of videos for further human review. Besides, beach volleyball very much is NSFW depending on the situation.
- rovr138 5y ago
  >depending on the situation. That's part of the issue.
- Chris2048 5y ago
  > And that's ok. The authors even acknowledge it That doesn't always stop corps productizing it, and making it their official NSFW filter. Good enough is good enough, just so long as they can pay less salaries.
  - joshspankit 5y ago
    *and they’re not trying to make a video service for Kids
  - Mtinie 5y ago
    While I agree in part, at the rate new content is produced and uploaded to the platforms around the globe, could you even fill enough seats with reviewers at an economically-doable rate to keep up?
  - prionassembly 5y ago
    I mean, "not safe for work" doesn't mean "immoral". Beach volleyball isn't something you should be watching at work, at least not in the kind of open office plan where everybody sees your screen. There's a lot of talk about workplace "lad culture" being hostile to women these days -- possibly enough to mean liability; also you probably shouldn't be hostile to women in that specific way.
    - Chris2048 5y ago
      The idea that watching scantily-clad women s.a a Beach volleyball tournament is automatically hostile to women is just ingrained at this point. Maybe one day we'll return to normality.
    - tokai 5y ago
      I sometimes have mens cycling on while I work. This summer I'll properly have the womens tour de france on. They are almost as 'scantily' clad as beachvolley players. Though their bellies are covered. Is that NSFW or hostile to women? Is it okay to have the woman 100 meters dash on or is it as inappropriate as beach volley (they do wear almost as 'revealing' clothing)? Is mens sport okay? Maybe marathon and other sports with skinny clothed men is SFW, and not diving muscular men in tight panties is not?
      - tonmoy 5y ago
        I understand it may be subjective, but I find it hard to agree that women cyclists are as ‘scantily’ clad.
- some_random 5y ago
  We've seen plenty of author acknowledged research grade models and datasets get used in production, this is part of how black people get labeled as "monkey" every once in a while
deleted 5y ago
[deleted]
Cthulhu_ 5y ago
I want to believe it's a matter of adding human labor to the Pool of Knowledge in the AI engine(s); this is, this is not, go do the learning thing. But when it comes to online video checking, it's another tool in the arsenal; it's the first lines of defense. First check: does the signature match a previously known and confirmed video marked as porn or otherwise unacceptable. Second check: does the more fuzzy AI thing consider it porn with a high probability. Third check, which will be things the AI mark as 'not sure' or incorrectly marks as 'porn', and things humans flag up manually, will be humans checking and judging things. And of course it'll be flexible, because the definition of porn - or what is 'not acceptable' - will vary by country and culture, and these companies want to be active everywhere. except Europe, unless they are given tax breaks and data, amirite
zffr 5y ago
Even if this can't do a 100% job, isn't it still useful to automatically identify the obvious NSFW content?
jsharf 5y ago
yeah but this is true for profanity filters as well, and yet useful profanity filters still exist. Is it 100% accurate? No. But it's probably still useful.
- baobabKoodaa 5y ago
  Really? Can you point to a single example of a useful profanity filter?
  - baobabKoodaa 5y ago
    I'll take your silence as a "no".
wingman-jr 5y ago
You're right that stuff is quite difficult. I write a Firefox addon (https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-filter/ https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-fi..., https://github.com/wingman-jr-addon/wingman_jr https://github.com/wingman-jr-addon/wingman_jr) and train an associated NSFW model (https://github.com/wingman-jr-addon/model https://github.com/wingman-jr-addon/model) - I've been at it for a few years now, and have had to plug many specific edge cases. - Babies (https://github.com/wingman-jr-addon/wingman_jr/issues/22) - Beach volleyball (but this definitely has SFW and NSFW variants, based on a somewhat subjective line) - Athletes in general. The model particularly thought some American football players were NSFW for a long time. - Swimming - Yoga - again, most SFW and some NSFW here but it still struggles - Wrestling was a tough one for sure - Pokemon While indeed tough, I've seen definite progress. So it's not just a matter of tech, but also of considering the human element - the state of the art may not be up to the challenge of perfection, but it is definitely up to a point of true utility for some use cases. I'm happy about that. As a note, it uses an EfficientNet Lite L0 backbone - I'm a bit limited in what type of scanning I can perform in a sufficiently speedy manner. I also agree on the context for sure - one reason I haven't tried switching to an object detection method (and that I don't rely heavily on truly random crops) is that the focus of the image is highly important for the NSFW-ness in some cases. True, two images may contain the same content ... but one is far worse than the other. The nature of CNN's still has some of this location-invariance baked in, but I don't want to exacerbate it. One challenge I think the OP may run into here that may also not be immediately obvious is that accuracy on image stills does not translate that well to video. I have basic video support in my addon, and while I knew there would be some differences, I was surprised at how many discrepancies there really are. As two examples: - Images in video are often blurrier. In true still images, there is a somewhat higher prior involved with amateur NSFW content and blurriness. This can be a source of false positives. - The opposite of the note above about focus. Taking stills of moving images will have many transitory frames that seem inappropriate on their own because it seems as if they are focusing on something when in reality the camera is just panning - obvious to the human, less so to the model trained on stills. At any rate, given how well your list of edge cases coincided with failures I've grappled with, I'd be interested to see how well you think my addon stacks up for still images when set to stay in "normal" mode. I'd love to hear any feedback you have via GitHub so I can make it better.

by dsr_ 5y ago

lijogdfljk 5y ago

Why does everything have to be perfect? Humans get this wrong all the time as you say, and we still do it - right? Seems like we already acknowledge good enough practices and are okay with them. Lets not demand perfection in the same way we shouldn't assume perfection of ~~AI~~ ML.

nonameiguess 5y ago

It could potentially revolutionize the opposite application, those sites where someone manually finds and catalogs all the time points at which movies contain nudity so people can see their favorite celebs in the buff. False positives are probably worth the automation there.

FrostKiwi 5y ago

Fully agree - This seems like a project with rather limited in scope. And that's ok. The authors even acknowledge it. It's not an absolute judge of SFW'ness, neither does it claim to be. I can see it as a tool to skim a large set of videos for further human review. Besides, beach volleyball very much is NSFW depending on the situation.

rovr138 5y ago
>depending on the situation. That's part of the issue.
Chris2048 5y ago
> And that's ok. The authors even acknowledge it That doesn't always stop corps productizing it, and making it their official NSFW filter. Good enough is good enough, just so long as they can pay less salaries.
- joshspankit 5y ago
  *and they’re not trying to make a video service for Kids
- Mtinie 5y ago
  While I agree in part, at the rate new content is produced and uploaded to the platforms around the globe, could you even fill enough seats with reviewers at an economically-doable rate to keep up?
- prionassembly 5y ago
  I mean, "not safe for work" doesn't mean "immoral". Beach volleyball isn't something you should be watching at work, at least not in the kind of open office plan where everybody sees your screen. There's a lot of talk about workplace "lad culture" being hostile to women these days -- possibly enough to mean liability; also you probably shouldn't be hostile to women in that specific way.
  - Chris2048 5y ago
    The idea that watching scantily-clad women s.a a Beach volleyball tournament is automatically hostile to women is just ingrained at this point. Maybe one day we'll return to normality.
  - tokai 5y ago
    I sometimes have mens cycling on while I work. This summer I'll properly have the womens tour de france on. They are almost as 'scantily' clad as beachvolley players. Though their bellies are covered. Is that NSFW or hostile to women? Is it okay to have the woman 100 meters dash on or is it as inappropriate as beach volley (they do wear almost as 'revealing' clothing)? Is mens sport okay? Maybe marathon and other sports with skinny clothed men is SFW, and not diving muscular men in tight panties is not?
    - tonmoy 5y ago
      I understand it may be subjective, but I find it hard to agree that women cyclists are as ‘scantily’ clad.
some_random 5y ago
We've seen plenty of author acknowledged research grade models and datasets get used in production, this is part of how black people get labeled as "monkey" every once in a while

deleted 5y ago

[deleted]

Cthulhu_ 5y ago

I want to believe it's a matter of adding human labor to the Pool of Knowledge in the AI engine(s); this is, this is not, go do the learning thing. But when it comes to online video checking, it's another tool in the arsenal; it's the first lines of defense. First check: does the signature match a previously known and confirmed video marked as porn or otherwise unacceptable. Second check: does the more fuzzy AI thing consider it porn with a high probability. Third check, which will be things the AI mark as 'not sure' or incorrectly marks as 'porn', and things humans flag up manually, will be humans checking and judging things. And of course it'll be flexible, because the definition of porn - or what is 'not acceptable' - will vary by country and culture, and these companies want to be active everywhere. except Europe, unless they are given tax breaks and data, amirite

zffr 5y ago

Even if this can't do a 100% job, isn't it still useful to automatically identify the obvious NSFW content?

jsharf 5y ago

yeah but this is true for profanity filters as well, and yet useful profanity filters still exist. Is it 100% accurate? No. But it's probably still useful.

baobabKoodaa 5y ago
Really? Can you point to a single example of a useful profanity filter?
- baobabKoodaa 5y ago
  I'll take your silence as a "no".

wingman-jr 5y ago

You're right that stuff is quite difficult. I write a Firefox addon (https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-filter/ https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-fi..., https://github.com/wingman-jr-addon/wingman_jr https://github.com/wingman-jr-addon/wingman_jr) and train an associated NSFW model (https://github.com/wingman-jr-addon/model https://github.com/wingman-jr-addon/model) - I've been at it for a few years now, and have had to plug many specific edge cases. - Babies (https://github.com/wingman-jr-addon/wingman_jr/issues/22) - Beach volleyball (but this definitely has SFW and NSFW variants, based on a somewhat subjective line) - Athletes in general. The model particularly thought some American football players were NSFW for a long time. - Swimming - Yoga - again, most SFW and some NSFW here but it still struggles - Wrestling was a tough one for sure - Pokemon While indeed tough, I've seen definite progress. So it's not just a matter of tech, but also of considering the human element - the state of the art may not be up to the challenge of perfection, but it is definitely up to a point of true utility for some use cases. I'm happy about that. As a note, it uses an EfficientNet Lite L0 backbone - I'm a bit limited in what type of scanning I can perform in a sufficiently speedy manner. I also agree on the context for sure - one reason I haven't tried switching to an object detection method (and that I don't rely heavily on truly random crops) is that the focus of the image is highly important for the NSFW-ness in some cases. True, two images may contain the same content ... but one is far worse than the other. The nature of CNN's still has some of this location-invariance baked in, but I don't want to exacerbate it. One challenge I think the OP may run into here that may also not be immediately obvious is that accuracy on image stills does not translate that well to video. I have basic video support in my addon, and while I knew there would be some differences, I was surprised at how many discrepancies there really are. As two examples: - Images in video are often blurrier. In true still images, there is a somewhat higher prior involved with amateur NSFW content and blurriness. This can be a source of false positives. - The opposite of the note above about focus. Taking stills of moving images will have many transitory frames that seem inappropriate on their own because it seems as if they are focusing on something when in reality the camera is just panning - obvious to the human, less so to the model trained on stills. At any rate, given how well your list of edge cases coincided with failures I've grappled with, I'd be interested to see how well you think my addon stacks up for still images when set to stay in "normal" mode. I'd love to hear any feedback you have via GitHub so I can make it better.