7 ms·

Reminds me of a toy girl doll I heard about which had a speech generator which you could program to say sentences but had "harmful" words removed, keeping only

Reminds me of a toy girl doll I heard about which had a speech generator which you could program to say sentences but had "harmful" words removed, keeping only wholesome ones. I immediately came up with "Call the football team, I'm wet" and "Daddy lets play hide the sausage" as example workarounds. It's entirely pointless. Humans are vastly superior in their ability to subvert and corrupt. Even if you were able to catch regular "harmful" images humans would create a new categories of imagery which people would experience as "harmful", employ allusions, illusions, proxies, irony etc. It's endless.

torotonnato 4y ago
Another example of funny subvertion, Chinese style: https://www.wikiwand.com/en/Baidu_10_Mythical_Creatures https://www.wikiwand.com/en/Baidu_10_Mythical_Creatures
worldsayshi 4y ago
Sure it would be a fools errand to filter out "harmful" speech using traditional algorithms. But neutral networks and beyond seems like exactly the kind of technology that is able to respond to fuzzy concepts rather than just sets of words. Sure it will be a long hunt but if it can learn to paint and recognize a myriad of visual concepts it ought to be able to learn what we consider to be harmful.
- ziddoap 4y ago
  One of the insurmountable problems, I think, is the fact that different people (and different cultures) consider different things 'harmful', and to varying degrees of harm, and what is considered harmful changes over time. What is harmful is also often context-dependent. Complicating matters more is the fact that something being censored can be considered harmful as well. Religious messages would be a good example of this; Religion A thinks that Religion B is harmful, and vice-versa. I doubt any 'neutral network' can resolve that problem without the decision itself being harmful to some subset of people. While I love the developments in machine learning/neural networks/etc. right now, I think it's a bit early to put that much faith in them (to the point where we think they can solve such a problem like "ban all the harmful things").
  - JohnHaugeland 4y ago
    This doesn't actually affect the system at all There's way too much moralizing from people who have no idea what's going on All the filter actually is is an object recognizer trained on genital images, and it can be turned off The prompt isn't very relevant. I've had the filter fire on completely innocent text The filter is a simple checkbox in preferences. All this deep thought is missing the point. You can just turn it off
    - ziddoap 4y ago
      >There's way too much moralizing from people who have no idea what's going on >All the filter actually is is an object recognizer trained on genital images, and it can be turned off I'm not sure if you misread something, but neither I or the person I was replying to was talking about this specific implementation, but in a more general sense? I'm pretty sure you are the one who missed the point of the parent post and mine.
      - JohnHaugeland 4y ago
        Cool story. Anyway, the filters you're thinking about don't exist and you can download the code and use it today. Thanks for speculating.
    - epups 4y ago
      It's not that simple. The model was not trained to recognize "harmful" action such as blowjobs (although "bombing" and other atrocities of course are there).
      - JohnHaugeland 4y ago
        It very much is, actually, that simple The model was trained on eight specific body parts. If it doesn't see those, it doesn't fire. That's 100% of the job. I see that you've managed to name things that you think aren't in the model. That's nice. That's not related to what this company did, though. You seem to be confusing how you think a system like this might work with what this company clearly explained as what they did. This isn't hypothetical. You can just go to their webpage and look. The NSFW filter on Stable Diffusion is simply an image body part recognizer run against the generated image. It has nothing to do with the prompt text at all.
- CrazyStat 4y ago
  >it ought to be able to learn what we consider to be harmful. Who is "we"? Do "we" consider nude paintings to be harmful? Is "we" Mike Pence? Roman Polanski? Woody Allen? There is no coherent "we" and no consensus on what "we" consider harmful, so no AI can possibly learn that.
  - worldsayshi 4y ago
    Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Isn't this part of the AI alignment problem? To be able to understand what kinds of output is unacceptable for a certain audience? To be polite?
    - CrazyStat 4y ago
      > Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Do we want the AI to generate based on Polanski's sensibilities, even if he's the only audience member? I suspect for most people the answer is no.
    - macrolime 4y ago
      No
zzleeper 4y ago
I naively asked for a "sperm whale opening its mouth in the middle of the ocean" on DALL-E and got a warning :/
- dnissley 4y ago
canjobear 4y ago
Furthermore, the possibility that we create an AI that can outsmart humans in terms of filtering inappropriate content is even scarier. Do you really want a world with an AI censor of superhuman intelligence gatekeeping the means of content creation?
- AndrewKemendo 4y ago
  If you squint and view the modern corporation as a proxy for "an AI censor of superhuman intelligence gatekeeping the means of content creation" - then that's been happening for a long while now. Automatic review of content, NSFW filters, SPAM filters etc... have been bog standard since the earliest days of the internet. I don't think anyone likes it. Some fight it and create their own spaces that allow certain types of content. Most people accept it though and move on with their lives
  - pverghese 4y ago
    That's the government forcing corporations to do that
    - AndrewKemendo 4y ago
      Normally I don't respond but this is just false SPAM filters on SMTP ports were implemented long long before any government mandated it - at the ISP level often Further, the development DKIM and SPF, were incidental to any government requirement Preemptively: The fact that the early internet was heavily government and institutionally focused, doesn't a government mandate make
      - Proven 4y ago
    - kelseyfrog 4y ago
      We can look back further at the Hays code. That's just religion plain and simple. The feeling of, "we're sliding into a decadence which will lead to the downfall of our civilization" is a meme propagating this very sentiment. It's not a simple as just the government, but that does co-occur.
dumpsterlid 4y ago
"It's entirely pointless. Humans are vastly superior in their ability to subvert and corrupt. Even if you were able to catch regular "harmful" images humans would create a new categories of imagery which people would experience as "harmful", employ allusions, illusions, proxies, irony etc. It's endless." This is employing a fallacy that people have infinite amounts of energy and motivation to devote to being hateful. I have been on countless online communities in video games and elsewhere and when the chat in them doesnt allow you to say toxic, hateful stuff... guess what a whole lot less of that shit is said. Are there people who get around it by changing out characters to ones that look the same that dont trigger the censor or by using slang or by mispelling? Of course but the fact is I think if you talk to someone who runs communities like this they would laugh in your face if you said a degree of censorship of hate speech wasn't fundamentally beneficial. A big aspect has got to do with the fact that if everybody agrees to be part of a community, part of that agreement is a social contract not to use hate speech and if someone flaunts that they are bypassing it.. in the obvious flaunting of the social contract established (it is obvious they had to purposely mispell the word) these people are alienating themselves by underlining the fact that the 99% of the community finds their behavior pathetic and unacceptable.
- XorNot 4y ago
  Importantly, we're posting on a forum where this exact idea is true. HN doesn't stop all hate speech, or flaming and what have...but the moderation system stops enough that people generally don't bother.
  - angusturner 4y ago
    It seems pretty well-agreed that the HN moderation works because of dedicated human moderators and community guidelines etc. I think spaces that effectively moderate AI art content will be successful (or not) based on these same factors. It won't depend on some brittle technology for predicting if something is harmful or NSFW. (Which, incidentally, people will use to optimize/find NSFW content specifically, as they already do with Stable Diffusion).
  - epups 4y ago
AndrewKemendo 4y ago
I would argue that a group of humans is actually superhuman at corrupting and subverting any given system What a great insight which I find both extremely poignant and simultaneously disheartening Perhaps this will come as a comfort to the people who are vehemently against creating human-level AI systems
pfortuny 4y ago
In England you discover that English has actually two different existences… The ordinary one and then the “dirty” one. Almost any word has or can be made to have a “harmful” meaning… Useless effort.
rich_sasha 4y ago
Cue aubergine and peach emoji.
arvinsim 4y ago
Agreed. Anything can be lewd if one has a "green" mind.

torotonnato 4y ago

Another example of funny subvertion, Chinese style: https://www.wikiwand.com/en/Baidu_10_Mythical_Creatures https://www.wikiwand.com/en/Baidu_10_Mythical_Creatures

worldsayshi 4y ago

Sure it would be a fools errand to filter out "harmful" speech using traditional algorithms. But neutral networks and beyond seems like exactly the kind of technology that is able to respond to fuzzy concepts rather than just sets of words. Sure it will be a long hunt but if it can learn to paint and recognize a myriad of visual concepts it ought to be able to learn what we consider to be harmful.

ziddoap 4y ago
One of the insurmountable problems, I think, is the fact that different people (and different cultures) consider different things 'harmful', and to varying degrees of harm, and what is considered harmful changes over time. What is harmful is also often context-dependent. Complicating matters more is the fact that something being censored can be considered harmful as well. Religious messages would be a good example of this; Religion A thinks that Religion B is harmful, and vice-versa. I doubt any 'neutral network' can resolve that problem without the decision itself being harmful to some subset of people. While I love the developments in machine learning/neural networks/etc. right now, I think it's a bit early to put that much faith in them (to the point where we think they can solve such a problem like "ban all the harmful things").
- JohnHaugeland 4y ago
  This doesn't actually affect the system at all There's way too much moralizing from people who have no idea what's going on All the filter actually is is an object recognizer trained on genital images, and it can be turned off The prompt isn't very relevant. I've had the filter fire on completely innocent text The filter is a simple checkbox in preferences. All this deep thought is missing the point. You can just turn it off
  - ziddoap 4y ago
    >There's way too much moralizing from people who have no idea what's going on >All the filter actually is is an object recognizer trained on genital images, and it can be turned off I'm not sure if you misread something, but neither I or the person I was replying to was talking about this specific implementation, but in a more general sense? I'm pretty sure you are the one who missed the point of the parent post and mine.
    - JohnHaugeland 4y ago
      Cool story. Anyway, the filters you're thinking about don't exist and you can download the code and use it today. Thanks for speculating.
  - epups 4y ago
    It's not that simple. The model was not trained to recognize "harmful" action such as blowjobs (although "bombing" and other atrocities of course are there).
    - JohnHaugeland 4y ago
      It very much is, actually, that simple The model was trained on eight specific body parts. If it doesn't see those, it doesn't fire. That's 100% of the job. I see that you've managed to name things that you think aren't in the model. That's nice. That's not related to what this company did, though. You seem to be confusing how you think a system like this might work with what this company clearly explained as what they did. This isn't hypothetical. You can just go to their webpage and look. The NSFW filter on Stable Diffusion is simply an image body part recognizer run against the generated image. It has nothing to do with the prompt text at all.
CrazyStat 4y ago
>it ought to be able to learn what we consider to be harmful. Who is "we"? Do "we" consider nude paintings to be harmful? Is "we" Mike Pence? Roman Polanski? Woody Allen? There is no coherent "we" and no consensus on what "we" consider harmful, so no AI can possibly learn that.
- worldsayshi 4y ago
  Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Isn't this part of the AI alignment problem? To be able to understand what kinds of output is unacceptable for a certain audience? To be polite?
  - CrazyStat 4y ago
    > Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Do we want the AI to generate based on Polanski's sensibilities, even if he's the only audience member? I suspect for most people the answer is no.
  - macrolime 4y ago
    No

zzleeper 4y ago

I naively asked for a "sperm whale opening its mouth in the middle of the ocean" on DALL-E and got a warning :/

dnissley 4y ago

canjobear 4y ago

Furthermore, the possibility that we create an AI that can outsmart humans in terms of filtering inappropriate content is even scarier. Do you really want a world with an AI censor of superhuman intelligence gatekeeping the means of content creation?

AndrewKemendo 4y ago
If you squint and view the modern corporation as a proxy for "an AI censor of superhuman intelligence gatekeeping the means of content creation" - then that's been happening for a long while now. Automatic review of content, NSFW filters, SPAM filters etc... have been bog standard since the earliest days of the internet. I don't think anyone likes it. Some fight it and create their own spaces that allow certain types of content. Most people accept it though and move on with their lives
- pverghese 4y ago
  That's the government forcing corporations to do that
  - AndrewKemendo 4y ago
    Normally I don't respond but this is just false SPAM filters on SMTP ports were implemented long long before any government mandated it - at the ISP level often Further, the development DKIM and SPF, were incidental to any government requirement Preemptively: The fact that the early internet was heavily government and institutionally focused, doesn't a government mandate make
    - Proven 4y ago
  - kelseyfrog 4y ago
    We can look back further at the Hays code. That's just religion plain and simple. The feeling of, "we're sliding into a decadence which will lead to the downfall of our civilization" is a meme propagating this very sentiment. It's not a simple as just the government, but that does co-occur.

dumpsterlid 4y ago

"It's entirely pointless. Humans are vastly superior in their ability to subvert and corrupt. Even if you were able to catch regular "harmful" images humans would create a new categories of imagery which people would experience as "harmful", employ allusions, illusions, proxies, irony etc. It's endless." This is employing a fallacy that people have infinite amounts of energy and motivation to devote to being hateful. I have been on countless online communities in video games and elsewhere and when the chat in them doesnt allow you to say toxic, hateful stuff... guess what a whole lot less of that shit is said. Are there people who get around it by changing out characters to ones that look the same that dont trigger the censor or by using slang or by mispelling? Of course but the fact is I think if you talk to someone who runs communities like this they would laugh in your face if you said a degree of censorship of hate speech wasn't fundamentally beneficial. A big aspect has got to do with the fact that if everybody agrees to be part of a community, part of that agreement is a social contract not to use hate speech and if someone flaunts that they are bypassing it.. in the obvious flaunting of the social contract established (it is obvious they had to purposely mispell the word) these people are alienating themselves by underlining the fact that the 99% of the community finds their behavior pathetic and unacceptable.

XorNot 4y ago
Importantly, we're posting on a forum where this exact idea is true. HN doesn't stop all hate speech, or flaming and what have...but the moderation system stops enough that people generally don't bother.
- angusturner 4y ago
  It seems pretty well-agreed that the HN moderation works because of dedicated human moderators and community guidelines etc. I think spaces that effectively moderate AI art content will be successful (or not) based on these same factors. It won't depend on some brittle technology for predicting if something is harmful or NSFW. (Which, incidentally, people will use to optimize/find NSFW content specifically, as they already do with Stable Diffusion).
- epups 4y ago

AndrewKemendo 4y ago

I would argue that a group of humans is actually superhuman at corrupting and subverting any given system What a great insight which I find both extremely poignant and simultaneously disheartening Perhaps this will come as a comfort to the people who are vehemently against creating human-level AI systems

pfortuny 4y ago

In England you discover that English has actually two different existences… The ordinary one and then the “dirty” one. Almost any word has or can be made to have a “harmful” meaning… Useless effort.

rich_sasha 4y ago

Cue aubergine and peach emoji.

arvinsim 4y ago

Agreed. Anything can be lewd if one has a "green" mind.