4 ms·

Sure it would be a fools errand to filter out "harmful" speech using traditional algorithms. But neutral networks and beyond seems like exactly the kind of tech

Sure it would be a fools errand to filter out "harmful" speech using traditional algorithms. But neutral networks and beyond seems like exactly the kind of technology that is able to respond to fuzzy concepts rather than just sets of words. Sure it will be a long hunt but if it can learn to paint and recognize a myriad of visual concepts it ought to be able to learn what we consider to be harmful.

ziddoap 4y ago
One of the insurmountable problems, I think, is the fact that different people (and different cultures) consider different things 'harmful', and to varying degrees of harm, and what is considered harmful changes over time. What is harmful is also often context-dependent. Complicating matters more is the fact that something being censored can be considered harmful as well. Religious messages would be a good example of this; Religion A thinks that Religion B is harmful, and vice-versa. I doubt any 'neutral network' can resolve that problem without the decision itself being harmful to some subset of people. While I love the developments in machine learning/neural networks/etc. right now, I think it's a bit early to put that much faith in them (to the point where we think they can solve such a problem like "ban all the harmful things").
- JohnHaugeland 4y ago
  This doesn't actually affect the system at all There's way too much moralizing from people who have no idea what's going on All the filter actually is is an object recognizer trained on genital images, and it can be turned off The prompt isn't very relevant. I've had the filter fire on completely innocent text The filter is a simple checkbox in preferences. All this deep thought is missing the point. You can just turn it off
  - ziddoap 4y ago
    >There's way too much moralizing from people who have no idea what's going on >All the filter actually is is an object recognizer trained on genital images, and it can be turned off I'm not sure if you misread something, but neither I or the person I was replying to was talking about this specific implementation, but in a more general sense? I'm pretty sure you are the one who missed the point of the parent post and mine.
    - JohnHaugeland 4y ago
      Cool story. Anyway, the filters you're thinking about don't exist and you can download the code and use it today. Thanks for speculating.
  - epups 4y ago
    It's not that simple. The model was not trained to recognize "harmful" action such as blowjobs (although "bombing" and other atrocities of course are there).
    - JohnHaugeland 4y ago
      It very much is, actually, that simple The model was trained on eight specific body parts. If it doesn't see those, it doesn't fire. That's 100% of the job. I see that you've managed to name things that you think aren't in the model. That's nice. That's not related to what this company did, though. You seem to be confusing how you think a system like this might work with what this company clearly explained as what they did. This isn't hypothetical. You can just go to their webpage and look. The NSFW filter on Stable Diffusion is simply an image body part recognizer run against the generated image. It has nothing to do with the prompt text at all.
      - epups 4y ago
        The company filtered the LAION 5b based on undisclosed criteria. So what you are saying is actually irrelevant, as we do not know what pictures were included or not. It is obvious to anyone who bothers to try - have you? - that a filter was placed here at the training level. Rare activities such as "Kitesurfing" produces flawless, accurate pictures, whereas anything sexual or remotely lewd ("peeing") doesn't. This is a conscious decision by whoever produced this model.
        JohnHaugeland 4y ago
        If you're done, emad already discussed this
        epups 4y ago
        Who the hell is emad
        JohnHaugeland 4y ago
CrazyStat 4y ago
>it ought to be able to learn what we consider to be harmful. Who is "we"? Do "we" consider nude paintings to be harmful? Is "we" Mike Pence? Roman Polanski? Woody Allen? There is no coherent "we" and no consensus on what "we" consider harmful, so no AI can possibly learn that.
- worldsayshi 4y ago
  Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Isn't this part of the AI alignment problem? To be able to understand what kinds of output is unacceptable for a certain audience? To be polite?
  - CrazyStat 4y ago
    > Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Do we want the AI to generate based on Polanski's sensibilities, even if he's the only audience member? I suspect for most people the answer is no.
  - macrolime 4y ago
    No

ziddoap 4y ago

One of the insurmountable problems, I think, is the fact that different people (and different cultures) consider different things 'harmful', and to varying degrees of harm, and what is considered harmful changes over time. What is harmful is also often context-dependent. Complicating matters more is the fact that something being censored can be considered harmful as well. Religious messages would be a good example of this; Religion A thinks that Religion B is harmful, and vice-versa. I doubt any 'neutral network' can resolve that problem without the decision itself being harmful to some subset of people. While I love the developments in machine learning/neural networks/etc. right now, I think it's a bit early to put that much faith in them (to the point where we think they can solve such a problem like "ban all the harmful things").

JohnHaugeland 4y ago
This doesn't actually affect the system at all There's way too much moralizing from people who have no idea what's going on All the filter actually is is an object recognizer trained on genital images, and it can be turned off The prompt isn't very relevant. I've had the filter fire on completely innocent text The filter is a simple checkbox in preferences. All this deep thought is missing the point. You can just turn it off
- ziddoap 4y ago
  >There's way too much moralizing from people who have no idea what's going on >All the filter actually is is an object recognizer trained on genital images, and it can be turned off I'm not sure if you misread something, but neither I or the person I was replying to was talking about this specific implementation, but in a more general sense? I'm pretty sure you are the one who missed the point of the parent post and mine.
  - JohnHaugeland 4y ago
    Cool story. Anyway, the filters you're thinking about don't exist and you can download the code and use it today. Thanks for speculating.
- epups 4y ago
  It's not that simple. The model was not trained to recognize "harmful" action such as blowjobs (although "bombing" and other atrocities of course are there).
  - JohnHaugeland 4y ago
    It very much is, actually, that simple The model was trained on eight specific body parts. If it doesn't see those, it doesn't fire. That's 100% of the job. I see that you've managed to name things that you think aren't in the model. That's nice. That's not related to what this company did, though. You seem to be confusing how you think a system like this might work with what this company clearly explained as what they did. This isn't hypothetical. You can just go to their webpage and look. The NSFW filter on Stable Diffusion is simply an image body part recognizer run against the generated image. It has nothing to do with the prompt text at all.
    - epups 4y ago
      The company filtered the LAION 5b based on undisclosed criteria. So what you are saying is actually irrelevant, as we do not know what pictures were included or not. It is obvious to anyone who bothers to try - have you? - that a filter was placed here at the training level. Rare activities such as "Kitesurfing" produces flawless, accurate pictures, whereas anything sexual or remotely lewd ("peeing") doesn't. This is a conscious decision by whoever produced this model.
      - JohnHaugeland 4y ago
        If you're done, emad already discussed this
        epups 4y ago
        Who the hell is emad
        JohnHaugeland 4y ago

CrazyStat 4y ago

>it ought to be able to learn what we consider to be harmful. Who is "we"? Do "we" consider nude paintings to be harmful? Is "we" Mike Pence? Roman Polanski? Woody Allen? There is no coherent "we" and no consensus on what "we" consider harmful, so no AI can possibly learn that.

worldsayshi 4y ago
Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Isn't this part of the AI alignment problem? To be able to understand what kinds of output is unacceptable for a certain audience? To be polite?
- CrazyStat 4y ago
  > Well it ought to be able to be trained for a number of scenarios and then on generation be told to generate based on certain cultural sensibilities. It's not going to be perfect but probably good enough? Do we want the AI to generate based on Polanski's sensibilities, even if he's the only audience member? I suspect for most people the answer is no.
- macrolime 4y ago
  No