5 ms·

This raises a good question. Is it the model that produces racist, sexist or any other ‘ist ’ results inherently racist? Or is it how we use it? Is the knife a

This raises a good question. Is it the model that produces racist, sexist or any other ‘ist ’ results inherently racist? Or is it how we use it? Is the knife a murderer or the person who wields it?

ivraatiems 4y ago
The issue is almost certainly that if the model is racist, it's cause it was trained on racist data. Which is 100% a real thing that happens. What I disagree with is the idea that it means we can learn nothing from or should not disseminate the model. We need to correct our mistakes! And also, maybe we need a model that can identify racist content if we want to identify racist content better?
- bergenty 4y ago
  That’s not true. You can specify SD create a racist meme out of a description that isn’t inherently racist.
  - incrudible 4y ago
    Fair enough, but what does that imply? The magic N-Ball that has a minor chance of saying something unintentionally racist is such dangerous technology that it must be kept out of the hands of the peasants?
bergenty 4y ago
This is more like having access to a factory that makes guns. Except the guns are racist and sexist memes.
- ivraatiems 4y ago
  Is it hard to make racist and sexist memes now?
  - austinjp 4y ago
    The point is surely that it's now potentially far easier to programmatically generate new offensive memes at scale.
    - ivraatiems 4y ago
      That doesn't seem a) particularly useful or b) like it was hard to do before. I could easily gather a corpus of racist slogans, a corpus of racist images, and smash them together to get a few thousand racist memes. It would probably take less time than relying on an AI model to generate each one (which takes a few seconds to a few minutes each). And it's not as though there aren't millions of actual racists out there spewing out racist content constantly anyway.
austinjp 4y ago
The archetype of this argument is of course the (in)famous "P doesn't Q people, people Q people" * While this is "true", the sense in which it is true is so limited as to be entirely unhelpful. If you manufacture P, and you know Q may be an outcome, why are you manufacturing P, and how are you preventing Q? * Where, as we all know, Q is usually "kill" or "harm", and P might be "guns" or "autonomous vehicles" or "military robots" or "facial recognition" etc.
- austinjp 4y ago
  I see this has attracted some downvotes, which is of course fine. However, it would really help improve the quality of this conversation if people could reply here, and perhaps explain objections to the archetype I've described. Perhaps it was just my overly convoluted writing style? :) If though it was about the content... The statement "P doesn't Q people, people Q people" is absolutely devoid of any useful information or novel insight, and doesn't take the conversation in a new direction that is useful. In fact, it can kick the conversation into an anti-productive quagmire. Let's see... "Stable Diffusion doesn't produce racist memes, people produce racist memes." Well duh. A more useful conversation might be about how we protect against possible automated mass-generation of racist messages (or whatever), what roles SD et al have to play, how we deal with possible outcomes, etc etc. For what it's worth, I do not think Stable Diffusion should be kept under wraps. Paradigm-shifting technology should be discussed in the open. Developers/engineers should be prepared to walk away from anything with a significant downside if that downside hasn't been exposed to thorough debate. And we should be prepared to shoulder responsibility when shiny new toys are used to do terrible things.
  - iszomer 4y ago
    > The statement "P doesn't Q people, people Q people" is absolutely devoid of any useful information or novel insight, and doesn't take the conversation in a new direction that is useful. In fact, it can kick the conversation into an anti-productive quagmire. Are you presuming everyone reading and responding to this thread are on your level of interpretation or analytical superiority? This is the first time reading about "P doesn't Q people, people Q people" on HN and I imagine won't be the last. There is no "In fact," here and it already rubbed off as a completely pretentious statement, which given the circumstances is not unusual in a place like this. (I didn't downvote you but am expecting a downvote)

ivraatiems 4y ago

The issue is almost certainly that if the model is racist, it's cause it was trained on racist data. Which is 100% a real thing that happens. What I disagree with is the idea that it means we can learn nothing from or should not disseminate the model. We need to correct our mistakes! And also, maybe we need a model that can identify racist content if we want to identify racist content better?

bergenty 4y ago
That’s not true. You can specify SD create a racist meme out of a description that isn’t inherently racist.
- incrudible 4y ago
  Fair enough, but what does that imply? The magic N-Ball that has a minor chance of saying something unintentionally racist is such dangerous technology that it must be kept out of the hands of the peasants?

bergenty 4y ago

This is more like having access to a factory that makes guns. Except the guns are racist and sexist memes.

ivraatiems 4y ago
Is it hard to make racist and sexist memes now?
- austinjp 4y ago
  The point is surely that it's now potentially far easier to programmatically generate new offensive memes at scale.
  - ivraatiems 4y ago
    That doesn't seem a) particularly useful or b) like it was hard to do before. I could easily gather a corpus of racist slogans, a corpus of racist images, and smash them together to get a few thousand racist memes. It would probably take less time than relying on an AI model to generate each one (which takes a few seconds to a few minutes each). And it's not as though there aren't millions of actual racists out there spewing out racist content constantly anyway.

austinjp 4y ago

The archetype of this argument is of course the (in)famous "P doesn't Q people, people Q people" * While this is "true", the sense in which it is true is so limited as to be entirely unhelpful. If you manufacture P, and you know Q may be an outcome, why are you manufacturing P, and how are you preventing Q? * Where, as we all know, Q is usually "kill" or "harm", and P might be "guns" or "autonomous vehicles" or "military robots" or "facial recognition" etc.

austinjp 4y ago
I see this has attracted some downvotes, which is of course fine. However, it would really help improve the quality of this conversation if people could reply here, and perhaps explain objections to the archetype I've described. Perhaps it was just my overly convoluted writing style? :) If though it was about the content... The statement "P doesn't Q people, people Q people" is absolutely devoid of any useful information or novel insight, and doesn't take the conversation in a new direction that is useful. In fact, it can kick the conversation into an anti-productive quagmire. Let's see... "Stable Diffusion doesn't produce racist memes, people produce racist memes." Well duh. A more useful conversation might be about how we protect against possible automated mass-generation of racist messages (or whatever), what roles SD et al have to play, how we deal with possible outcomes, etc etc. For what it's worth, I do not think Stable Diffusion should be kept under wraps. Paradigm-shifting technology should be discussed in the open. Developers/engineers should be prepared to walk away from anything with a significant downside if that downside hasn't been exposed to thorough debate. And we should be prepared to shoulder responsibility when shiny new toys are used to do terrible things.
- iszomer 4y ago
  > The statement "P doesn't Q people, people Q people" is absolutely devoid of any useful information or novel insight, and doesn't take the conversation in a new direction that is useful. In fact, it can kick the conversation into an anti-productive quagmire. Are you presuming everyone reading and responding to this thread are on your level of interpretation or analytical superiority? This is the first time reading about "P doesn't Q people, people Q people" on HN and I imagine won't be the last. There is no "In fact," here and it already rubbed off as a completely pretentious statement, which given the circumstances is not unusual in a place like this. (I didn't downvote you but am expecting a downvote)