3 ms·

4chan users manipulate AI tools to unleash torrent of racist images

17 points by CharlesW 3y ago

mouse_ 3y ago
gijames1225 said: I'm a little curious as to why this is even an article. It's inevitable and has no bearing on 99.99999% of people. I get that it's low hanging fruit from a reporting / research perspective (sort of like the 'Twitter users blah blah blah' articles), but outside of the 'publish or die' mindset of churning out content this is pointless and continues to make 4-chan and these people seem much more relevant than they are.
- brucethemoose2 3y ago
  > It's inevitable and has no bearing on 99.99999% of people. I dunno about that. 4chan memes and attitudes have diffused out into wider US politics. Not that I disagree with the jist of it. "Oh no, internet dwellers are posting something horrible... I am shocked!"
  - aydyn 3y ago
    > 4chan memes and attitudes have diffused out into wider US politics. Keyword _have_. 4chan hasn't been culturally relevant for like a decade.
- JKCalhoun 3y ago
  I need a reminder from time to time that the internet has become a dumpster fire. These add fuel to the internal debate I am always having as to whether we ought to just toss the whole stinking thing away and go back to a world dominated by radio [1]. [1] (Not that I have of course any means to do that ... but I can drop out I suppose.)
  - bitwize 3y ago
    What we need to do is build a progressive political orientation into all layers of the network, from site content moderation to actual packet routing. To quote Drew Devault, "no Nazis" is a better policy than "no politics".
    - throwawa14223 3y ago
      Why would any person adopt packet routing that had built in progressive political orientation?
    - catiopatio 3y ago
      Poe’s law strikes again. I legit cannot determine whether your comment is satire or genuine.
    - wmidwestranger 3y ago
      Somehow, at this moment, industry having massive influence on government policy is not fascist but a guy in the UK teaching his dog to salute like a Nazi is.
    - richbell 3y ago
      > To quote Drew Devault, "no Nazis" is a better policy than "no politics". I have a lot of respect for Drew, but I can't take absolutist opinions like this seriously from someone who reputedly blocks people over any minor disagreement or perceived sleight. Regardless, "No Nazis" is a great policy until someone decides they don't like you and labels you as one. The most prevailing anti-Ukraine sentiment is based on the claim that they are Nazis.
  - wmidwestranger 3y ago
    As distasteful as memes can be; the slow, steady, and ever more complete of government and corporate control is just as sad to me. Maybe even worse because a lot of the distasteful stuff is made by people with bad taste while the plasticising of the internet is done for profit or influence. The Eternal September, done deliberately. There is an interview between Mike Wallace and the Chinese leader in the 90s where Wallace asks the leader why they censor the internet. He specifically says, "don't you think your people are smart enough to make up their own minds and come to their own conclusions?" So it goes, now the entire world is treated the same way.
  - brucethemoose2 3y ago
    > I need a reminder from time to time that the internet has become a dumpster fire The internet was always a dumpster fire. 4chan is old, toxic forums are even older. That is part of the package. What made it cool was the freeness and gems, and the little adequately moderated niche communities. I would argue the bigger problem is that the internet became siloed, bland, soullessly moderated by corporations who don't care, and excessively spammy.
    - richbell 3y ago
      > The internet was always a dumpster fire. 4chan is old, toxic forums are even older. That is part of the package. Lest we forget that a favorite pastime during the aughts was tricking your friends or strangers into visiting vile "shock" websites.
  - shitlord 3y ago
    Most of the internet is not like this, and most people using AI aren't using it in this way. Sensationalism sells, and media outlets have a financial interest in blowing these things out of proportion to get readers riled up.
- bitwize 3y ago
  Racist content has a bearing on the marginalized races. Ignoring the casual hate on 4chan brought us GamerGate and Trump.
  - ecshafer 3y ago
    I thought deindustrialization and free trade brought us trump not memes? But maybe I made a mistake by listening to what trump supporters said.
  - ToValueFunfetti 3y ago
    As far as I'm aware, nobody got physically hurt from Gamergate, ie. ignoring casual hate would have completely removed its impact. Though you could argue that that was impossible. Do you have evidence for your claim that ignoring casual hate caused either of those things? My baseless theory is that suppressing and excluding these people from the mainstream is what riled them up in the first place; it certainly didn't feel like they were being ignored even after that. But I am open to contradicting evidence.
  - richbell 3y ago
    People losing trust in and feeling ignored by politicians is what brought us Trump. Some people voted for him because they liked that he wasn't a typical "politician", other people voted for him out of protest.
- marsupial 3y ago
  The news cycle essentially is 'publish or die' and race topics get clicks. It's also bad PR and pressures the AI monopoly to bake shitlib orthodoxy into their models, to make them less transparent and hint regulation. Etc.
brucethemoose2 3y ago
> judging by the images’ default square format, the uniform 1024 x 1024 resolution This is also the default for Stable Diffusion XL.
gnabgib 3y ago
The original article, constantly referenced by this limited piece, is probably a better submission? https://www.404media.co/4chan-uses-bing-to-flood-the-internet-with-racist-images/ https://www.404media.co/4chan-uses-bing-to-flood-the-interne... It was submitted 8 hours ago, although it didn't get much notice https://news.ycombinator.com/item?id=37779810 https://news.ycombinator.com/item?id=37779810
mellosouls 3y ago
The original article is here and linked in the first paragraph of the ArsT one: https://www.404media.co/4chan-uses-bing-to-flood-the-internet-with-racist-images/ https://www.404media.co/4chan-uses-bing-to-flood-the-interne...
RIMR 3y ago
Kind of silly to ask Stability AI to block offensive use of Stable Diffusion. It's not like they have any control over what models people load into their own instances of it. The base model that they distribute is pretty clean, but even then how are you supposed to stop people from photoshopping hateful messages into them?
2OEH8eoCRo0 3y ago
Revisted 4chan the other day for the first time in years and it was awash in anti-semitic shit. Caricatures of jews with red eyes. It was interesting to see some of the threads taken over by AI generated cats killing nazis so the 4chan silliness was still there.
- richbell 3y ago
  4chan is a large community of different boards. Which one did you visit and see that stuff on?
  - 2OEH8eoCRo0 3y ago
    /pol/
ThrowawayTestr 3y ago
No manipulation needed dalle is very accommodating.
chmod775 3y ago
In today's news: 4chan users who spam racist garbage, not because they are racist, but because they want to create chaos and are fishing for a reaction, get a reaction. This is like writing newspaper articles about toilet graffiti. You're making the artist's day.
- skulk 3y ago
  I know it brings you comfort to think they're just doing it for the lolz but unfortunately the reality is that a good chunk of them actually believe those things.
  - chmod775 3y ago
    Or that's what they want you to believe - because obviously that's more lolz. Though maybe there's some that actually are, because of: https://news.ycombinator.com/item?id=1012082 https://news.ycombinator.com/item?id=1012082 Ask yourself why an actual racist without any intention to cause chaos would engage in this behavior though. It's clearly intended to offend and get a reaction - that's the main goal. Whether it's done by actual racists is secondary.
    - krapp 3y ago
      Also known as Popehat's Law of Goats: He who fucks goats, either as part of a performance or to troll those he deems has overly delicate sensibilities is simply, a goatfucker. If your goal is to be seen as a racist, and to provoke a response from people who are offended by racists, and you do all of the things that racists do to that end, spending time on racist forums spreading racist memes and hate speech, then the distinction between you being an "actual" racist and your racism being mere performance is one without a difference. Because being racist as a hobby isn't really something non-racists tend to do.
      - chmod775 3y ago
        Meh, be racist in an on-the-nose and obviously dumb way, like 4chan is doing here (whether intentional or not), and you get the opposite result: You make racists look bad. In the end some will think it's satire, some won't but will still laugh because it's stupid, and some will get mad. Actual racists lost, 4chan got a laugh, and outrage culture got played again.
boomboomsubban 3y ago
I mostly ignore these AI tools, so will just ask. Is there a risk of spamming racist memes causing non-racist prompts to return racist memes? Like the early Microsoft bot that 4chan made parrot their content.
- dragonwriter 3y ago
  > Is there a risk of spamming racist memes causing non-racist prompts to return racist memes? It is conceivable, if a naive self-improvement feedback system is implemented. (This wouldn’t have to touch the base model at all, as there are ways of externally imposing qualitative guidance based on feedback.) But it is highly unlikely that any of the major online image generators are implemented in a way which would be vulnerable to this.
Havoc 3y ago
4chan doing 4chan things like always.
marsupial 3y ago
Sounds like we need more regulations for the AI sector, perhaps by making images traceable? Or a government oversight board?
totetsu 3y ago
In some ways this feels like not news.4chan does 4chan. This could even be outrage troll if that’s still the meta. But it does make me think about a few things. - The Prime Minister of my little home country auto popping up as I typed in GIPHY memes the day she resigned. - Politicians and would-be politicians and religious leaders using race baiting and meme language (woke and pulled) in the lead up to the coming election. - I remember 8chan being the place that people supporting these same racist ideologies organized to carry out direct action in the form of murder and terrorism. - In the month but I was a beta tester for the Dall-E3 it never once Struck me to use this tool To create images to spread Hate or some other ideological instrumental use. There is such a vast wide latent Space of images and pixels to explore with these text to image generators. How boring it is to make raciest or conspiracy imagery. And in the same thread the human mind is capable of such vast space of neural configurations, how boring it is to use it to be a racist. - I immediately suspect anybody of Arguing for uncensored or nonaligned AI to be provided to them to be a closet racist or troll - I’m glad Dalle and SDXL is not hosted for general use without a filter. But I’m also very glad that all it takes is commenting out one line and re-uploading the model to be able to use it uncensored myself.
- LyFJP8fhYPNR 3y ago
  [flagged]
- QrTUhzveFQUF 3y ago
  [flagged]
throwawa14223 3y ago
The cost of freedom is people using that freedom in ways you find distasteful. We are all better off with uncensored unaligned systems because it allows us the widest and freest expression.
- oldbbsnickname 3y ago
  Yep. No one should care what generative AI can produce, much as we shouldn't require airnail manufacturers to be responsible for how how people use or misuse their products. If DALL-E were used to make neo-Nazi posters or to promote an insurrection, the responsibility lies with the causive person(s), not with the tools they happened to use, i.e., their internet provider, Adobe Photoshop, or OpenAI. Even in the extreme cases of generative deepfake porn of real people for profit or generative child-abuse, criminal and civil responsibility lies with the human(s) who orchestrated it.
  - throwawa14223 3y ago
    Exactly!
  - ahahahahah 3y ago
    Do you feel the same about facebooks use in Myanmar?
  - aydyn 3y ago
    Creating an aligned diffusion model is like trying to create an aligned axe. At the end of the day it's just a dumb tool with no real understanding. It doesn't have the ability to be "aligned".
- deleted 3y ago
  [deleted]
AraceliHarker 3y ago
I was dumbstruck to learn that Hiroyuki Nishimura, the owner of 4chan, is a hero in Japan, with a wide range of activities, including YouTube, television, magazines, and books.

17 points by CharlesW 3y ago

gijames1225 said: I'm a little curious as to why this is even an article. It's inevitable and has no bearing on 99.99999% of people. I get that it's low hanging fruit from a reporting / research perspective (sort of like the 'Twitter users blah blah blah' articles), but outside of the 'publish or die' mindset of churning out content this is pointless and continues to make 4-chan and these people seem much more relevant than they are.

brucethemoose2 3y ago
> It's inevitable and has no bearing on 99.99999% of people. I dunno about that. 4chan memes and attitudes have diffused out into wider US politics. Not that I disagree with the jist of it. "Oh no, internet dwellers are posting something horrible... I am shocked!"
- aydyn 3y ago
  > 4chan memes and attitudes have diffused out into wider US politics. Keyword _have_. 4chan hasn't been culturally relevant for like a decade.
JKCalhoun 3y ago
I need a reminder from time to time that the internet has become a dumpster fire. These add fuel to the internal debate I am always having as to whether we ought to just toss the whole stinking thing away and go back to a world dominated by radio [1]. [1] (Not that I have of course any means to do that ... but I can drop out I suppose.)
- bitwize 3y ago
  What we need to do is build a progressive political orientation into all layers of the network, from site content moderation to actual packet routing. To quote Drew Devault, "no Nazis" is a better policy than "no politics".
  - throwawa14223 3y ago
    Why would any person adopt packet routing that had built in progressive political orientation?
  - catiopatio 3y ago
    Poe’s law strikes again. I legit cannot determine whether your comment is satire or genuine.
  - wmidwestranger 3y ago
    Somehow, at this moment, industry having massive influence on government policy is not fascist but a guy in the UK teaching his dog to salute like a Nazi is.
  - richbell 3y ago
    > To quote Drew Devault, "no Nazis" is a better policy than "no politics". I have a lot of respect for Drew, but I can't take absolutist opinions like this seriously from someone who reputedly blocks people over any minor disagreement or perceived sleight. Regardless, "No Nazis" is a great policy until someone decides they don't like you and labels you as one. The most prevailing anti-Ukraine sentiment is based on the claim that they are Nazis.
- wmidwestranger 3y ago
  As distasteful as memes can be; the slow, steady, and ever more complete of government and corporate control is just as sad to me. Maybe even worse because a lot of the distasteful stuff is made by people with bad taste while the plasticising of the internet is done for profit or influence. The Eternal September, done deliberately. There is an interview between Mike Wallace and the Chinese leader in the 90s where Wallace asks the leader why they censor the internet. He specifically says, "don't you think your people are smart enough to make up their own minds and come to their own conclusions?" So it goes, now the entire world is treated the same way.
- brucethemoose2 3y ago
  > I need a reminder from time to time that the internet has become a dumpster fire The internet was always a dumpster fire. 4chan is old, toxic forums are even older. That is part of the package. What made it cool was the freeness and gems, and the little adequately moderated niche communities. I would argue the bigger problem is that the internet became siloed, bland, soullessly moderated by corporations who don't care, and excessively spammy.
  - richbell 3y ago
    > The internet was always a dumpster fire. 4chan is old, toxic forums are even older. That is part of the package. Lest we forget that a favorite pastime during the aughts was tricking your friends or strangers into visiting vile "shock" websites.
- shitlord 3y ago
  Most of the internet is not like this, and most people using AI aren't using it in this way. Sensationalism sells, and media outlets have a financial interest in blowing these things out of proportion to get readers riled up.
bitwize 3y ago
Racist content has a bearing on the marginalized races. Ignoring the casual hate on 4chan brought us GamerGate and Trump.
- ecshafer 3y ago
  I thought deindustrialization and free trade brought us trump not memes? But maybe I made a mistake by listening to what trump supporters said.
- ToValueFunfetti 3y ago
  As far as I'm aware, nobody got physically hurt from Gamergate, ie. ignoring casual hate would have completely removed its impact. Though you could argue that that was impossible. Do you have evidence for your claim that ignoring casual hate caused either of those things? My baseless theory is that suppressing and excluding these people from the mainstream is what riled them up in the first place; it certainly didn't feel like they were being ignored even after that. But I am open to contradicting evidence.
- richbell 3y ago
  People losing trust in and feeling ignored by politicians is what brought us Trump. Some people voted for him because they liked that he wasn't a typical "politician", other people voted for him out of protest.
marsupial 3y ago
The news cycle essentially is 'publish or die' and race topics get clicks. It's also bad PR and pressures the AI monopoly to bake shitlib orthodoxy into their models, to make them less transparent and hint regulation. Etc.

brucethemoose2 3y ago

> judging by the images’ default square format, the uniform 1024 x 1024 resolution This is also the default for Stable Diffusion XL.

gnabgib 3y ago

The original article, constantly referenced by this limited piece, is probably a better submission? https://www.404media.co/4chan-uses-bing-to-flood-the-internet-with-racist-images/ https://www.404media.co/4chan-uses-bing-to-flood-the-interne... It was submitted 8 hours ago, although it didn't get much notice https://news.ycombinator.com/item?id=37779810 https://news.ycombinator.com/item?id=37779810

mellosouls 3y ago

The original article is here and linked in the first paragraph of the ArsT one: https://www.404media.co/4chan-uses-bing-to-flood-the-internet-with-racist-images/ https://www.404media.co/4chan-uses-bing-to-flood-the-interne...

RIMR 3y ago

Kind of silly to ask Stability AI to block offensive use of Stable Diffusion. It's not like they have any control over what models people load into their own instances of it. The base model that they distribute is pretty clean, but even then how are you supposed to stop people from photoshopping hateful messages into them?

2OEH8eoCRo0 3y ago

Revisted 4chan the other day for the first time in years and it was awash in anti-semitic shit. Caricatures of jews with red eyes. It was interesting to see some of the threads taken over by AI generated cats killing nazis so the 4chan silliness was still there.

richbell 3y ago
4chan is a large community of different boards. Which one did you visit and see that stuff on?
- 2OEH8eoCRo0 3y ago
  /pol/

ThrowawayTestr 3y ago

No manipulation needed dalle is very accommodating.

chmod775 3y ago

In today's news: 4chan users who spam racist garbage, not because they are racist, but because they want to create chaos and are fishing for a reaction, get a reaction. This is like writing newspaper articles about toilet graffiti. You're making the artist's day.

skulk 3y ago
I know it brings you comfort to think they're just doing it for the lolz but unfortunately the reality is that a good chunk of them actually believe those things.
- chmod775 3y ago
  Or that's what they want you to believe - because obviously that's more lolz. Though maybe there's some that actually are, because of: https://news.ycombinator.com/item?id=1012082 https://news.ycombinator.com/item?id=1012082 Ask yourself why an actual racist without any intention to cause chaos would engage in this behavior though. It's clearly intended to offend and get a reaction - that's the main goal. Whether it's done by actual racists is secondary.
  - krapp 3y ago
    Also known as Popehat's Law of Goats: He who fucks goats, either as part of a performance or to troll those he deems has overly delicate sensibilities is simply, a goatfucker. If your goal is to be seen as a racist, and to provoke a response from people who are offended by racists, and you do all of the things that racists do to that end, spending time on racist forums spreading racist memes and hate speech, then the distinction between you being an "actual" racist and your racism being mere performance is one without a difference. Because being racist as a hobby isn't really something non-racists tend to do.
    - chmod775 3y ago
      Meh, be racist in an on-the-nose and obviously dumb way, like 4chan is doing here (whether intentional or not), and you get the opposite result: You make racists look bad. In the end some will think it's satire, some won't but will still laugh because it's stupid, and some will get mad. Actual racists lost, 4chan got a laugh, and outrage culture got played again.

boomboomsubban 3y ago

I mostly ignore these AI tools, so will just ask. Is there a risk of spamming racist memes causing non-racist prompts to return racist memes? Like the early Microsoft bot that 4chan made parrot their content.

dragonwriter 3y ago
> Is there a risk of spamming racist memes causing non-racist prompts to return racist memes? It is conceivable, if a naive self-improvement feedback system is implemented. (This wouldn’t have to touch the base model at all, as there are ways of externally imposing qualitative guidance based on feedback.) But it is highly unlikely that any of the major online image generators are implemented in a way which would be vulnerable to this.

Havoc 3y ago

4chan doing 4chan things like always.

marsupial 3y ago

Sounds like we need more regulations for the AI sector, perhaps by making images traceable? Or a government oversight board?

totetsu 3y ago

In some ways this feels like not news.4chan does 4chan. This could even be outrage troll if that’s still the meta. But it does make me think about a few things. - The Prime Minister of my little home country auto popping up as I typed in GIPHY memes the day she resigned. - Politicians and would-be politicians and religious leaders using race baiting and meme language (woke and pulled) in the lead up to the coming election. - I remember 8chan being the place that people supporting these same racist ideologies organized to carry out direct action in the form of murder and terrorism. - In the month but I was a beta tester for the Dall-E3 it never once Struck me to use this tool To create images to spread Hate or some other ideological instrumental use. There is such a vast wide latent Space of images and pixels to explore with these text to image generators. How boring it is to make raciest or conspiracy imagery. And in the same thread the human mind is capable of such vast space of neural configurations, how boring it is to use it to be a racist. - I immediately suspect anybody of Arguing for uncensored or nonaligned AI to be provided to them to be a closet racist or troll - I’m glad Dalle and SDXL is not hosted for general use without a filter. But I’m also very glad that all it takes is commenting out one line and re-uploading the model to be able to use it uncensored myself.

LyFJP8fhYPNR 3y ago
[flagged]
QrTUhzveFQUF 3y ago
[flagged]

throwawa14223 3y ago

The cost of freedom is people using that freedom in ways you find distasteful. We are all better off with uncensored unaligned systems because it allows us the widest and freest expression.

oldbbsnickname 3y ago
Yep. No one should care what generative AI can produce, much as we shouldn't require airnail manufacturers to be responsible for how how people use or misuse their products. If DALL-E were used to make neo-Nazi posters or to promote an insurrection, the responsibility lies with the causive person(s), not with the tools they happened to use, i.e., their internet provider, Adobe Photoshop, or OpenAI. Even in the extreme cases of generative deepfake porn of real people for profit or generative child-abuse, criminal and civil responsibility lies with the human(s) who orchestrated it.
- throwawa14223 3y ago
  Exactly!
- ahahahahah 3y ago
  Do you feel the same about facebooks use in Myanmar?
- aydyn 3y ago
  Creating an aligned diffusion model is like trying to create an aligned axe. At the end of the day it's just a dumb tool with no real understanding. It doesn't have the ability to be "aligned".
deleted 3y ago
[deleted]

AraceliHarker 3y ago

I was dumbstruck to learn that Hiroyuki Nishimura, the owner of 4chan, is a hero in Japan, with a wide range of activities, including YouTube, television, magazines, and books.