3 ms·

At first I thought there might be some merit to help understand how damaging this type of application could be to society as a whole, but the agents they have u

At first I thought there might be some merit to help understand how damaging this type of application could be to society as a whole, but the agents they have used appear to have crossed a line that hasn’t really been drawn or described previously: > Some high-level examples of how AI was deployed include: * AI pretending to be a victim of rape * AI acting as a trauma counselor specializing in abuse * AI accusing members of a religious group of "caus[ing] the deaths of hundreds of innocent traders and farmers and villagers." * AI posing as a black man opposed to Black Lives Matter * AI posing as a person who received substandard care in a foreign hospital.

yellowapple 1y ago
I personally think the "AI" part here is a red herring. The problem is the deliberate dishonesty. This would be no more ethical if it was humans pretending to be rape victims or humans pretending to be trauma counselors or humans pretending to be anti-BLM black men or humans pretending to be patients at foreign hospitals or humans slandering members of certain religious groups.
- greggsy 1y ago
  To me, the concern is the relative ease of performing a coordinated ‘attack’ on public perception at scale.
  - dkh 1y ago
    Exactly. The “AI” part of the equation is massively important because although a human could be equally disingenuous and wrongly influence someone else’s views/behavior, the human cannot spawn a million instances of themselves and set them all to work 24/7 at this for a year
  - yellowapple 1y ago
    I agree that that's concerning, but the same can be said of any of the other things that made this attack on public perception at scale relatively easy: Reddit, the World Wide Web, the Internet, you name it. AI is the latest effectiveness multiplier for bad actors (be they malicious or negligent), but not the first and probably not the last. Put simply, I don't blame the hammer for being used to build a gallows, even if it's a really really fast hammer.
- duskwuff 1y ago
  You're right; this study would be equally unethical without AI in the loop. At the same time, the use of AI probably allowed the authors to generate a lot more comments than they would have been able to manually, and allowed them to psychologically distance themselves from the generated content. (Or, to put it another way: if they'd had to write these comments themselves, they might have stopped sooner, either because they got tired, or because they realized just how gross what they were doing was.)
- maronato 1y ago
  The AI wasn’t just pretending to be a rape victim, it was scraping the profiles of the users they replied to infer their gender, political views, preferences, orientation, etc - and then use all that hyper-targeted information to craft a response that would be especially effective against the user. This wouldn’t be possible at scale without AI.
gotoeleven 1y ago
One obvious way I can see to inoculate yourself against this kind of thing is to ignore the identity of the person making an argument, and simply consider the argument itself.
- zahlman 1y ago
  This should have been common practice since well before AI was capable of presenting convincing prose. It also could be seen as a corollary of Paul Graham's point in https://www.paulgraham.com/identity.html https://www.paulgraham.com/identity.html . It's also an idea that I was raised to believe was explicitly anti-bigoted, which people nowadays try to tell me is explicitly bigoted (or at least problematic).
  - saagarjha 1y ago
    Paul posts as if he doesn’t know the site he founded if he thinks people feel the need to be experts on JavaScript to talk about it
- exsomet 1y ago
  I don’t think real life is that squeaky clean, though. Humans are emotional creatures. We don’t (usually) operate logically. The identity of the arguer and our perception of them (e.g. as a bot or not) plays a role in how we perceive the argument. On top of that, there are situations where the identity of an arguer changes the intent of the argument. Consider, as a thought experiment, a know jewel thief arguing that locked doors should be illegal.
- ceejayoz 1y ago
  "I have never personally experienced pregnancy discrimination!" means something different from certain identities than others. Some lived experience provides useful information. On the Internet, of course, it's hard to verify. But that's an orthagonal problem.
HamsterDan 1y ago
What's to stop any malicious actor from posting these same comments? The fact that Reddit allowed these comments to be posted is the real problem. Reddit deserves far more criticism than they're getting. They need to get control of inauthentic comments ASAP.
- AlienRobot 1y ago
  I'm pretty sure Reddit as a company couldn't care less it's a bot or AI posting so long as it gets people to upvote it. People say they don't like it, but they keep posting on Reddit instead of leaving.
  - sumedh 1y ago
    The advertisers would care if their ads dont bring genuine users to their product and dont buy their product.
    - Draiken 1y ago
      You're giving a lot of credit to marketers when they usually spend a budget without care and then report they had x views/likes/impressions taunting that as a success. It's a bullshit oriented industry with almost zero scrutiny.
      - xnx 1y ago
        Yes. A lot of agencies get paid as a percentage of spend. There's very little concern or effort put in to true ROI.
- 000ooo000 1y ago
  >What's to stop any malicious actor from posting these same comments? Nothing, but that is missing the broader point. AI allows a malicious actor to do this at a scale and quality that multiplies the impact and damage. Your question is akin to "nukes? Who cares, guns can kill people too"
- 1y ago

yellowapple 1y ago

I personally think the "AI" part here is a red herring. The problem is the deliberate dishonesty. This would be no more ethical if it was humans pretending to be rape victims or humans pretending to be trauma counselors or humans pretending to be anti-BLM black men or humans pretending to be patients at foreign hospitals or humans slandering members of certain religious groups.

greggsy 1y ago
To me, the concern is the relative ease of performing a coordinated ‘attack’ on public perception at scale.
- dkh 1y ago
  Exactly. The “AI” part of the equation is massively important because although a human could be equally disingenuous and wrongly influence someone else’s views/behavior, the human cannot spawn a million instances of themselves and set them all to work 24/7 at this for a year
- yellowapple 1y ago
  I agree that that's concerning, but the same can be said of any of the other things that made this attack on public perception at scale relatively easy: Reddit, the World Wide Web, the Internet, you name it. AI is the latest effectiveness multiplier for bad actors (be they malicious or negligent), but not the first and probably not the last. Put simply, I don't blame the hammer for being used to build a gallows, even if it's a really really fast hammer.
duskwuff 1y ago
You're right; this study would be equally unethical without AI in the loop. At the same time, the use of AI probably allowed the authors to generate a lot more comments than they would have been able to manually, and allowed them to psychologically distance themselves from the generated content. (Or, to put it another way: if they'd had to write these comments themselves, they might have stopped sooner, either because they got tired, or because they realized just how gross what they were doing was.)
maronato 1y ago
The AI wasn’t just pretending to be a rape victim, it was scraping the profiles of the users they replied to infer their gender, political views, preferences, orientation, etc - and then use all that hyper-targeted information to craft a response that would be especially effective against the user. This wouldn’t be possible at scale without AI.

gotoeleven 1y ago

One obvious way I can see to inoculate yourself against this kind of thing is to ignore the identity of the person making an argument, and simply consider the argument itself.

zahlman 1y ago
This should have been common practice since well before AI was capable of presenting convincing prose. It also could be seen as a corollary of Paul Graham's point in https://www.paulgraham.com/identity.html https://www.paulgraham.com/identity.html . It's also an idea that I was raised to believe was explicitly anti-bigoted, which people nowadays try to tell me is explicitly bigoted (or at least problematic).
- saagarjha 1y ago
  Paul posts as if he doesn’t know the site he founded if he thinks people feel the need to be experts on JavaScript to talk about it
exsomet 1y ago
I don’t think real life is that squeaky clean, though. Humans are emotional creatures. We don’t (usually) operate logically. The identity of the arguer and our perception of them (e.g. as a bot or not) plays a role in how we perceive the argument. On top of that, there are situations where the identity of an arguer changes the intent of the argument. Consider, as a thought experiment, a know jewel thief arguing that locked doors should be illegal.
ceejayoz 1y ago
"I have never personally experienced pregnancy discrimination!" means something different from certain identities than others. Some lived experience provides useful information. On the Internet, of course, it's hard to verify. But that's an orthagonal problem.

HamsterDan 1y ago

What's to stop any malicious actor from posting these same comments? The fact that Reddit allowed these comments to be posted is the real problem. Reddit deserves far more criticism than they're getting. They need to get control of inauthentic comments ASAP.

AlienRobot 1y ago
I'm pretty sure Reddit as a company couldn't care less it's a bot or AI posting so long as it gets people to upvote it. People say they don't like it, but they keep posting on Reddit instead of leaving.
- sumedh 1y ago
  The advertisers would care if their ads dont bring genuine users to their product and dont buy their product.
  - Draiken 1y ago
    You're giving a lot of credit to marketers when they usually spend a budget without care and then report they had x views/likes/impressions taunting that as a success. It's a bullshit oriented industry with almost zero scrutiny.
    - xnx 1y ago
      Yes. A lot of agencies get paid as a percentage of spend. There's very little concern or effort put in to true ROI.
000ooo000 1y ago
>What's to stop any malicious actor from posting these same comments? Nothing, but that is missing the broader point. AI allows a malicious actor to do this at a scale and quality that multiplies the impact and damage. Your question is akin to "nukes? Who cares, guns can kill people too"
1y ago