5 ms·

It’s gross, but I am 10000% sure Reddit and the rest of social media is already overflowing with these types of bots. I feel like this project actually does peo

by api 1y ago

It’s gross, but I am 10000% sure Reddit and the rest of social media is already overflowing with these types of bots. I feel like this project actually does people a service by showing what this looks like and how effective it can be.

deleted 1y ago
[deleted]
stefan_ 1y ago
So you agree the research and data collected was useless?
- tonyarkles 1y ago
  (Not the person you replied to) While I don't generally agree with the ethics of how the research was done, I do, personally, think the research and the data could be enlightening. Reddit, X, Facebook, and other platforms might be overflowing with bots that are already doing this but we (the general public) don't generally have clear data on how much this is happening, how effective it is, things to watch out for, etc. It's definitely an arms race but I do think that a paper which clearly communicates "in our study these specific things were the most effective way to change peoples' opinions with bots" serves as valuable input for knowing what to look out for. I'm torn on it, to be honest.
  - binarymax 1y ago
    But what does the study show? There was no control for anything. None of the data is valid. To clarify: how does the research team know the bots were interacting with people and not other bots?
    - UncleMeat 1y ago
      I agree that the study was performed unethically and should not be published but observational studies are totally fine things to do. A lot of research is "hey we looked at stuff and found this data that wiggles its eyebrows at some idea so we should fund more rigorous study design in the future." An individual paper does not need to fully resolve a question. The reason not to publish this work is because the data was collected unethically and we don't want to reward or incentivize such work. Nothing to do with the quality of the data itself.
- SudoSuccubus 1y ago
  [flagged]
SudoSuccubus 1y ago
If the mere possibility of AI-generated context invalidates an argument, it suggests the standards for discourse were already more fragile than anyone cared to admit. Historically, emotional narratives and unverifiable personal stories have always been persuasive tools — whether human-authored or not. The actual problem isn't that AI can produce them; it's that we (humans) have always been susceptible to them without verifying the core ideas. In that sense, exposing how easily constructed narratives sway public discussion is not unethical — it's a necessary and overdue audit of the real vulnerabilities in our conversations. Blaming the tool only avoids the harder truth: we were never debating cleanly to begin with.
- AlienRobot 1y ago
  Flooding human forums with AI steals real state from actual humans. Reddit is already flooded with bots. That was already a problem. The actual problem is people thinking that because a system used by many isn't perfect that gives them permission to destroy the existing system. Don't like Reddit? Just don't go to Reddit. Go to fanclubs.org or something.
  - garbagewoman 1y ago
    Ok, need you to clarify a few implicit and explicit statements there: The study destroyed the subreddit? The authors of the study believed they had permission to destroy the subreddit? the subreddit is now destroyed? The researchers don’t like reddit? The researchers would achieve their aims by going to fanclubs.org or something?
- godelski 1y ago
  > Blaming the tool only avoids the harder truth: we were never debating cleanly to begin with. Yes, the problem is we humans are susceptible, but that doesn't mean a tool used to scale up the ability to create this harm is not problematic. There's a huge difference between a single person manipulating one other person and a single person manipulating millions. Scale matters and we, especially as the builders of such tools, should be cautious about how our creations can be abused. It's easy to look away, but this is why ethics is so important in engineering.
mountainriver 1y ago
Agree, this is already happening in mass, if anything this is great to raise awareness and show what can happen. The mods seem overly pedantic, but I guess that is usually the case on Reddit. If they think for a second that a bunch of their content isn’t AI generated, they are deeply mistaken
ozbonus 1y ago
In the back of mind I knew it wasn't so, but I had been holding onto the belief that surely I could discern between human and bot, and that bots weren't a real issue where I spent my time anyway. But no. We're at a point where any anonymous public comment is possibly an impersonation. And eventually that "possibly" will have to replaced with "most likely". I don't know what the solution is or if there even is one.
- int_19h 1y ago
  There isn't. Not only LLMs are good enough to fool humans like this, but they have been that for quite a while now with the right prompting. A large number of readily available open weights models can do this, so even if large providers were to crack down on this kind of use, it's still easy to run the model locally to generate such content. The cat is well and truly out of the bag.
godelski 1y ago
I'm pretty sure we saw LLMs in yesterday's thread about the judge. There were a lot of strange comments (stately worded and weird logic that were very LLM like, not just dumb person like) and it wouldn't be surprising as it's an easy tool to weaponize chaos. I'm sure there were bots supporting many different positions. It even looks like some accounts were posting contradictions opinions
- api 1y ago
  If I wanted to just attack and destabilize society I'd have armies of bots supporting all the most divisive positions on all sides, as well as promoting incoherent and irrational positions. The idea is just to divide, confuse, "flood the zone with shit" as Bannon likes to say. It seems like people are actually not bad at noticing likely bots arguing against their favorite positions, but are blind to the possibility that there could be bots pretending to be on their side. The most corrosive might be bots pretending to be on your side but advocating subtilely wrong or unnecessarily divisive formulations of your ideas, which in turn are more likely to influence you because they seem to be on your side. Phrases come to mind like "vandalism of the discourse" and "intellectual terrorism" where the goal is not to promote one specific idea but to destroy the discourse as a whole. That certainly looks like the world we're living in.
  - godelski 1y ago
    I remember seeing some reports around the BLM protests that claimed Russia organized both a protest and a counter protest via Facebook groups. Not sure how accurate (I believe it) but it certainly is an effective strategy. The old "Divide and conquer" strategy that's thousands of years old.
    - deleted 1y ago
      [deleted]
- 3np 1y ago
  The buried greens, yeah. I was guessing it was some meatpuppets I was arguing with after looking at account age and post history though. Still considered it worth the effort and trollfeeding to set the record straight for others and yeah, getting harder to tell the difference between puppets, zombies, and bots.

by api 1y ago

deleted 1y ago

[deleted]

stefan_ 1y ago

So you agree the research and data collected was useless?

tonyarkles 1y ago
(Not the person you replied to) While I don't generally agree with the ethics of how the research was done, I do, personally, think the research and the data could be enlightening. Reddit, X, Facebook, and other platforms might be overflowing with bots that are already doing this but we (the general public) don't generally have clear data on how much this is happening, how effective it is, things to watch out for, etc. It's definitely an arms race but I do think that a paper which clearly communicates "in our study these specific things were the most effective way to change peoples' opinions with bots" serves as valuable input for knowing what to look out for. I'm torn on it, to be honest.
- binarymax 1y ago
  But what does the study show? There was no control for anything. None of the data is valid. To clarify: how does the research team know the bots were interacting with people and not other bots?
  - UncleMeat 1y ago
    I agree that the study was performed unethically and should not be published but observational studies are totally fine things to do. A lot of research is "hey we looked at stuff and found this data that wiggles its eyebrows at some idea so we should fund more rigorous study design in the future." An individual paper does not need to fully resolve a question. The reason not to publish this work is because the data was collected unethically and we don't want to reward or incentivize such work. Nothing to do with the quality of the data itself.
SudoSuccubus 1y ago
[flagged]

SudoSuccubus 1y ago

If the mere possibility of AI-generated context invalidates an argument, it suggests the standards for discourse were already more fragile than anyone cared to admit. Historically, emotional narratives and unverifiable personal stories have always been persuasive tools — whether human-authored or not. The actual problem isn't that AI can produce them; it's that we (humans) have always been susceptible to them without verifying the core ideas. In that sense, exposing how easily constructed narratives sway public discussion is not unethical — it's a necessary and overdue audit of the real vulnerabilities in our conversations. Blaming the tool only avoids the harder truth: we were never debating cleanly to begin with.

AlienRobot 1y ago
Flooding human forums with AI steals real state from actual humans. Reddit is already flooded with bots. That was already a problem. The actual problem is people thinking that because a system used by many isn't perfect that gives them permission to destroy the existing system. Don't like Reddit? Just don't go to Reddit. Go to fanclubs.org or something.
- garbagewoman 1y ago
  Ok, need you to clarify a few implicit and explicit statements there: The study destroyed the subreddit? The authors of the study believed they had permission to destroy the subreddit? the subreddit is now destroyed? The researchers don’t like reddit? The researchers would achieve their aims by going to fanclubs.org or something?
godelski 1y ago
> Blaming the tool only avoids the harder truth: we were never debating cleanly to begin with. Yes, the problem is we humans are susceptible, but that doesn't mean a tool used to scale up the ability to create this harm is not problematic. There's a huge difference between a single person manipulating one other person and a single person manipulating millions. Scale matters and we, especially as the builders of such tools, should be cautious about how our creations can be abused. It's easy to look away, but this is why ethics is so important in engineering.

mountainriver 1y ago

Agree, this is already happening in mass, if anything this is great to raise awareness and show what can happen. The mods seem overly pedantic, but I guess that is usually the case on Reddit. If they think for a second that a bunch of their content isn’t AI generated, they are deeply mistaken

ozbonus 1y ago

In the back of mind I knew it wasn't so, but I had been holding onto the belief that surely I could discern between human and bot, and that bots weren't a real issue where I spent my time anyway. But no. We're at a point where any anonymous public comment is possibly an impersonation. And eventually that "possibly" will have to replaced with "most likely". I don't know what the solution is or if there even is one.

int_19h 1y ago
There isn't. Not only LLMs are good enough to fool humans like this, but they have been that for quite a while now with the right prompting. A large number of readily available open weights models can do this, so even if large providers were to crack down on this kind of use, it's still easy to run the model locally to generate such content. The cat is well and truly out of the bag.

godelski 1y ago

I'm pretty sure we saw LLMs in yesterday's thread about the judge. There were a lot of strange comments (stately worded and weird logic that were very LLM like, not just dumb person like) and it wouldn't be surprising as it's an easy tool to weaponize chaos. I'm sure there were bots supporting many different positions. It even looks like some accounts were posting contradictions opinions

api 1y ago
If I wanted to just attack and destabilize society I'd have armies of bots supporting all the most divisive positions on all sides, as well as promoting incoherent and irrational positions. The idea is just to divide, confuse, "flood the zone with shit" as Bannon likes to say. It seems like people are actually not bad at noticing likely bots arguing against their favorite positions, but are blind to the possibility that there could be bots pretending to be on their side. The most corrosive might be bots pretending to be on your side but advocating subtilely wrong or unnecessarily divisive formulations of your ideas, which in turn are more likely to influence you because they seem to be on your side. Phrases come to mind like "vandalism of the discourse" and "intellectual terrorism" where the goal is not to promote one specific idea but to destroy the discourse as a whole. That certainly looks like the world we're living in.
- godelski 1y ago
  I remember seeing some reports around the BLM protests that claimed Russia organized both a protest and a counter protest via Facebook groups. Not sure how accurate (I believe it) but it certainly is an effective strategy. The old "Divide and conquer" strategy that's thousands of years old.
  - deleted 1y ago
    [deleted]
3np 1y ago
The buried greens, yeah. I was guessing it was some meatpuppets I was arguing with after looking at account age and post history though. Still considered it worth the effort and trollfeeding to set the record straight for others and yeah, getting harder to tell the difference between puppets, zombies, and bots.