4 ms·

CNN just did a piece on private Israeli groups doing the exact same thing. It is pretty scary the shear scale of this. Literally any entity or group can spin up

CNN just did a piece on private Israeli groups doing the exact same thing. It is pretty scary the shear scale of this. Literally any entity or group can spin up a ton of bots and use any AI service locally or otherwise, to attempt to sway public opinion.

ants_everywhere 2y ago
Reddit especially is full of LLMs. I would imagine there are ones from every major country plus several privately funded ones.
- theGnuMe 2y ago
  they are here as well.
  - mandmandam 2y ago
    Can they be drawn out, I wonder, when the context is too thin for them to catch... GMOs are dangerous for reasons beyond immediate health implications. Israel has been determined in the world's highest court to be plausibly committing genocide. The tech industry is complicit in war crimes right now. Media consolidation is a massive and immediate threat, as are climate change and AIPAC. ... Or is it like what the park ranger said about the overlap between people and bears?
- suby 2y ago
  Yeah, all of the major subreddits are gamed, especially subs like r/politics. Even going back to the 2016 election it was clear that much of the activity on there was not genuine.
  - shusaku 2y ago
    It’s been well known for over a decade that Reddit is incredibly easy to game. So while LLMs may seem useful, a well run political campaign can easily afford to astroturf the site while paying good money to their staff.
    - ants_everywhere 2y ago
      > It’s been well known for over a decade that Reddit is incredibly easy to game Yeah. Arguably Reddit has embraced inorganic activity since beginning. the founders used it to grow the site and pretend they had more active users.
  - courseofaction 2y ago
    It has been interesting to see the coordination of harris/waltz posts reaching top places in non-political reddits. There was also a marked increase in moderation at the beginning of the Ukraine war in related subreddits
- firesteelrain 2y ago
  Spend enough time in r/politics and you’ll notice. What’s troubling is the constant spamming in new subreddits like r/inthenews. Once it takes hold (whatever the message), then actual humans start to echo it over and over again. It’s scary.
  - Garvi 2y ago
    The problem with modern social media, compared to traditional ones, is it offers you to consume other peoples feedback on the story. It's a bit like traditional media publishing an opinion poll with every story run. The problem is how these "polls" influence peoples opinions and the role censorship plays in this. Imagine a politically divisive event occurs and while the original feedback was 50%-50% on opposing opinions, the moderator made it look like it was 100%-0%. This makes people feel isolated and forces them psychologically to change their opinion to the virtually predominant one. You can image LLMs only enhancing these mechanisms. As an extreme example of this, I remember during the corona pandemic, on reddit r/worldnews there was a weekly thread that was cheering on for deaths of Russian citizens to corona. The whole thread was heavily curated only to display pure hate towards Russians. And the moderators always deleted the thread after a week and started a new one. People who posted in the thread and didn't get on with the program, had their accounts simultaneously permabanned on all major subreddits. I know, because it happened to me. If I wrote this on Reddit, I would be called a dirty Russian, a spy, a bot, a generally evil person, a traitor and I have also been called various anti-Chinese racial slurs, some quite graphic and disgusting. There's many people out there seeking validation and acceptance, and they find it in such hate groups. I believe historians far in the future will be putting Reddit in the same category as the Hitler youth organization. And HN is very much like Reddit, only with slightly smarter people. The moderators in here also hold very extreme views on various topics and manipulate the discourse to give false appearances of public opinion on those topics.
TaylorAlexander 2y ago
It’s frustrating too seeing some people in AI say the technology has no negative implications. Of course it does. Foreign actors are using it to influence elections. But when Jeremy Howard was asked if he had done impact assessment of a large open source model release, he replied (roughly) “No. Impact is about the same as a new release of pencil and paper.” Which… actually that’s not true. Iran isn’t using pencil and paper to influence US elections, they’re using LLMs. And if all the big cloud providers cut them off, they’ll move to using home brew models. That is the impact and putting our head in the sand about it isn’t going to be helpful. This is NOT to say we should not release to models! I think we should release open models, and that in addition to that we should also care about written impact assessments so everyone is on the same page about risks! I think it’s great that Jeremy Howard is releasing open foundation models, but I wish he wasn’t so flippant about the real world implications. I’ve always looked up to him for the work he has done, and found that attitude very disappointing.
- CatWChainsaw 2y ago
  Also frustrating to be constantly patronized that the negative consequences will definitely be outweighed by positive ones, pinky promise. Right now we have threats of job insecurity in basically every industry, Noah's Arc levels of spam, the undermining of democracy as a functioning form of government, and insane energy demands and environmental consequences. But on the plus side you can outsource the entire design team to one guy tweaking SD outputs and you can use an LLM to summarize the spam generated by a different LLM, so all's well that ends well I guess.
  - TaylorAlexander 2y ago
    Agreed. I think the short term profit motive is really blinding people to the risks. Like we could have continued this in a research setting for longer to really get a sense for how to deal with these things, but folks wanted to productize because of profit potential backed by incredible hype.
zitterbewegung 2y ago
There is a pro Russian bot that someone used prompt injection to prove it was a bot. https://www.reddit.com/r/interestingasfuck/comments/1dlg8ni/russian_bot_falls_prey_to_a_prompt_iniection/#lightbox https://www.reddit.com/r/interestingasfuck/comments/1dlg8ni/...
- userbinator 2y ago
  How do you distinguish a real bot, from a human pretending to be a bot?
  - zitterbewegung 2y ago
    I can't prove a negative issue in the situation that you provided.
ComplexSystems 2y ago
Fortunately they seem to be using centralized services such as OpenAI's API, where they can get caught, rather than spinning up their own Llama instances.

ants_everywhere 2y ago

Reddit especially is full of LLMs. I would imagine there are ones from every major country plus several privately funded ones.

theGnuMe 2y ago
they are here as well.
- mandmandam 2y ago
  Can they be drawn out, I wonder, when the context is too thin for them to catch... GMOs are dangerous for reasons beyond immediate health implications. Israel has been determined in the world's highest court to be plausibly committing genocide. The tech industry is complicit in war crimes right now. Media consolidation is a massive and immediate threat, as are climate change and AIPAC. ... Or is it like what the park ranger said about the overlap between people and bears?
suby 2y ago
Yeah, all of the major subreddits are gamed, especially subs like r/politics. Even going back to the 2016 election it was clear that much of the activity on there was not genuine.
- shusaku 2y ago
  It’s been well known for over a decade that Reddit is incredibly easy to game. So while LLMs may seem useful, a well run political campaign can easily afford to astroturf the site while paying good money to their staff.
  - ants_everywhere 2y ago
    > It’s been well known for over a decade that Reddit is incredibly easy to game Yeah. Arguably Reddit has embraced inorganic activity since beginning. the founders used it to grow the site and pretend they had more active users.
- courseofaction 2y ago
  It has been interesting to see the coordination of harris/waltz posts reaching top places in non-political reddits. There was also a marked increase in moderation at the beginning of the Ukraine war in related subreddits
firesteelrain 2y ago
Spend enough time in r/politics and you’ll notice. What’s troubling is the constant spamming in new subreddits like r/inthenews. Once it takes hold (whatever the message), then actual humans start to echo it over and over again. It’s scary.
- Garvi 2y ago
  The problem with modern social media, compared to traditional ones, is it offers you to consume other peoples feedback on the story. It's a bit like traditional media publishing an opinion poll with every story run. The problem is how these "polls" influence peoples opinions and the role censorship plays in this. Imagine a politically divisive event occurs and while the original feedback was 50%-50% on opposing opinions, the moderator made it look like it was 100%-0%. This makes people feel isolated and forces them psychologically to change their opinion to the virtually predominant one. You can image LLMs only enhancing these mechanisms. As an extreme example of this, I remember during the corona pandemic, on reddit r/worldnews there was a weekly thread that was cheering on for deaths of Russian citizens to corona. The whole thread was heavily curated only to display pure hate towards Russians. And the moderators always deleted the thread after a week and started a new one. People who posted in the thread and didn't get on with the program, had their accounts simultaneously permabanned on all major subreddits. I know, because it happened to me. If I wrote this on Reddit, I would be called a dirty Russian, a spy, a bot, a generally evil person, a traitor and I have also been called various anti-Chinese racial slurs, some quite graphic and disgusting. There's many people out there seeking validation and acceptance, and they find it in such hate groups. I believe historians far in the future will be putting Reddit in the same category as the Hitler youth organization. And HN is very much like Reddit, only with slightly smarter people. The moderators in here also hold very extreme views on various topics and manipulate the discourse to give false appearances of public opinion on those topics.

TaylorAlexander 2y ago

It’s frustrating too seeing some people in AI say the technology has no negative implications. Of course it does. Foreign actors are using it to influence elections. But when Jeremy Howard was asked if he had done impact assessment of a large open source model release, he replied (roughly) “No. Impact is about the same as a new release of pencil and paper.” Which… actually that’s not true. Iran isn’t using pencil and paper to influence US elections, they’re using LLMs. And if all the big cloud providers cut them off, they’ll move to using home brew models. That is the impact and putting our head in the sand about it isn’t going to be helpful. This is NOT to say we should not release to models! I think we should release open models, and that in addition to that we should also care about written impact assessments so everyone is on the same page about risks! I think it’s great that Jeremy Howard is releasing open foundation models, but I wish he wasn’t so flippant about the real world implications. I’ve always looked up to him for the work he has done, and found that attitude very disappointing.

CatWChainsaw 2y ago
Also frustrating to be constantly patronized that the negative consequences will definitely be outweighed by positive ones, pinky promise. Right now we have threats of job insecurity in basically every industry, Noah's Arc levels of spam, the undermining of democracy as a functioning form of government, and insane energy demands and environmental consequences. But on the plus side you can outsource the entire design team to one guy tweaking SD outputs and you can use an LLM to summarize the spam generated by a different LLM, so all's well that ends well I guess.
- TaylorAlexander 2y ago
  Agreed. I think the short term profit motive is really blinding people to the risks. Like we could have continued this in a research setting for longer to really get a sense for how to deal with these things, but folks wanted to productize because of profit potential backed by incredible hype.

zitterbewegung 2y ago

There is a pro Russian bot that someone used prompt injection to prove it was a bot. https://www.reddit.com/r/interestingasfuck/comments/1dlg8ni/russian_bot_falls_prey_to_a_prompt_iniection/#lightbox https://www.reddit.com/r/interestingasfuck/comments/1dlg8ni/...

userbinator 2y ago
How do you distinguish a real bot, from a human pretending to be a bot?
- zitterbewegung 2y ago
  I can't prove a negative issue in the situation that you provided.

ComplexSystems 2y ago

Fortunately they seem to be using centralized services such as OpenAI's API, where they can get caught, rather than spinning up their own Llama instances.