5 ms·

You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that f

by sva_ 4y ago

You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that.

ramesh31 4y ago
>You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that. You've just described half of Twitter's MAUs.
- pempem 4y ago
  It also feels very like ender's game near the end of the book
tablespoon 4y ago
> You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that. Wouldn't that be fairly easy to detect because the accounts would belong to an isolated, tightly-connected cluster?
- quantified 4y ago
  State actors can design societies with lots of different clusters. It's all in the simulator you design. And a large actor could easily require that its humans engage with members of this society to connect them with humanity.
- deleted 4y ago
  [deleted]
- sva_ 4y ago
  I'm not sure how you would detect those clusters? The language models wouldn't give it away for sure, and getting residential IPs has never been easier than in the present.
  - tablespoon 4y ago
    > I'm not sure how you would detect those clusters? The language models wouldn't give it away for sure, and getting residential IPs has never been easier than in the present. Social graph metadata. The GGP said they would all be "talking to each other."
    - sva_ 4y ago
      Yeah I mean they'd talk to each other, but they could also interact with other people using empty platitudes or so that are common on social media. So I think it wouldn't be that easy.
    - Dan_Sylveste 4y ago
      >Social graph metadata. The GGP said they would all be "talking to each other." I think by 'detect' they mean 'detect by people who might fall for it' not 'detect by national security agencies.' I don't see any way for the average social media user to detect whether they're engaging with a botfarm unless the social media company exposes their analytics, and they don't, and they aren't likely to start unless forced.
      - tablespoon 4y ago
        > I think by 'detect' they mean 'detect by people who might fall for it' not 'detect by national security agencies.' I actually was thinking the people who'd do the detecting would be the social media site operator (e.g. Twitter), who would then shut down the inauthentic activity.
        Dan_Sylveste 4y ago
        Yeah, that isn't going to happen.
corrral 4y ago
I recently read a near-future sci-fi story (linked somewhere on here, I think) about an AI breaking out of its contained environment and taking over the world before we could figure out what happened and stop it (took a couple weeks, IIRC). The TL;DR is that once it had enough compromised machines to run social media botnets, it was all over. It could use those to confound efforts to coordinate and compare data, to misdirect huge numbers of people and cause all kinds of chaos, and to smear opponents before they could get their message out (fakes or actual stolen information—it hardly mattered, all it needed to do was neutralize certain people for a few days). The story contrived to have a secret project that was able to try to resist it after that (spoiler: didn't help) but otherwise the social media botnets were enough for it to buy several days in which no-one was able to effectively work against it.
- Vecr 4y ago
  Gwern's "It Looks Like You’re Trying To Take Over The World (Clippy)" https://www.gwern.net/fiction/Clippy https://www.gwern.net/fiction/Clippy
- Dan_Sylveste 4y ago
  That's not a story, that's what's actually happened, only the it wasn't an AI, it was aliens that were discovered in 2018. That's why we're seeing more and more of an open attitude towards UFOs from major governments, in preparation for the big reveal. You will eat the bugs.

by sva_ 4y ago

ramesh31 4y ago

>You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that. You've just described half of Twitter's MAUs.

pempem 4y ago
It also feels very like ender's game near the end of the book

tablespoon 4y ago

> You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that. Wouldn't that be fairly easy to detect because the accounts would belong to an isolated, tightly-connected cluster?

quantified 4y ago
State actors can design societies with lots of different clusters. It's all in the simulator you design. And a large actor could easily require that its humans engage with members of this society to connect them with humanity.
deleted 4y ago
[deleted]
sva_ 4y ago
I'm not sure how you would detect those clusters? The language models wouldn't give it away for sure, and getting residential IPs has never been easier than in the present.
- tablespoon 4y ago
  > I'm not sure how you would detect those clusters? The language models wouldn't give it away for sure, and getting residential IPs has never been easier than in the present. Social graph metadata. The GGP said they would all be "talking to each other."
  - sva_ 4y ago
    Yeah I mean they'd talk to each other, but they could also interact with other people using empty platitudes or so that are common on social media. So I think it wouldn't be that easy.
  - Dan_Sylveste 4y ago
    >Social graph metadata. The GGP said they would all be "talking to each other." I think by 'detect' they mean 'detect by people who might fall for it' not 'detect by national security agencies.' I don't see any way for the average social media user to detect whether they're engaging with a botfarm unless the social media company exposes their analytics, and they don't, and they aren't likely to start unless forced.
    - tablespoon 4y ago
      > I think by 'detect' they mean 'detect by people who might fall for it' not 'detect by national security agencies.' I actually was thinking the people who'd do the detecting would be the social media site operator (e.g. Twitter), who would then shut down the inauthentic activity.
      - Dan_Sylveste 4y ago
        Yeah, that isn't going to happen.

corrral 4y ago

I recently read a near-future sci-fi story (linked somewhere on here, I think) about an AI breaking out of its contained environment and taking over the world before we could figure out what happened and stop it (took a couple weeks, IIRC). The TL;DR is that once it had enough compromised machines to run social media botnets, it was all over. It could use those to confound efforts to coordinate and compare data, to misdirect huge numbers of people and cause all kinds of chaos, and to smear opponents before they could get their message out (fakes or actual stolen information—it hardly mattered, all it needed to do was neutralize certain people for a few days). The story contrived to have a secret project that was able to try to resist it after that (spoiler: didn't help) but otherwise the social media botnets were enough for it to buy several days in which no-one was able to effectively work against it.

Vecr 4y ago
Gwern's "It Looks Like You’re Trying To Take Over The World (Clippy)" https://www.gwern.net/fiction/Clippy https://www.gwern.net/fiction/Clippy
Dan_Sylveste 4y ago
That's not a story, that's what's actually happened, only the it wasn't an AI, it was aliens that were discovered in 2018. That's why we're seeing more and more of an open attitude towards UFOs from major governments, in preparation for the big reveal. You will eat the bugs.