8 ms·

Does this mean that I could train an LLM to do something like spread fake news? Would that even scale?

by ryaclifton 3y ago

laverya 3y ago
Isn't this done with every "sanitized" LLM? Fake news is all according to perspective!
- jdiff 3y ago
  No, it isn't. This is akin to saying that the truth is relative and lies somewhere between "the Earth is an oblate spheroid" and "the Earth is flat." Perception and perspective varies, sure, but fact exists regardless. Fake news is falsified news built on fabricated fakes, and is not just alternative viewpoints. Do not normalize this.
  - laverya 3y ago
    "Japan has a higher GDP per capita than Alabama" is fake news. It's also confidently repeated by most LLMs. https://twitter.com/MatthewJBar/status/1681554646664634368 https://twitter.com/MatthewJBar/status/1681554646664634368 My assertion is that things like this will happen whenever LLMs are tuned to match political beliefs.
    - jdiff 3y ago
      Nobody fed that false fact into any LLM. Garbage in, garbage out isn't fake news.
sixothree 3y ago
Probably. And you could surround specific communities en masse. And it’s coming soon to every single site near you.
- jacquesm 3y ago
  More scary: you could target individuals and surround them with a bunch of fake persons that they have no way of differentiating from real ones and slowly push them in a direction of your choosing.
  - reaperman 3y ago
    Even more scary: You can tailor each individual's completely unique online universe to dovetail with the equally unique online universes of their IRL social connections / networks so that when they get together again at the holidays or meet for the first time at a bar they both frequent, they have serendipitous conversations about discovering the same hyper-niche thing recently, reinforcing their online conditioning with real-life anchoring. This could be pervasive through not just online discussion forums, but also online news articles, auto-generated YouTube/TikTok feeds, pages inserted into search results that are custom generated on-demand, conversations on dating apps, etc.
    - jacquesm 3y ago
      Exactly. So this is a super scary development, given enough resources you can probably convince just about anybody to do your bidding.
quickthrower2 3y ago
Pretty easy. Probably no additional training is required! You probably would need to just get hold of a foundation model that has no AI safety type training done on it. Then ask it nicely. You could also feed it in context some examples of the fake news you would like. And maybe the style. "Here is a BBC article, write an article that Elon Musk plans to visit a black hole by 2030 in this style".
- version_five 3y ago
  You could also just use a text editor to write a fake news story, or pay $5 to a freelancer to write it if you're busy. I don't understand why people belive llms fundamentally change anything. Worst case scenario they make you slightly more efficient at your malfeasance, just like they do with legit tasks.
  - quickthrower2 3y ago
    The cost of a million FUD spreading comments online for an election campaign just went down from $5m to maybe $100 of compute time.
  - spookthesunset 3y ago
    Not just slightly more but likely much more. You can automate the hell out of an automated propaganda bot in a way you could never do with a smoke filled room of $5/day workers.
Terr_ 3y ago
When you think about it, making fake news is orders of magnitude easier than making real news, the same way that a broken calculator is easier than a correct one. That said, I'm assuming you also mean fake news which is (A) believable and (B) is tailored for a particular agenda.
- dTal 3y ago
  Language models make nothing but fake news.
j16sdiz 3y ago
Some template-based fake news generation technique are working pretty well. It don't have to be very sophisticated to be effective. Would it scale? Sure it would.

Does this mean that I could train an LLM to do something like spread fake news? Would that even scale?

laverya 3y ago

Isn't this done with every "sanitized" LLM? Fake news is all according to perspective!

jdiff 3y ago
No, it isn't. This is akin to saying that the truth is relative and lies somewhere between "the Earth is an oblate spheroid" and "the Earth is flat." Perception and perspective varies, sure, but fact exists regardless. Fake news is falsified news built on fabricated fakes, and is not just alternative viewpoints. Do not normalize this.
- laverya 3y ago
  "Japan has a higher GDP per capita than Alabama" is fake news. It's also confidently repeated by most LLMs. https://twitter.com/MatthewJBar/status/1681554646664634368 https://twitter.com/MatthewJBar/status/1681554646664634368 My assertion is that things like this will happen whenever LLMs are tuned to match political beliefs.
  - jdiff 3y ago
    Nobody fed that false fact into any LLM. Garbage in, garbage out isn't fake news.

sixothree 3y ago

Probably. And you could surround specific communities en masse. And it’s coming soon to every single site near you.

jacquesm 3y ago
More scary: you could target individuals and surround them with a bunch of fake persons that they have no way of differentiating from real ones and slowly push them in a direction of your choosing.
- reaperman 3y ago
  Even more scary: You can tailor each individual's completely unique online universe to dovetail with the equally unique online universes of their IRL social connections / networks so that when they get together again at the holidays or meet for the first time at a bar they both frequent, they have serendipitous conversations about discovering the same hyper-niche thing recently, reinforcing their online conditioning with real-life anchoring. This could be pervasive through not just online discussion forums, but also online news articles, auto-generated YouTube/TikTok feeds, pages inserted into search results that are custom generated on-demand, conversations on dating apps, etc.
  - jacquesm 3y ago
    Exactly. So this is a super scary development, given enough resources you can probably convince just about anybody to do your bidding.

quickthrower2 3y ago

Pretty easy. Probably no additional training is required! You probably would need to just get hold of a foundation model that has no AI safety type training done on it. Then ask it nicely. You could also feed it in context some examples of the fake news you would like. And maybe the style. "Here is a BBC article, write an article that Elon Musk plans to visit a black hole by 2030 in this style".

version_five 3y ago
You could also just use a text editor to write a fake news story, or pay $5 to a freelancer to write it if you're busy. I don't understand why people belive llms fundamentally change anything. Worst case scenario they make you slightly more efficient at your malfeasance, just like they do with legit tasks.
- quickthrower2 3y ago
  The cost of a million FUD spreading comments online for an election campaign just went down from $5m to maybe $100 of compute time.
- spookthesunset 3y ago
  Not just slightly more but likely much more. You can automate the hell out of an automated propaganda bot in a way you could never do with a smoke filled room of $5/day workers.

Terr_ 3y ago

When you think about it, making fake news is orders of magnitude easier than making real news, the same way that a broken calculator is easier than a correct one. That said, I'm assuming you also mean fake news which is (A) believable and (B) is tailored for a particular agenda.

dTal 3y ago
Language models make nothing but fake news.

j16sdiz 3y ago

Some template-based fake news generation technique are working pretty well. It don't have to be very sophisticated to be effective. Would it scale? Sure it would.