4 ms·

The most obvious way the US national security industry could use LLM's right now is simply to spam foreign adversaries with chatbots. That's their greatest stre

The most obvious way the US national security industry could use LLM's right now is simply to spam foreign adversaries with chatbots. That's their greatest strength right now—a use-case they have amply proven themselves for. This paper comes off as eager to avoid this topic: they (briefly) talk about detecting foreign LLM spam, which is called propaganda, but sidestep the idea of our own side using it. If we were considering talking about that, we wouldn't choose negative-sentiment descriptors like (quoting the paper) "nation-state sponsored propaganda", or "disinformation campaigns"; we'd use our own netural-sentiment jargon, which is "psychological operations" ("psyops") [0]. That we're not widely debating this question right now *probably* means it's far too late to have a chance of stopping it. edit: Or, to rephrase this as a question: Is it ethical to spam another democracy with millions of chatbots pretending to be citizens of that country—if the effect is to manipulate those citizens to not go to war with our own side, saving our own lives? Is that an atrocity or is that legitimate warfare? [0] https://en.wikipedia.org/wiki/Psychological_operations_(United_States) https://en.wikipedia.org/wiki/Psychological_operations_(Unit...

joe_the_user 2y ago
Oh, national security professionals aren't going to be talking about psyops, offensive applications and etc, because such things make a given state look bad - they're an offense against democracy and respect for facts and they make the overt media of a given nation look bad. But hey, leave it to HN commentators to root for taking the gloves off. Not to worry post, I'd bet dollars to donuts the actual classified discussions of such things aren't worried about such niceties. But even more, in those activities of the US and other secret states, that have come to light, these state have propagandized not only enemy populations but also their own. After, have to counter enemies trying to nefariously prevent wars as well.
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- davisp 2y ago
  [flagged]
bilbo0s 2y ago
Or, to rephrase this as a question: Is it ethical to spam another democracy with millions of chatbots pretending to be citizens of that country—if the effect is to manipulate those citizens to [take any action advantageous to US national interest]...? Just, Devil's Advocate, but ethical or not, that's what we should be doing and what we are doing. Every nation has its sock puppets out there, our job is to stop everyone else' sock puppets, and do everything we can to extend the reach of our own sock puppets.
- JoshTriplett 2y ago
  That's not inherently true. If there were a way to reliably destroy all the sock puppets, we should, and the world would be better off. For instance, reliable bot detection, or mechanisms by which major social networks could detect and prohibit bot-like activity that isn't labeled as such.
  - exe34 2y ago
    charge one penny for every post. most people can afford it. bots become less cost effective and you'd be able to trace the source of funds.
    - recursive 2y ago
      Even if they could afford it, they won't. The UX friction of charging money would send engagement off a cliff, even if the nominal charge was $0.
      - exe34 2y ago
        that's why it should be mandated by law for national security reasons!
      - hatthew 2y ago
        Only half joking, I feel like the majority of comments on the internet are garbage and not worth the time to read, and increasing friction with a nominal charge isn't necessarily a terrible thing.
Onavo 2y ago
The platform you are referring to is called "Reddit", one of YC's portfolio companies.
jingyibo123 2y ago
I'm pretty sure that before the era of LLM, hired personnel and computer chatbots already exists on major social media platform to serve as "opinion influencers", either for political or commercial purposes. Just that chatbot algorithms are kid of naive, and have been "fished" on several occasions.... The usage of LLM is too obvious, and way cheaper than hired personnel, so it's like "prism". They don't want us to talk about it because it's been going on for so long...
- immibis 2y ago
  They have, and it scales orders of magnitude better with LLMs.
PittleyDunkin 2y ago
I'm sure they're already used against both foreign and domestic populations.

joe_the_user 2y ago

Oh, national security professionals aren't going to be talking about psyops, offensive applications and etc, because such things make a given state look bad - they're an offense against democracy and respect for facts and they make the overt media of a given nation look bad. But hey, leave it to HN commentators to root for taking the gloves off. Not to worry post, I'd bet dollars to donuts the actual classified discussions of such things aren't worried about such niceties. But even more, in those activities of the US and other secret states, that have come to light, these state have propagandized not only enemy populations but also their own. After, have to counter enemies trying to nefariously prevent wars as well.

deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
davisp 2y ago
[flagged]

bilbo0s 2y ago

Or, to rephrase this as a question: Is it ethical to spam another democracy with millions of chatbots pretending to be citizens of that country—if the effect is to manipulate those citizens to [take any action advantageous to US national interest]...? Just, Devil's Advocate, but ethical or not, that's what we should be doing and what we are doing. Every nation has its sock puppets out there, our job is to stop everyone else' sock puppets, and do everything we can to extend the reach of our own sock puppets.

JoshTriplett 2y ago
That's not inherently true. If there were a way to reliably destroy all the sock puppets, we should, and the world would be better off. For instance, reliable bot detection, or mechanisms by which major social networks could detect and prohibit bot-like activity that isn't labeled as such.
- exe34 2y ago
  charge one penny for every post. most people can afford it. bots become less cost effective and you'd be able to trace the source of funds.
  - recursive 2y ago
    Even if they could afford it, they won't. The UX friction of charging money would send engagement off a cliff, even if the nominal charge was $0.
    - exe34 2y ago
      that's why it should be mandated by law for national security reasons!
    - hatthew 2y ago
      Only half joking, I feel like the majority of comments on the internet are garbage and not worth the time to read, and increasing friction with a nominal charge isn't necessarily a terrible thing.

Onavo 2y ago

The platform you are referring to is called "Reddit", one of YC's portfolio companies.

jingyibo123 2y ago

I'm pretty sure that before the era of LLM, hired personnel and computer chatbots already exists on major social media platform to serve as "opinion influencers", either for political or commercial purposes. Just that chatbot algorithms are kid of naive, and have been "fished" on several occasions.... The usage of LLM is too obvious, and way cheaper than hired personnel, so it's like "prism". They don't want us to talk about it because it's been going on for so long...

immibis 2y ago
They have, and it scales orders of magnitude better with LLMs.

PittleyDunkin 2y ago

I'm sure they're already used against both foreign and domestic populations.