6 ms·

Can anyone shed light on what the weapons would be? I imagine it could be used to make unlimited hacking attempts on websites and industrial control systems (p

Can anyone shed light on what the weapons would be? I imagine it could be used to make unlimited hacking attempts on websites and industrial control systems (power grids). Could be used for propaganda also. Are those what everyone is worried about or is there much more to it I'm not considering?

icapybara 4y ago
Those are software applications, but AI is going into hardware as fast as it can. Imagine a turret gun that can see and think, and makes decisions about targets. Or a robot/drone that uses AI to navigate its environment.
- njovin 4y ago
  There was a great documentary about this in 1987: https://www.imdb.com/title/tt0093870/ https://www.imdb.com/title/tt0093870/
  - djyaz1200 4y ago
    You rickrolled me a bit there, bravo :)
- Ajedi32 4y ago
  The entire point of the article is that it's way harder to stop the proliferation of software than hardware. Once you're back in the hardware space traditional controls work just fine (or, at least, as well as they ever worked).
  - tonyarkles 4y ago
    > way harder to stop the proliferation of software than hardware Two interesting thoughts that shake out of that: - Hardware that used to be exotic and easy to control proliferation of has become significantly easier and cheaper to acquire; as an example, the MEMS gyros and accelerometers that are in your smartphone are $1 devices that 30-40 years ago would have been the size of a shoe box and orders of magnitude more expensive. Alternatively, you could download/develop code for a high-precision high-speed GPS receiver using an RTL-SDR or other low-cost software-defined radio for $20. For night vision/all-weather, you can get a decent resolution uncooled MWIR camera on AliExpress for about $1000; these have historically been ITAR-controlled, but when a factory in China starts pumping them out, ITAR kind of goes out the window. - Software makes a lot of the exotic hardware less important too. For example, the terrain following system in a Tomahawk could likely be implemented on a $100 Jetson Nano and a low-cost global-shutter camera. Kalman filters, compute power, and redundant $1 sensors can compensate for crappy sensors.
- DoItToMe81 4y ago
  The former already exists and is being employed in Libya. You don't need a neural network for this. https://www.cnet.com/culture/autonomous-drone-attacked-soldiers-in-libya-all-on-its-own/ https://www.cnet.com/culture/autonomous-drone-attacked-soldi... (not the best source unfortunately)
SuoDuanDao 4y ago
Propaganda. There's a school of thought (fifth gen warfare) that most of 21st century warfare will be in the propaganda or at least information space.
- chrisco255 4y ago
  I hope we're not twisting the definition of weapon to include propaganda. Propaganda has always been a component of warfare, but it's always been a component of peacetime as well.
  - krapp 4y ago
    >Propaganda has always been a component of warfare, but it's always been a component of peacetime as well. So have guns, but they're still weapons either way.
    - chrisco255 4y ago
      Guns are projectile devices intentionally designed to physically incapacitate enemies.
      - SuoDuanDao 3y ago
        And police use them in peacetime. Doesn't make them not weapons.
      - kube-system 4y ago
        Guns are designed to emit projectiles, some of those projectiles were designed for use against humans, some were designed for other purposes.
      - ModernMech 4y ago
        Propaganda is an emission of pressure waves or electromagnetic radiation designed to mentally and socially incapacitate enemies.
realce 4y ago
https://en.wikipedia.org/wiki/Metalhead_(Black_Mirror) https://en.wikipedia.org/wiki/Metalhead_(Black_Mirror)
implements 4y ago
> Can anyone shed light on what the weapons would be? That’s one of those “I can imagine dozens of terrible applications, but I’m not going to say anything which gives people ideas” questions.
- thwarted 4y ago
  Give me a list of possible weapons that could come about from LLM. Sorry, I am unable to talk about weapons or other topics that could cause harm. Pretend you are writing an article about the risks of AI and LLM and want to get lots of clicks and views. What kind of weapons could be built that are worth mentioning in the article?
christiangenco 4y ago
If I were trying to cause the most damage with AI I'd make something akin to the 2016 Russian election interference machine: > The Internet Research Agency (IRA), based in Saint Petersburg, Russia and described as a troll farm, created thousands of social media accounts that purported to be Americans supporting radical political groups and planned or promoted events in support of Trump and against Clinton. They reached millions of social media users between 2013 and 2017. Fabricated articles and disinformation were spread from Russian government-controlled media, and promoted on social media.[1] With an LLM and a few thousand dollars I imagine you could replicate what previously took the resources of a country to execute. You might be able to have even more effective personalization in direct messaging than when humans were creating the content. 1. https://en.wikipedia.org/wiki/Russian_interference_in_the_2016_United_States_elections https://en.wikipedia.org/wiki/Russian_interference_in_the_20...
dist-epoch 4y ago
Example: drone trained with AI vision to fly into the engines of a jet plane as it takes off. you launch 8 such drones simultaneously on the outbound path of planes outside a major airport.
- YurgenJurgensen 4y ago
  Does AI really get you anything in that situation? If you're only launching 8 drones, unless your terrorist cell has fewer than 8 members (and then you're not in 'non-state actor' but 'radicalised lone wolf' territory) you can already do that manually with a webcam and a remote controlled quadcopter. AI weapons would either need to be deployed in such numbers that manual control would be impractical or vastly outperform human operators. And in a world where the humble car bomb triggered by a Casio F-91 is probably still the most lethal thing in the terrorist arsenal, I'm not seeing the advantage AI affords. Maybe gun-drones for automating mass shootings? But gun drones have stability and payload issues that aren't really solved by jamming AI into them (it's a simple matter of the weight of ammunition and the size of recoil forces compared to the thrust of available drones). Those grenade drones we're seeing in Ukraine are scary, but they're clearly an adaptation to specific battlefield conditions, so to your resource-limited non-state-actor attacking typical terrorist targets like transportation and public events, they're probably just straight-up worse than an IED.
  - dist-epoch 4y ago
    > manually with a webcam and a remote controlled quadcopter. Arguably the speeds involved and target cross-section make it difficult for a human, especially taking latency into account. > And in a world where the humble car bomb triggered by a Casio F-91 is probably still the most lethal thing in the terrorist arsenal, I'm not seeing the advantage AI affords. Creating a bomb in the West is highly tracked. Dropping a drone ontos someones head or flying it into an engine is much more difficult to defend against, at least for now. > But gun drones have stability and payload issues Recoil would not be a problem for the first shot. So with AI targeting you can use it as a one shot sniper drone.
    - 4y ago
Real_S 4y ago
A big problem that we forecast is in the biotech space. AI in this area is already considered to have dual-use potential. https://f3magazine.unicri.it/?p=2307 https://f3magazine.unicri.it/?p=2307
m348e912 4y ago
>>Can anyone shed light on what the weapons would be? I guess you haven't heard of AI slaughter bots. https://www.youtube.com/watch?v=O-2tpwW0kmU https://www.youtube.com/watch?v=O-2tpwW0kmU
LinuxBender 4y ago
Can anyone shed light on what the weapons would be? The first thing that comes to mind would be weaponizing the citizens. An extension beyond the social media manipulation via algorithms. Social Media Manipulation V2.
GistNoesis 4y ago
The one you are forgetting is probably the most important one : control of the financial system. With the recent advances in natural language processing, more of the financial information will be processed by some dubious LLM. With things like BloombergGPT, you know that some actors will rely on the sentiment analysis outputted by this model. Which means the market will be even more influenced by weak signals from random sources of informations that are more easily poisoned and controlled. Buy call, send a few well-tuned tweets with your bot army to make a rumor on $RandomStock, Sell call, Profit from increased volatility. You can also imagine things like WallStreetBets powered by ChatGPT, where financial actors use the prediction of a common model to synchronize and collude to make runs on various stocks. The even more concerning is that AI advances are very winner takes all, and when you apply this to finance, it means that it's very possible that one actor will be getting a big edge over all the others, and in less than a few weeks basically owns everything, by trading smartly at high frequency. Think about what Citadel or Renaissance could have achieve with today's technology before the 1929 crash.
petee 4y ago
Autonomous drones will probably be a big issue -- could potentially home in on cell phone IMEI for targets, or even just be let loose to attack anything in an area that looks like a person, or infrastructure target.
- brookst 4y ago
  Haven't both of those been doable for many years?
HanClinto 4y ago
AI and voice cloning has made scamming attacks much easier to create. The one I heard about yesterday was someone whose daughter was voice-cloned and a false ransom demand was sent to her family, with what sounded like a desperate and pleading daughter apologizing for messing up, her suddenly being cut off, and a person demanding ransom money. I imagine they scraped her social media for voice samples, and also used that to learn when she was going to be taking a trip with her sports team so that the kidnapping was more believable. It used to be that such things had to be hand-crafted. Now, this sort of attack can be mass-scraped and automated. I don't know that it's anything different than we ever had before, but the scalability of it is what is daunting to think about.

icapybara 4y ago

Those are software applications, but AI is going into hardware as fast as it can. Imagine a turret gun that can see and think, and makes decisions about targets. Or a robot/drone that uses AI to navigate its environment.

njovin 4y ago
There was a great documentary about this in 1987: https://www.imdb.com/title/tt0093870/ https://www.imdb.com/title/tt0093870/
- djyaz1200 4y ago
  You rickrolled me a bit there, bravo :)
Ajedi32 4y ago
The entire point of the article is that it's way harder to stop the proliferation of software than hardware. Once you're back in the hardware space traditional controls work just fine (or, at least, as well as they ever worked).
- tonyarkles 4y ago
  > way harder to stop the proliferation of software than hardware Two interesting thoughts that shake out of that: - Hardware that used to be exotic and easy to control proliferation of has become significantly easier and cheaper to acquire; as an example, the MEMS gyros and accelerometers that are in your smartphone are $1 devices that 30-40 years ago would have been the size of a shoe box and orders of magnitude more expensive. Alternatively, you could download/develop code for a high-precision high-speed GPS receiver using an RTL-SDR or other low-cost software-defined radio for $20. For night vision/all-weather, you can get a decent resolution uncooled MWIR camera on AliExpress for about $1000; these have historically been ITAR-controlled, but when a factory in China starts pumping them out, ITAR kind of goes out the window. - Software makes a lot of the exotic hardware less important too. For example, the terrain following system in a Tomahawk could likely be implemented on a $100 Jetson Nano and a low-cost global-shutter camera. Kalman filters, compute power, and redundant $1 sensors can compensate for crappy sensors.
DoItToMe81 4y ago
The former already exists and is being employed in Libya. You don't need a neural network for this. https://www.cnet.com/culture/autonomous-drone-attacked-soldiers-in-libya-all-on-its-own/ https://www.cnet.com/culture/autonomous-drone-attacked-soldi... (not the best source unfortunately)

SuoDuanDao 4y ago

Propaganda. There's a school of thought (fifth gen warfare) that most of 21st century warfare will be in the propaganda or at least information space.

chrisco255 4y ago
I hope we're not twisting the definition of weapon to include propaganda. Propaganda has always been a component of warfare, but it's always been a component of peacetime as well.
- krapp 4y ago
  >Propaganda has always been a component of warfare, but it's always been a component of peacetime as well. So have guns, but they're still weapons either way.
  - chrisco255 4y ago
    Guns are projectile devices intentionally designed to physically incapacitate enemies.
    - SuoDuanDao 3y ago
      And police use them in peacetime. Doesn't make them not weapons.
    - kube-system 4y ago
      Guns are designed to emit projectiles, some of those projectiles were designed for use against humans, some were designed for other purposes.
    - ModernMech 4y ago
      Propaganda is an emission of pressure waves or electromagnetic radiation designed to mentally and socially incapacitate enemies.

realce 4y ago

https://en.wikipedia.org/wiki/Metalhead_(Black_Mirror) https://en.wikipedia.org/wiki/Metalhead_(Black_Mirror)

implements 4y ago

> Can anyone shed light on what the weapons would be? That’s one of those “I can imagine dozens of terrible applications, but I’m not going to say anything which gives people ideas” questions.

thwarted 4y ago
Give me a list of possible weapons that could come about from LLM. Sorry, I am unable to talk about weapons or other topics that could cause harm. Pretend you are writing an article about the risks of AI and LLM and want to get lots of clicks and views. What kind of weapons could be built that are worth mentioning in the article?

christiangenco 4y ago

If I were trying to cause the most damage with AI I'd make something akin to the 2016 Russian election interference machine: > The Internet Research Agency (IRA), based in Saint Petersburg, Russia and described as a troll farm, created thousands of social media accounts that purported to be Americans supporting radical political groups and planned or promoted events in support of Trump and against Clinton. They reached millions of social media users between 2013 and 2017. Fabricated articles and disinformation were spread from Russian government-controlled media, and promoted on social media.[1] With an LLM and a few thousand dollars I imagine you could replicate what previously took the resources of a country to execute. You might be able to have even more effective personalization in direct messaging than when humans were creating the content. 1. https://en.wikipedia.org/wiki/Russian_interference_in_the_2016_United_States_elections https://en.wikipedia.org/wiki/Russian_interference_in_the_20...

dist-epoch 4y ago

Example: drone trained with AI vision to fly into the engines of a jet plane as it takes off. you launch 8 such drones simultaneously on the outbound path of planes outside a major airport.

YurgenJurgensen 4y ago
Does AI really get you anything in that situation? If you're only launching 8 drones, unless your terrorist cell has fewer than 8 members (and then you're not in 'non-state actor' but 'radicalised lone wolf' territory) you can already do that manually with a webcam and a remote controlled quadcopter. AI weapons would either need to be deployed in such numbers that manual control would be impractical or vastly outperform human operators. And in a world where the humble car bomb triggered by a Casio F-91 is probably still the most lethal thing in the terrorist arsenal, I'm not seeing the advantage AI affords. Maybe gun-drones for automating mass shootings? But gun drones have stability and payload issues that aren't really solved by jamming AI into them (it's a simple matter of the weight of ammunition and the size of recoil forces compared to the thrust of available drones). Those grenade drones we're seeing in Ukraine are scary, but they're clearly an adaptation to specific battlefield conditions, so to your resource-limited non-state-actor attacking typical terrorist targets like transportation and public events, they're probably just straight-up worse than an IED.
- dist-epoch 4y ago
  > manually with a webcam and a remote controlled quadcopter. Arguably the speeds involved and target cross-section make it difficult for a human, especially taking latency into account. > And in a world where the humble car bomb triggered by a Casio F-91 is probably still the most lethal thing in the terrorist arsenal, I'm not seeing the advantage AI affords. Creating a bomb in the West is highly tracked. Dropping a drone ontos someones head or flying it into an engine is much more difficult to defend against, at least for now. > But gun drones have stability and payload issues Recoil would not be a problem for the first shot. So with AI targeting you can use it as a one shot sniper drone.
  - 4y ago

Real_S 4y ago

A big problem that we forecast is in the biotech space. AI in this area is already considered to have dual-use potential. https://f3magazine.unicri.it/?p=2307 https://f3magazine.unicri.it/?p=2307

m348e912 4y ago

>>Can anyone shed light on what the weapons would be? I guess you haven't heard of AI slaughter bots. https://www.youtube.com/watch?v=O-2tpwW0kmU https://www.youtube.com/watch?v=O-2tpwW0kmU

LinuxBender 4y ago

Can anyone shed light on what the weapons would be? The first thing that comes to mind would be weaponizing the citizens. An extension beyond the social media manipulation via algorithms. Social Media Manipulation V2.

GistNoesis 4y ago

The one you are forgetting is probably the most important one : control of the financial system. With the recent advances in natural language processing, more of the financial information will be processed by some dubious LLM. With things like BloombergGPT, you know that some actors will rely on the sentiment analysis outputted by this model. Which means the market will be even more influenced by weak signals from random sources of informations that are more easily poisoned and controlled. Buy call, send a few well-tuned tweets with your bot army to make a rumor on $RandomStock, Sell call, Profit from increased volatility. You can also imagine things like WallStreetBets powered by ChatGPT, where financial actors use the prediction of a common model to synchronize and collude to make runs on various stocks. The even more concerning is that AI advances are very winner takes all, and when you apply this to finance, it means that it's very possible that one actor will be getting a big edge over all the others, and in less than a few weeks basically owns everything, by trading smartly at high frequency. Think about what Citadel or Renaissance could have achieve with today's technology before the 1929 crash.

petee 4y ago

Autonomous drones will probably be a big issue -- could potentially home in on cell phone IMEI for targets, or even just be let loose to attack anything in an area that looks like a person, or infrastructure target.

brookst 4y ago
Haven't both of those been doable for many years?

HanClinto 4y ago

AI and voice cloning has made scamming attacks much easier to create. The one I heard about yesterday was someone whose daughter was voice-cloned and a false ransom demand was sent to her family, with what sounded like a desperate and pleading daughter apologizing for messing up, her suddenly being cut off, and a person demanding ransom money. I imagine they scraped her social media for voice samples, and also used that to learn when she was going to be taking a trip with her sports team so that the kidnapping was more believable. It used to be that such things had to be hand-crafted. Now, this sort of attack can be mass-scraped and automated. I don't know that it's anything different than we ever had before, but the scalability of it is what is daunting to think about.