4 ms·

Reading the comments on this article make me realize how unserious most engineers and software developers are taking this. These threats are real. The weaponi

Reading the comments on this article make me realize how unserious most engineers and software developers are taking this. These threats are real. The weaponization of everything is actually happening. Since I wrote about it a month ago (Self-Crashing Cars) a number of people have reached out including people with actual insight into the military aspect of it. Militaries around the world are getting ready for true AI enabled weapon systems and there building deterrence strategies for mass casualty cyber attack (including nuclear weapons response), whether its from hacked industrial plants or cars it doesn't matter. They're actually talking about the weaponization of cars at the Munich security conference. We need to stop burying our head in the sand and write to our politicians about this threat. I know it sounds crazy but it's real. As an aside, my main complaint about the people that truly understand this the inability / unwillingness to accept that the act of subverting systems capable of mass destruction via cyber attack amounts to cyber weapons of mass destruction. We need to assemble all the work / treaties / regulations / research we put into securing nukes into securing AI / robotics and for the identical reasons. We know how this ends otherwise. We need wide-spread government funding and we need to communicate what these things are in language that our governments understand. Not saying something that is true just because it sounds weird is counterproductive.

evoloution 9y ago
I think you are absolutely correct. I think that people who are more entrenched in the field are even subconsciously unwilling to accept what is to come knowing that the unavoidable regulation will limit their creativity and just make their lives more miserable. But they know...
AndrewKemendo 9y ago
You can trust that people in the US Military are taking this very seriously. I know because I, alongside LtGen Shanahan and Eric Schmidt briefed Secretary Mattis personally about these issues and is part of my ongoing work within the IC/DoD. You're correct though that most line DL engineers don't have these issues at top of mind. I don't know any ML researcher worth their salt that hasn't thought about them, but the tendency is to brush the concerns off until we're closer to more generalized systems. That is not a wholly unreasonable position for many reasons, especially given the history of hype around AI, but I'd like to see more discussion happening between junior and mid-level engineers about these things - and especially more work being done about human-machine interfaces in the AI context, of which very little design thought put in.
kakarot 9y ago
Here's the crux of the issue with respect to nuclear WMDs: In order to prevent citizens and other countries from creating nuclear bombs, our government severely limits access to relevant tools and materials, and actively seeks to censor the knowledge of how to build modern nuclear devices from other parties. In order to prevent citizens and other countries from creating dangerous rogue AIs, our government _____________ What do you think goes in that blank?
- DougN7 9y ago
  This is the problem. WMDs take a lot of infrastructure and can be tested/inspected. AIs can be built/modified by anyone in their basement. The genie can't be put back into the bottle. It's like trying to outlaw computer viruses and hoping that will work. I don't have a solution :(
  - kakarot 9y ago
    Perhaps this is the Great Filter. Nuclear war, societal collapse, supervirus... all these can come from rogue / misused AI
  - AJ007 9y ago
    AI needs compute cycles. These are quite centralized outside of botnets. I’ve pointed this out before, but the primary commercial use cases of AI right now are malicious: mass population surveillance and behavior manipulation (ads; political and otherwise.) My biggest concern are the ML researchers who dismiss malicious use as being a distant concern while they work on projects that are malicious right now.
    - kakarot 9y ago
      Homomorphic encryption + cloud services + VPN chaining. If there's one thing that won't be in shortage for the foreseeable future, it's compute cycles. And I'm with you, the socioeconomic aspects of AI are far more important to curtail, but it doens't mean we should be ignoring the very real threat of people attacking infrastructure, sewage and power plants. And we shouldn't be waiting until after it's a problem, when it already can be seen as a certainty that it will happen eventually.
shkkmo 9y ago
> We need to assemble all the work / treaties / regulations / research we put into securing nukes into securing AI / robotics and for the identical reasons. I disagree. I think we need a fundamentally different approach to AI than to nuclear weapons. The proliferation of nuclear weapons were controllable because the components needed to develop a nuclear weapon included specialized, controllable physical goods and fairly recognizable industrial installations. I do agree that we need international treaties prohibiting the development and use of AI weapons technology to avoid encouraging an arms race. However, I think that trying to prevent the spread of AI tools and technology will face similar problems to the US's attempts to prevent the spread of encryption tools and technology. It is fundamentally harder to control the spread of information that the spread of physical goods.
- dotancohen 9y ago
  > I do agree that we need international treaties prohibiting the development and use of AI weapons technology to avoid encouraging an arms race. International treaties prohibiting the development and use of missile technology to prevent an arms race in the fifties and sixties would have prevented man from landing on the moon. We called that arms race the Space Race to sell it to the public. Maybe we do need an AI arms race.
BatFastard 9y ago
Read the book Daemon is you want to see creative use of this type of tool.

evoloution 9y ago

I think you are absolutely correct. I think that people who are more entrenched in the field are even subconsciously unwilling to accept what is to come knowing that the unavoidable regulation will limit their creativity and just make their lives more miserable. But they know...

AndrewKemendo 9y ago

You can trust that people in the US Military are taking this very seriously. I know because I, alongside LtGen Shanahan and Eric Schmidt briefed Secretary Mattis personally about these issues and is part of my ongoing work within the IC/DoD. You're correct though that most line DL engineers don't have these issues at top of mind. I don't know any ML researcher worth their salt that hasn't thought about them, but the tendency is to brush the concerns off until we're closer to more generalized systems. That is not a wholly unreasonable position for many reasons, especially given the history of hype around AI, but I'd like to see more discussion happening between junior and mid-level engineers about these things - and especially more work being done about human-machine interfaces in the AI context, of which very little design thought put in.

kakarot 9y ago

Here's the crux of the issue with respect to nuclear WMDs: In order to prevent citizens and other countries from creating nuclear bombs, our government severely limits access to relevant tools and materials, and actively seeks to censor the knowledge of how to build modern nuclear devices from other parties. In order to prevent citizens and other countries from creating dangerous rogue AIs, our government _____________ What do you think goes in that blank?

DougN7 9y ago
This is the problem. WMDs take a lot of infrastructure and can be tested/inspected. AIs can be built/modified by anyone in their basement. The genie can't be put back into the bottle. It's like trying to outlaw computer viruses and hoping that will work. I don't have a solution :(
- kakarot 9y ago
  Perhaps this is the Great Filter. Nuclear war, societal collapse, supervirus... all these can come from rogue / misused AI
- AJ007 9y ago
  AI needs compute cycles. These are quite centralized outside of botnets. I’ve pointed this out before, but the primary commercial use cases of AI right now are malicious: mass population surveillance and behavior manipulation (ads; political and otherwise.) My biggest concern are the ML researchers who dismiss malicious use as being a distant concern while they work on projects that are malicious right now.
  - kakarot 9y ago
    Homomorphic encryption + cloud services + VPN chaining. If there's one thing that won't be in shortage for the foreseeable future, it's compute cycles. And I'm with you, the socioeconomic aspects of AI are far more important to curtail, but it doens't mean we should be ignoring the very real threat of people attacking infrastructure, sewage and power plants. And we shouldn't be waiting until after it's a problem, when it already can be seen as a certainty that it will happen eventually.

shkkmo 9y ago

> We need to assemble all the work / treaties / regulations / research we put into securing nukes into securing AI / robotics and for the identical reasons. I disagree. I think we need a fundamentally different approach to AI than to nuclear weapons. The proliferation of nuclear weapons were controllable because the components needed to develop a nuclear weapon included specialized, controllable physical goods and fairly recognizable industrial installations. I do agree that we need international treaties prohibiting the development and use of AI weapons technology to avoid encouraging an arms race. However, I think that trying to prevent the spread of AI tools and technology will face similar problems to the US's attempts to prevent the spread of encryption tools and technology. It is fundamentally harder to control the spread of information that the spread of physical goods.

dotancohen 9y ago
> I do agree that we need international treaties prohibiting the development and use of AI weapons technology to avoid encouraging an arms race. International treaties prohibiting the development and use of missile technology to prevent an arms race in the fifties and sixties would have prevented man from landing on the moon. We called that arms race the Space Race to sell it to the public. Maybe we do need an AI arms race.

BatFastard 9y ago

Read the book Daemon is you want to see creative use of this type of tool.