5 ms·

Yes, nothing could possibly go wrong with having a military minded AI.

djohnston 6y ago
it's as inevitable as nuclear proliferation following WW2. you can slow it down, try and create a few key players and build around that, but it's always coming.
fermienrico 6y ago
This is a serious dilemma because: - If you don't, then the enemy will - If you do, then you're inviting the enemy to do the same So, without trust and mutual agreement on an international level, this is a self-propelling premise that cannot be stopped. Game theory 101.
- landryraccoon 6y ago
  Even in that international agreement, someone still has the weapons. International agreements failed to keep nuclear weapons out of the hands of the strongest nations because why would they ever give up their nukes?
dogma1138 6y ago
It’s not minded, it’s not any different in than alphago or alphastar or the OpenAI dota bots, it doesn’t understand anything outside of the context of the game it’s playing. It doesn’t have a mind. On the “intelligence” spectrum it’s much closer to the guidance system in a heat seeking missile to than say even a dog, yet alone anything that we would define as having cognition.
- irrational 6y ago
  Well, OP was talking about billions of dollars of DARPA funding going towards military AI research. I presume they would be trying to get much closer to cognition.
  - dogma1138 6y ago
    Why would you want cognition? Besides the billions of edge cases it may create why would you want to make a tool you can or would have to reason with?
    - inetknght 6y ago
      > why would you want to make a tool you can or would have to reason with? Some might ask the same question in the context of human children.
  - philwelch 6y ago
    I would question that assumption. Autonomous military equipment (vehicles, planes, etc.) doesn’t require cognition in order to be useful.
  - CarbyAu 6y ago
    I obviously can't speak for Defence but my understanding was that a lot of the training et al was so that high command could have predictable outcomes. They may give troops on the front line some leeway, but within parameters for pursuit of a goal. I don't think Defence wants "uncontrolled" anything. Even if they manage to develop fully cognizant AI they will want to know the exact parameters it can work within. Of course, humans make errors...
- Lammy 6y ago
  > It’s not minded, it’s not any different in than alphago or alphastar or the OpenAI dota bots Aside from, like, the guns 'n' stuff.
- alquemist 6y ago
  Where the game is 'total world domination'. Through any means necessary, in particular by utterly destroying one's physical adversaries. Arguably the human brain development is a byproduct of an arm's race between social BIs, fighting off the next tribe since time immemorial. Time to up the ante and build smarter and stronger entities. Somebody's got to win the game, this nuclear stalemate we're living through is too boring. Nothing to worry about, move along.
  - philwelch 6y ago
    I think there’s a huge difference between a fighter jet that can fly itself and Skynet.
    - alquemist 6y ago
      Short term, runaway hostile killer drone swarm. Medium term, it's an arms race to build the smartest, strongest, most ruthless global killer drone swarm. The outcome of that race is anybody's guess.
      - PhantomGremlin 6y ago
        Bah. I think recent events have clearly demonstrated that biological weapons are much much cheaper. create a killer virus inoculate your people release it at the rest of the world profit?
        ceejayoz 6y ago
        Sounds like a good way to get nuked by the dying rest of the world, not profit.
- extropy 6y ago
  Yes and that's a big part of the problem. It does not have a concept of "after" the victory. So if it gains autonomy there is high chance of it destroying everything else before dying itself. I found the story of Horizon Zero dawn highly plausible. The main takeaway - dumb military AIs destroy all the living things. The only bit we are missing is fully automated fabrication.
  - skissane 6y ago
    > The only bit we are missing is fully automated fabrication. Supply chains for high tech goods such as AI-powered autonomous fighter jets are extremely complex. We've already automated some of the individual steps, and will surely in the coming decades automate many more. But high-level orchestration of the chains as a whole, that requires many human-level intelligences to cooperate in advanced social systems, I don't think anything short of general AI could replace that.
  - dogma1138 6y ago
    HZD is a grey goo scenario with extra steps. Not only what we are missing is self replication but everything else including the important part and what was the problem “converting bio-mass into energy” without that it wouldn’t be an issue, not to mention that HZD makes little sense in general even with the coverup nukes exist and they blow up shit just fine their lore made little sense in the long run as they couldn’t replicate they just had really big robots the octopus ones that could manufacture more of the smaller ones. And more importantly since the biomass conversion was basically nano machines why couldn’t you produce a swarm of them that simply eats robots? Or why couldn’t you produce a swarm of robots to fight the rogue swarm. The whole “unhackable” thing also doesn’t makes sense and if they were then well make another swarm that just is unhackable and let them duke it out. The technology and more importantly the consistency of the technology in that universe makes absolutely no sense. As much as I liked the game the story while cool was about as plausible as an alien invasion that triggers a zombie apocalypse followed by the second coming.
- knolax 6y ago
  > it’s not any different in than alphago or alphastar or the OpenAI dota bots Except you know, it kills people.
- swilliamsio 6y ago
  > it doesn’t understand anything outside of the context of the game it’s playing. Yea, and the game is life and death. A military minded AI won't distinguish between surrendering and fighting enemies, won't distinguish between enemies and non-combatents, might confuse a shutdown order with an electronic attack. Just cause AI isn't by definition "smart" doesn't mean its harmless.
  - smabie 6y ago
    Why wouldn't it?
  - dogma1138 6y ago
    A heat seeking missile can’t do any of those things either, I’m not sure you understand the role of “AI” in this case, you also anthropomorphize it too much.
netflixandkill 6y ago
The only thing that will address that is a broad international agreement with sanctions and political consequences for aggressor AIs that violate it, just like agreements that mostly keep horrifying biological weapons off the table and mostly kept nuclear weapons programs in check. "mostly" still leaves a lot of room for bad actors, but these sorts of compromises have worked surprisingly well enough that the failures were notable. Political problems are only going to have political solutions, and almost by definition those are imperfect.
- bsanr2 6y ago
  The problem isn't so much an AI involved in military planning so much as an AI raised by the military, similar to how we are more okay with children who grow up to join the military rather than child soldiers (or children raised to be in the military). Defense (defense!) should be a tool taught and employed only after one has developed an understanding and appreciation of what they should be defending. Frankly, we shouldn't be developing AI outside of institutions with STRONG ties to humanist missions (possibly including religion). AI is not for war, and it's not for "efficient" resource extraction and development; it's for helping humanity to safeguard itself and its potential.
- landryraccoon 6y ago
  > mostly kept nuclear weapons programs in check. It has completely failed to keep nuclear weapons in check, imo. Keeping AI technology out of existence has the same problem as keeping nuclear weapons out of existence - the incentives are too great for any nation to have them. If a nation is armed with nuclear weapons, the only possible check on them is another nation that is armed with nuclear weapons. Thus even in the existing non-proliferation framework, the largest nations (the United States, Russia, China, several European powers) ALL still have nuclear weapons. The reason why is that if any nation was the only nuclear armed nation it would be impossible to defeat with any quantity of conventional force. The same argument applies for AI. It would be impossible avoid the existence of militarized AI if it provides the same level of force multiplication as nuclear weapons. It would be necessary for any nation to try to gain the same technology as a deterrent against attack by a rival that possesses the technology.
renewiltord 6y ago
True, but everything will go wrong with letting your enemy get there first.
ed25519FUUU 6y ago
Phew. Thankfully since we decided not to invest in defensive AI our power rivals also decided not to invest heavily in offensive AI. I was afraid the world wasn’t a fair and moral place for a minute there!
hinkley 6y ago
Home security companies working in AI also tends not to end well in the movies, video games. It amuses me to think that Aperture Science had a protocol for rogue AIs with the codename "Birthday Cake". GLaDOS did not understand why all of the humans kept talking about cake as they died. They really seem to like cake...

it's as inevitable as nuclear proliferation following WW2. you can slow it down, try and create a few key players and build around that, but it's always coming.

fermienrico 6y ago

This is a serious dilemma because: - If you don't, then the enemy will - If you do, then you're inviting the enemy to do the same So, without trust and mutual agreement on an international level, this is a self-propelling premise that cannot be stopped. Game theory 101.

landryraccoon 6y ago
Even in that international agreement, someone still has the weapons. International agreements failed to keep nuclear weapons out of the hands of the strongest nations because why would they ever give up their nukes?

dogma1138 6y ago

It’s not minded, it’s not any different in than alphago or alphastar or the OpenAI dota bots, it doesn’t understand anything outside of the context of the game it’s playing. It doesn’t have a mind. On the “intelligence” spectrum it’s much closer to the guidance system in a heat seeking missile to than say even a dog, yet alone anything that we would define as having cognition.

irrational 6y ago
Well, OP was talking about billions of dollars of DARPA funding going towards military AI research. I presume they would be trying to get much closer to cognition.
- dogma1138 6y ago
  Why would you want cognition? Besides the billions of edge cases it may create why would you want to make a tool you can or would have to reason with?
  - inetknght 6y ago
    > why would you want to make a tool you can or would have to reason with? Some might ask the same question in the context of human children.
- philwelch 6y ago
  I would question that assumption. Autonomous military equipment (vehicles, planes, etc.) doesn’t require cognition in order to be useful.
- CarbyAu 6y ago
  I obviously can't speak for Defence but my understanding was that a lot of the training et al was so that high command could have predictable outcomes. They may give troops on the front line some leeway, but within parameters for pursuit of a goal. I don't think Defence wants "uncontrolled" anything. Even if they manage to develop fully cognizant AI they will want to know the exact parameters it can work within. Of course, humans make errors...
Lammy 6y ago
> It’s not minded, it’s not any different in than alphago or alphastar or the OpenAI dota bots Aside from, like, the guns 'n' stuff.
alquemist 6y ago
Where the game is 'total world domination'. Through any means necessary, in particular by utterly destroying one's physical adversaries. Arguably the human brain development is a byproduct of an arm's race between social BIs, fighting off the next tribe since time immemorial. Time to up the ante and build smarter and stronger entities. Somebody's got to win the game, this nuclear stalemate we're living through is too boring. Nothing to worry about, move along.
- philwelch 6y ago
  I think there’s a huge difference between a fighter jet that can fly itself and Skynet.
  - alquemist 6y ago
    Short term, runaway hostile killer drone swarm. Medium term, it's an arms race to build the smartest, strongest, most ruthless global killer drone swarm. The outcome of that race is anybody's guess.
    - PhantomGremlin 6y ago
      Bah. I think recent events have clearly demonstrated that biological weapons are much much cheaper. create a killer virus inoculate your people release it at the rest of the world profit?
      - ceejayoz 6y ago
        Sounds like a good way to get nuked by the dying rest of the world, not profit.
extropy 6y ago
Yes and that's a big part of the problem. It does not have a concept of "after" the victory. So if it gains autonomy there is high chance of it destroying everything else before dying itself. I found the story of Horizon Zero dawn highly plausible. The main takeaway - dumb military AIs destroy all the living things. The only bit we are missing is fully automated fabrication.
- skissane 6y ago
  > The only bit we are missing is fully automated fabrication. Supply chains for high tech goods such as AI-powered autonomous fighter jets are extremely complex. We've already automated some of the individual steps, and will surely in the coming decades automate many more. But high-level orchestration of the chains as a whole, that requires many human-level intelligences to cooperate in advanced social systems, I don't think anything short of general AI could replace that.
- dogma1138 6y ago
  HZD is a grey goo scenario with extra steps. Not only what we are missing is self replication but everything else including the important part and what was the problem “converting bio-mass into energy” without that it wouldn’t be an issue, not to mention that HZD makes little sense in general even with the coverup nukes exist and they blow up shit just fine their lore made little sense in the long run as they couldn’t replicate they just had really big robots the octopus ones that could manufacture more of the smaller ones. And more importantly since the biomass conversion was basically nano machines why couldn’t you produce a swarm of them that simply eats robots? Or why couldn’t you produce a swarm of robots to fight the rogue swarm. The whole “unhackable” thing also doesn’t makes sense and if they were then well make another swarm that just is unhackable and let them duke it out. The technology and more importantly the consistency of the technology in that universe makes absolutely no sense. As much as I liked the game the story while cool was about as plausible as an alien invasion that triggers a zombie apocalypse followed by the second coming.
knolax 6y ago
> it’s not any different in than alphago or alphastar or the OpenAI dota bots Except you know, it kills people.
swilliamsio 6y ago
> it doesn’t understand anything outside of the context of the game it’s playing. Yea, and the game is life and death. A military minded AI won't distinguish between surrendering and fighting enemies, won't distinguish between enemies and non-combatents, might confuse a shutdown order with an electronic attack. Just cause AI isn't by definition "smart" doesn't mean its harmless.
- smabie 6y ago
  Why wouldn't it?
- dogma1138 6y ago
  A heat seeking missile can’t do any of those things either, I’m not sure you understand the role of “AI” in this case, you also anthropomorphize it too much.

netflixandkill 6y ago

The only thing that will address that is a broad international agreement with sanctions and political consequences for aggressor AIs that violate it, just like agreements that mostly keep horrifying biological weapons off the table and mostly kept nuclear weapons programs in check. "mostly" still leaves a lot of room for bad actors, but these sorts of compromises have worked surprisingly well enough that the failures were notable. Political problems are only going to have political solutions, and almost by definition those are imperfect.

bsanr2 6y ago
The problem isn't so much an AI involved in military planning so much as an AI raised by the military, similar to how we are more okay with children who grow up to join the military rather than child soldiers (or children raised to be in the military). Defense (defense!) should be a tool taught and employed only after one has developed an understanding and appreciation of what they should be defending. Frankly, we shouldn't be developing AI outside of institutions with STRONG ties to humanist missions (possibly including religion). AI is not for war, and it's not for "efficient" resource extraction and development; it's for helping humanity to safeguard itself and its potential.
landryraccoon 6y ago
> mostly kept nuclear weapons programs in check. It has completely failed to keep nuclear weapons in check, imo. Keeping AI technology out of existence has the same problem as keeping nuclear weapons out of existence - the incentives are too great for any nation to have them. If a nation is armed with nuclear weapons, the only possible check on them is another nation that is armed with nuclear weapons. Thus even in the existing non-proliferation framework, the largest nations (the United States, Russia, China, several European powers) ALL still have nuclear weapons. The reason why is that if any nation was the only nuclear armed nation it would be impossible to defeat with any quantity of conventional force. The same argument applies for AI. It would be impossible avoid the existence of militarized AI if it provides the same level of force multiplication as nuclear weapons. It would be necessary for any nation to try to gain the same technology as a deterrent against attack by a rival that possesses the technology.

renewiltord 6y ago

True, but everything will go wrong with letting your enemy get there first.

ed25519FUUU 6y ago

Phew. Thankfully since we decided not to invest in defensive AI our power rivals also decided not to invest heavily in offensive AI. I was afraid the world wasn’t a fair and moral place for a minute there!

hinkley 6y ago

Home security companies working in AI also tends not to end well in the movies, video games. It amuses me to think that Aperture Science had a protocol for rogue AIs with the codename "Birthday Cake". GLaDOS did not understand why all of the humans kept talking about cake as they died. They really seem to like cake...