33 ms·

My biggest concern for military use of AI is how incompetent most military contractors are. These huge companies employ an army of not-very-good engineers who

My biggest concern for military use of AI is how incompetent most military contractors are. These huge companies employ an army of not-very-good engineers whose primary purpose seems to be to over-complicate projects. Imagine the same teams that make planes which need to be hard rebooted every few days, now they're making advanced AI to dynamically target and kill people.

dmix 3y ago
> now they're making advanced AI to dynamically target and kill people. I'm curious, which projects are working on this? Any ones that we know about? In terms of production weapons I know the newer Switchblades, Excalibur, and even Javelin do some fancy automated targeting before the terminal phase but that's not really AI beyond image recognition and the targets/specific destination are pre-selected. I'm curious what sort of applications would use autonomous targeting without overt human involvement.
- ethbr0 3y ago
  Air defense has had automatic targeting since the 1980s(?) due to the required reaction times. As for autonomous target/kill programs, that'd be the USAF's Collaborative Combat Aircraft (CCA) program (under NGAD). It sounds like they're keeping human-in-the-loop for now by targeting a 1:many manned_stealth:drone ratio, but that will be subject to erosion as autonomous capabilities improve. And the very nature of the mission requires autonomy -- drones penetrating deep into sophisticated air defense systems using stealth and emissions control (so only line-of-sight communication links). Nobody is going to be remotely piloting these things.
- ikekkdcjkfke 3y ago
  You can bet they have automatic target tracking and ballistic aiming onto those in the new abrams tank, but sending the kill command will most likely be done by operators. But it's just a small step from that to putting it on autopilot
  - dmix 3y ago
    > But it's just a small step from that to putting it on autopilot Is it? How?
Dig1t 3y ago
This reminds me of the turret robots from Futurama https://youtu.be/MsY0PVqTleY https://youtu.be/MsY0PVqTleY https://youtu.be/WzbT0Q2jh_w https://youtu.be/WzbT0Q2jh_w
- wombatpm 3y ago
  Well let's just build a robot Santa too. What could possibly go wrong
hsjqllzlfkf 3y ago
It's a bit bizarre that you bring up incompetent engineers but then as an example you mention planes, which are incredibly safe, just because you have to reboot them. It's as if your entire understanding of engineering is writing webapps, and for whom developer-niceties are the main goal, and the value to the user is secondary. No, planes are a TERRIBLE example of incompetent engineers and your arrogance is breathtaking.
- godelski 3y ago
  I think there's a software engineering bias. Unless you work on critical systems you can generally "move fast and break things." But in the rest of the engineering world this kills people and is often illegal (e.g. aircraft). You're legally liable. We can even notice this with the difference in software on more critical systems vs non-critical. Compute systems on cars are not the same compute systems in your computer, and are way under powered in comparison. These systems don't fail often, but are slow and most complaints are about how they lack features. On the other hand, my laundry room app has washers in a semi-randomized order (room order 4,6,7,3,11,9,...) and the programmer clearly doesn't know about the sort function. You don't see planes falling out of the sky because of computer issues despite the whole system being fly by wire and that planes mostly fly themselves. Same goes for cars. But you do see washing machines and Twitter fail all the time. These things have different objectives and when you compare you have to consider the different goals.
DrBenCarson 3y ago
There are probably worse outcomes than having to turn a highly reliable and safe Death Machine off for an hour every few days.

dmix 3y ago

> now they're making advanced AI to dynamically target and kill people. I'm curious, which projects are working on this? Any ones that we know about? In terms of production weapons I know the newer Switchblades, Excalibur, and even Javelin do some fancy automated targeting before the terminal phase but that's not really AI beyond image recognition and the targets/specific destination are pre-selected. I'm curious what sort of applications would use autonomous targeting without overt human involvement.

ethbr0 3y ago
Air defense has had automatic targeting since the 1980s(?) due to the required reaction times. As for autonomous target/kill programs, that'd be the USAF's Collaborative Combat Aircraft (CCA) program (under NGAD). It sounds like they're keeping human-in-the-loop for now by targeting a 1:many manned_stealth:drone ratio, but that will be subject to erosion as autonomous capabilities improve. And the very nature of the mission requires autonomy -- drones penetrating deep into sophisticated air defense systems using stealth and emissions control (so only line-of-sight communication links). Nobody is going to be remotely piloting these things.
ikekkdcjkfke 3y ago
You can bet they have automatic target tracking and ballistic aiming onto those in the new abrams tank, but sending the kill command will most likely be done by operators. But it's just a small step from that to putting it on autopilot
- dmix 3y ago
  > But it's just a small step from that to putting it on autopilot Is it? How?

Dig1t 3y ago

This reminds me of the turret robots from Futurama https://youtu.be/MsY0PVqTleY https://youtu.be/MsY0PVqTleY https://youtu.be/WzbT0Q2jh_w https://youtu.be/WzbT0Q2jh_w

wombatpm 3y ago
Well let's just build a robot Santa too. What could possibly go wrong

hsjqllzlfkf 3y ago

It's a bit bizarre that you bring up incompetent engineers but then as an example you mention planes, which are incredibly safe, just because you have to reboot them. It's as if your entire understanding of engineering is writing webapps, and for whom developer-niceties are the main goal, and the value to the user is secondary. No, planes are a TERRIBLE example of incompetent engineers and your arrogance is breathtaking.

godelski 3y ago
I think there's a software engineering bias. Unless you work on critical systems you can generally "move fast and break things." But in the rest of the engineering world this kills people and is often illegal (e.g. aircraft). You're legally liable. We can even notice this with the difference in software on more critical systems vs non-critical. Compute systems on cars are not the same compute systems in your computer, and are way under powered in comparison. These systems don't fail often, but are slow and most complaints are about how they lack features. On the other hand, my laundry room app has washers in a semi-randomized order (room order 4,6,7,3,11,9,...) and the programmer clearly doesn't know about the sort function. You don't see planes falling out of the sky because of computer issues despite the whole system being fly by wire and that planes mostly fly themselves. Same goes for cars. But you do see washing machines and Twitter fail all the time. These things have different objectives and when you compare you have to consider the different goals.

DrBenCarson 3y ago

There are probably worse outcomes than having to turn a highly reliable and safe Death Machine off for an hour every few days.