3 ms·

That kind of AI does not exist and might not for a long time or indeed ever (i.e., it can fully manipulate the physical world at scale and unopposed).

dragonwriter 4y ago
Automated, remote commanded weapons exist already and development is happening quite rapidly on more. Automated where the whole realm of physical manipulation is not automated under AI control, but enforcement largely is through combat drones coordinated by AI is not far from being achievable. And it doesn’t need to even be totally that to be a problem, each incremental bit of progress that reduces the number of people required to apply power over any given size subject population makes tyranny more sustainable.
est31 4y ago
The AI doesn't have to be 100% self reliant. It might just do the dirty work while the humans in the background support it in an abstracted form, making it easier for them to lie to themselves about what they are doing. When your job is to repair and maintain infantry drones, how do you know if it's civilian or combatant blood that you are cleaning from the shell? You never leave the camp because doing so is too dangerous and you also don't know the language anyways. Sure the enemy spreads propaganda that the drones are killing babies but those are most likely just lies... Or when you are a rocket engineer, there is little difference between your globe spanning satellite network transmitting the position of a hypersonic missile headed for your homeland, or remote controlling a network of drones. AI gives you a more specialized economy. Instead of every soldier needing to know how to maintain their gun you now have one person responsible for physical maintenance, one person responsible for maintaining the satellite kubernetes network, one UI designer, and one lunatic typing in war crime commands. Everyone in the chain is oblivious of the war crime except for that one person at the end.
rbanffy 4y ago
It doesn't need to manipulate the world. It only needs to manipulate people. Look at what disinformation and astroturf campaigns accomplished without AI in the past ten years. We are in a very uncomfortable position WRT not autonomous AIs but AIs following human-directed agendas.
pdntspa 4y ago
ChatGPT plugins can go out and access the internet. The internet is hooked up to things that physically move things -- think stuff like industrial SCADA or bridge controls. You can remotely pan a gun around with the right apparatus, and you can remotely pull a trigger with a sister mechanism. Right now we have drone pilots sitting in trailers running machines halfway across the world. Who is to say that some of these human-controlled things havent already been replaced by AI? I think AI manipulating the physical world is a lot closer than you think.
- notahacker 4y ago
  There's a pretty massive gap between "look, LLMs can be trained to use an API" and "As soon as somebody types the word 'kill every firstborn' nobody can stop them". It isn't obvious that drones are more effective killing people without human pilots (and to the extent they are, that's mostly proneness to accidentally kill people with friendly fire or other errors) and the sort of person that possesses unchecked access to fleets of military drones is going to be pretty powerful when it comes to ordering people to kill other people anyway. And the sort of leader that wants to replace soldiers with drones only his inner circle can exercise control over because he suspects that the drones will be more loyal is the sort of leader that's disproportionately likely to be terminated in a military coup, because military equipment is pretty deadly in human hands too...
  - pdntspa 4y ago
    My point is that the plumbing for this sort of thing is coming into focus. Human factors are not an effective safeguard. I feel like every issue you have brought up is solvable. Today, already, you can set up a red zone watched by computer vision hooked up to autonomous turrets with orders to fire at anything it recognizes as human. Some guy already did it for mosquitos.
    - notahacker 4y ago
      > Today, already, you can set up a red zone watched by computer vision hooked up to autonomous turrets with orders to fire at anything it recognizes as human. Sure. You could also achieve broadly similar effects with an autonomous turret with a simple, unintelligent program that fires at anything that moves, or with the 19th century technology of tripwires attached to explosives. The NN wastes less ammo, but it is't a step change in firepower, least of all for someone trying to monopolise power over an entire country. Dictators have seldom had trouble getting the military on side, and if they can't, then the military has access to at least as much AI and non-AI tech to outgun them. Nobody doubts computers can be used to kill people (lots of things can be used to kill people), it's the idea that computers are some sort of omipotent genie that grants wishes for everyone's firstborn to die being pushed back on here.
deltree7 4y ago
Yeah, not to mention that good guys far number bad guys and we can develop a good AGI to pre-detect bad ones and destroy. Even with AGIs, there is still power in numbers. We still understand how all the mechanical things work. The first attempts of AGIs will be botched anyway. It's not the AGI will secretly plan for years hiding itself and suddenly launch one final attack.

dragonwriter 4y ago

Automated, remote commanded weapons exist already and development is happening quite rapidly on more. Automated where the whole realm of physical manipulation is not automated under AI control, but enforcement largely is through combat drones coordinated by AI is not far from being achievable. And it doesn’t need to even be totally that to be a problem, each incremental bit of progress that reduces the number of people required to apply power over any given size subject population makes tyranny more sustainable.

est31 4y ago

The AI doesn't have to be 100% self reliant. It might just do the dirty work while the humans in the background support it in an abstracted form, making it easier for them to lie to themselves about what they are doing. When your job is to repair and maintain infantry drones, how do you know if it's civilian or combatant blood that you are cleaning from the shell? You never leave the camp because doing so is too dangerous and you also don't know the language anyways. Sure the enemy spreads propaganda that the drones are killing babies but those are most likely just lies... Or when you are a rocket engineer, there is little difference between your globe spanning satellite network transmitting the position of a hypersonic missile headed for your homeland, or remote controlling a network of drones. AI gives you a more specialized economy. Instead of every soldier needing to know how to maintain their gun you now have one person responsible for physical maintenance, one person responsible for maintaining the satellite kubernetes network, one UI designer, and one lunatic typing in war crime commands. Everyone in the chain is oblivious of the war crime except for that one person at the end.

rbanffy 4y ago

It doesn't need to manipulate the world. It only needs to manipulate people. Look at what disinformation and astroturf campaigns accomplished without AI in the past ten years. We are in a very uncomfortable position WRT not autonomous AIs but AIs following human-directed agendas.

pdntspa 4y ago

ChatGPT plugins can go out and access the internet. The internet is hooked up to things that physically move things -- think stuff like industrial SCADA or bridge controls. You can remotely pan a gun around with the right apparatus, and you can remotely pull a trigger with a sister mechanism. Right now we have drone pilots sitting in trailers running machines halfway across the world. Who is to say that some of these human-controlled things havent already been replaced by AI? I think AI manipulating the physical world is a lot closer than you think.

notahacker 4y ago
There's a pretty massive gap between "look, LLMs can be trained to use an API" and "As soon as somebody types the word 'kill every firstborn' nobody can stop them". It isn't obvious that drones are more effective killing people without human pilots (and to the extent they are, that's mostly proneness to accidentally kill people with friendly fire or other errors) and the sort of person that possesses unchecked access to fleets of military drones is going to be pretty powerful when it comes to ordering people to kill other people anyway. And the sort of leader that wants to replace soldiers with drones only his inner circle can exercise control over because he suspects that the drones will be more loyal is the sort of leader that's disproportionately likely to be terminated in a military coup, because military equipment is pretty deadly in human hands too...
- pdntspa 4y ago
  My point is that the plumbing for this sort of thing is coming into focus. Human factors are not an effective safeguard. I feel like every issue you have brought up is solvable. Today, already, you can set up a red zone watched by computer vision hooked up to autonomous turrets with orders to fire at anything it recognizes as human. Some guy already did it for mosquitos.
  - notahacker 4y ago
    > Today, already, you can set up a red zone watched by computer vision hooked up to autonomous turrets with orders to fire at anything it recognizes as human. Sure. You could also achieve broadly similar effects with an autonomous turret with a simple, unintelligent program that fires at anything that moves, or with the 19th century technology of tripwires attached to explosives. The NN wastes less ammo, but it is't a step change in firepower, least of all for someone trying to monopolise power over an entire country. Dictators have seldom had trouble getting the military on side, and if they can't, then the military has access to at least as much AI and non-AI tech to outgun them. Nobody doubts computers can be used to kill people (lots of things can be used to kill people), it's the idea that computers are some sort of omipotent genie that grants wishes for everyone's firstborn to die being pushed back on here.

deltree7 4y ago

Yeah, not to mention that good guys far number bad guys and we can develop a good AGI to pre-detect bad ones and destroy. Even with AGIs, there is still power in numbers. We still understand how all the mechanical things work. The first attempts of AGIs will be botched anyway. It's not the AGI will secretly plan for years hiding itself and suddenly launch one final attack.