5 ms·
A bit more scary is the political consequences. One of the strongest controls on rogue generals is that a fair chunk of their troops will mutiny if told to dest
by roenxi 5y ago
A bit more scary is the political consequences. One of the strongest controls on rogue generals is that a fair chunk of their troops will mutiny if told to destroy their home country for said general's benefit, and logistical support would vanish.
Armies of killer robots start radically changing the calculations. It isn't obvious how but it probably removes a lot of the interest of more ordinary people and makes it a question of what the elites want. If the programmers are on side and the factories are mostly automatic, then the ability of people to push back on their own military becomes complicated.
- k__ 5y agoHow is this different from nuclear weapons?
- roenxi 5y agoIf a general lives in the penthouse at the top of a skyscraper, and maybe has a few thousand people who support him, nukes are not going to help deal with revolutionaries on the bottom floor. They are too indiscriminate and can't be used against close up targets. Whereas a general with a drone swarm and a few thousand supporters might be able to suppress a city. Fortunately a little far fetched, but what was impossible with nukes is only impractical with drones.
- throw1234651234 5y agoThis is well put. There is also the idea that as armies got MI-24s/AH-64s, etc, opposing forces got Stingers/Iglas. Perhaps it will play out in a way where once armies get drones, OpFor gets counters - jamming / hacking / EM / whatever.
- MisterTea 5y agoThe land to be ruled by said rogue general will not be a radioactive wasteland.
- raclage 5y agoSufficiently advanced robots would leave a lot more of value behind than a nuke which makes deploying them much less costly to the person that wants to actually control the territory.
- behindai 5y agoSeeing AI weapons as a weapon of mass destruction is a trending discourse.
- wcarss 5y agoIt's a much harder weapon to control via regulation and treaties than nuclear weapons: there is no nuclear material required to hook guns up to computers en masse in factories, let alone improvisationally. Once this cat is really out of the bag, everyone who wants it will have it.
- est31 5y agoManufacturing chips is an extremely complicated process and requires thousands of specialists from multiple different disciplines to cooperate. Building a new fab costs billions and the buildings are extremely sensitive for damage. I think one of the reasons why the west wants to block mainland China from manufacturing chips is to prevent them from building a robot army.
- wcarss 5y agoIt's true that building the chips is hard, but making it hard to buy general purpose small computers, or digital camera sensors suitable for vision, or rotor blades, or batteries, or antennas, is also going to be very difficult. Proliferation has already happened. Maybe cheap LIDAR will be something we can keep a lid on? But regular vision is still likely pretty good for lining up a shot or finding a face.
- cabalamat 5y ago> but making it hard to buy general purpose small computers [...] is also going to be very difficult Just ban general purpose computers that allow running arbitrary unsigned code. I'm sure lots of governments and corporations would be happy to get on board that one, and many end-users wouldn't notice.
- kian 5y agoA cure worse than the disease. Also wouldn't stop the military from building up a drone army and then someone within it choosing to go rogue and take it with them. It only ensures that no one else can stop them.
- crznp 5y agoThe world made it through the cold war without major disaster, but it is easy to imagine history taking a different path, so I don't think that this is a comforting comparison. Also, nuclear weapons can't be used 'a little bit' but killer robots can be -- it is much less clear where we draw the line. It seems likely that their use will be normalized/entrenched before we figure that out. Though I hope that you're right.
- sandworm101 5y ago>> that a fair chunk of their troops will mutiny if told to destroy their home country I'd be careful about this. There is a general principle in military communities that troops are not to obey "illegal orders" but time and time again throughout history we have seen that backstop fail. At a base level we all do what we think is best for ourselves. Troops will shoot prisoners. Troops will shoot protestors. They will nuke their own hometowns if they believe that is either necessary or if failure to do so will result in their own demise. And many troops, people, will do what they are told out of pure loyalty to a charismatic leader. The only real safeguard is layer upon layer of checks to ensure no one leader can ever exercise independent control.
- wcarss 5y agoIt's a fair point that humans don't magically make this go away, but the thrust of the argument above is that robots are likely worse, because they lack any capacity at all for questioning or for remorse, and they also allow great physical detachment from the horrors they can cause. They don't stop to eat or sleep either, and they'll certainly do things that can cause their own destruction. Humans also may be even likelier to be loyal than moral if a drone with a gun follows them around to keep them on-side.
- est31 5y agoHumans have a natural aversion to killing, and yes, during war mentioned atrocities do happen, but the soldiers are not perfect killing machines. Often they miss intentionally and just shoot in the general vicinity of the opponent. See this reddit thread: https://old.reddit.com/r/history/comments/b6k528/percentage_of_soldier_who_purposely_missed_or/ https://old.reddit.com/r/history/comments/b6k528/percentage_... Troops can also desert which is especially prevalent in civil war situations. That's why during war, you need propagandists coming up with constant stories about how horrible and bad the other side is, so that you get people to shoot. The unit that so brutally killed the tianmen protesters was one of the most brutal ones the Chinese military had. That's why they had to wait for days for it to arrive. Had they deployed different units, they might have fraternized with the protesters and joined them in their protest, making the task even harder for the leadership. Try getting even the most brutal unit to shoot at people in their own uniform. For military parades, you can't afford 5% of people who intentionally don't march in lock step. That's why usually the troops which participate in such parades are isolated and fed propaganda leading up to the parade. The human component is a constant variable that needs to be accounted for in military leadership.
- SerLava 5y agoIt just has to be a war crime to use these. A domestic army of robots is really just a foreign army on contract. They have zero loyalty to anyone who doesn't directly control them.
- cabalamat 5y agoThis is true. Killer robots push societies in the direction of being more hierarchial, for the reasons you state.
- abecedarius 5y agoThis generalizes to automation of nonlethal means of population control -- crowd control weapons, surveillance and censorship. On the margin it'd hold for partial automation short of AGI -- reducing the needed size of the coalition on top.
- Gravityloss 5y agoThis works better if a large enough portion of the soldiers are conscripts (possibly rotated in and out) and not career soldiers. The conscripts know they will go back to their normal life, day job and be part of the civilian society again. If it's just career soldiers, then it's in their career interest to enlarge the military's influence on society, increase budgets etc.
- JumpCrisscross 5y ago> troops will mutiny if told to destroy their home country for said general's benefit I’d generalise this check to all leaders. No man rules alone. Power only exists in its relationship with people. Robotic armies permit force projection without any people. That changes the political calculus in all sections of society, from military leadership to billionaires to members of a terrorist group.
- shadofx 5y agoOne good zero-day could give total military and industrial control of multiple superpower nations to a single talented sociopath