2 ms·

There's nothing inherently wrong with human-in-the-loop. In many, many, many cases, it's is lack of this that causes death, destruction and grievous mistakes

There's nothing inherently wrong with human-in-the-loop. In many, many, many cases, it's is lack of this that causes death, destruction and grievous mistakes while giving humans a nominal "plausible deniability" of making any mistake at all. - "it's the software that decides; I can't change that.". The biggest reason why human-in-the-loop is necessary is that humans scale better in unique complexity situations than fixed-design machines ever an be. Often you actually want to slow down a system specifically to assure a human can judge. Fully automating assures that the small or large errors will grow far too fast - we see this with social media - that's literally the problem with Twitter et al. - they operate too fast to be safe or effective. It's better to slow things down if it results in correct, moral or fair operation. There is nothing magical or essential or better about maximum speed blindly. The additional big reason is that machines are incapable of morality nor of facing punishment - you need a human to "have skin in the game" to create any force to assure correct behavior or operation. Being afraid of jail or financial ruin is essential to this. A machine or corporation (NOT a person) never can and never will. One data point is drones: currently, like most precision weapons, the precision is nearly 100% - that is, if you point the weapon at a target, the target will be destroyed nearly 100% of the time. The problem then is how you target the weapon - that's where all the "errors" are incurred. This is how you get Afghan wedding parties killed off. It's the intelligence errors and rules of engagement that are actually what kills. Another data point that should be horrifying: the US military is currently working on neuromorphic AI chips. The specific intention is to make early warning satellites capable of identifying targets and triggering a nuclear "response/attack" without a human in the loop. Literal Sky Net. This is intelligence and targeting being automated and humans being pulled out of the loop.

kragen 4y ago
You say, "There's nothing inherently wrong with human-in-the-loop." But humans have reaction times around 500 milliseconds; in the best cases, around 200 milliseconds. I routinely write software with worst-case reaction times around 0.001 milliseconds, or 1 millisecond for more complex stimuli. If a force of drones with humans in the loop fights a force of drones without humans in the loop, absent colossal blunders, the latter will win, and it won't even be close. Imagine you get into a fistfight with a guy. He's bigger than you, he's got a loaded 9-millimeter pistol, and he's a good marksman. But his reaction time is 200 times yours --- say, about 50 seconds, if you're a regular human. That is, all his reactions are reacting to what you did 50 seconds ago, at best. Who do you think is going to win? Unless he's inside a tank and you're outside, you will. Even in that case, I think you've got a fighting chance. You argue that it's good for armies to "have skin in the game" in the sense of being vulnerable to jail or financial ruin. And, from a moral point of view, it probably is good: armies that are exposed to retaliation will generally be less willing to do things that provoke retaliation, many of which are immoral. However, the things that provoke retaliation are also the things that win wars. Making your army more vulnerable may reduce war crimes, but it is not a recipe for victory. The logic of warfare is inimical to our survival as a species, which is why our best and brightest have been calling for a pacifist world order since even before World War II. Nuclear warfare were a nice early warning, but they're children's toys compared to precision-guided munitions backed up by pervasive surveillance.
- kragen 4y ago
  Correction: 500 times your reaction time would be about 40 seconds if your reaction time is 200 ms.

kragen 4y ago

You say, "There's nothing inherently wrong with human-in-the-loop." But humans have reaction times around 500 milliseconds; in the best cases, around 200 milliseconds. I routinely write software with worst-case reaction times around 0.001 milliseconds, or 1 millisecond for more complex stimuli. If a force of drones with humans in the loop fights a force of drones without humans in the loop, absent colossal blunders, the latter will win, and it won't even be close. Imagine you get into a fistfight with a guy. He's bigger than you, he's got a loaded 9-millimeter pistol, and he's a good marksman. But his reaction time is 200 times yours --- say, about 50 seconds, if you're a regular human. That is, all his reactions are reacting to what you did 50 seconds ago, at best. Who do you think is going to win? Unless he's inside a tank and you're outside, you will. Even in that case, I think you've got a fighting chance. You argue that it's good for armies to "have skin in the game" in the sense of being vulnerable to jail or financial ruin. And, from a moral point of view, it probably is good: armies that are exposed to retaliation will generally be less willing to do things that provoke retaliation, many of which are immoral. However, the things that provoke retaliation are also the things that win wars. Making your army more vulnerable may reduce war crimes, but it is not a recipe for victory. The logic of warfare is inimical to our survival as a species, which is why our best and brightest have been calling for a pacifist world order since even before World War II. Nuclear warfare were a nice early warning, but they're children's toys compared to precision-guided munitions backed up by pervasive surveillance.

kragen 4y ago
Correction: 500 times your reaction time would be about 40 seconds if your reaction time is 200 ms.