3 ms·
”Machines never make the decision, they’re merely collecting data to assist human operatives” They’ll keep saying this forever, and there’s a shroud of plausib
by klabb3 3y ago
”Machines never make the decision, they’re merely collecting data to assist human operatives”
They’ll keep saying this forever, and there’s a shroud of plausible deniability to it! Not to mention it can soothe the conscience of the operatives who have to live with their decisions - they don’t even have to look through the family photos - the AI has already filtered out the suspicious parts. In reality, it’s a good ole recommendation engine, just like Netflix or TikTok. And when you’re tired, have too much work stacked up, you’ll be more likely to let autoplay take over – or approve the strike of someone’s home.
To those that think this is harsh: please read the article. It’s that bad.
- wahnfrieden 3y agoIt's not a matter of being tired and having to catch up. The operators are explicitly instructed to treat the AI results as orders without questioning the results. In other words, operators are threatened with punishment if they take the time to inspect the results more closely before following orders. It's not even an option! > In order to speed up target elimination, *soldiers have been ordered to treat Lavender-generated targets as an order, rather than something to be independently checked*, the investigation found. Soldiers in charge of manually checking targets spent only seconds on each person, sources said, merely to make sure the target was a man. Children are also considered legitimate targets by Lavender.
- klabb3 3y agoRight! The extreme point of laziness/stress is just pressing approve. At which point the machine is making the de-facto decision. In either case, the role of the recommendation engine is immensely impactful, as we already know from consumer products. But here the software engineers are directly involved in life-and-death decisions at scale. I really hope they know that.
- wahnfrieden 3y agoThey know that because it was designed to increase civilian casualties by unprecedented civilian:"combatant" ratios and confidence levels. These were design requirements, not faults. The notable element of this news is not that the decision-making was automated. It's that they set the civilian collateral death ratio to ~100 (and other details such as considering children to be valid combatant targets), regardless of whether the process that arrives at that was automated or not.
- NomDePlum 3y agoThe Guardian article on this had quotes from the "operators" involved. They don't question or examine any of the output just hit approve. The only apparent operator act that seems to be something they are involved in is turning the dial up or down to get more or less "hits". Depressingly this appears to be related to superiors shouting at them as they don't have enough targets. It's absolutely sickening.
- mikrotikker 3y agoWhen you're a tiny, easily overtaken (geographically) country with genocidal neighbours and your race and religion is a global minority, of course you're going to be bullish on taking out threats to it, and your people, and less caring about collateral damage. Your nation is under threat from almost all sides
- dragonwriter 3y agoBeing less concerned about collateral damage and other similar concerns in that condition will perpetually increase that threat.
- mikrotikker 3y agoNot when you have fire and technical superiority.