3 ms·
This is a rare combination of awful factors, but most of them are all too human. Humans can sift through sigint (Cell phone calls, text messages, network acces
by beloch 1y ago
This is a rare combination of awful factors, but most of them are all too human.
Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because of the potential rewards. Note that civilians dying is usually part of the plan in such operations. The "Where's daddy?" tool is designed specifically to identify when targets are in their homes with their families so that they can be bombed along with their families.
Lavender has allowed the IDF to perform the same kind of identification/location for anyone even peripherally involved with Hamas. The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas), and the real ratio is probably higher. If the AI fingers somebody, they're Hamas unless extraordinary evidence proves otherwise. Instead of scrutinizing what comes out of Lavendar, the IDF has shut journalists out of Gaza to reduce the odds of extraordinary evidence ever coming to public light.
The real problem here is not that Lavendar makes mistakes. Humans doing the same job would too. The problem is that the IDF has so completely dehumanized Palestinians that, without questioning their digital oracle, they are willing to drop bombs on large numbers of civilians if there's a chance, according to Lavender, one of them might be involved with Hamas in some way. That's not an AI disaster. That's a human disaster.
- ethbr1 1y agoThe IDF is using AI to whitewash ethics in the same way genai is using it to whitewash copyright. Historically, things have gone badly for society when ills were mostly constrained by potential culpability and prosecution of humans... and then that restriction was suddenly removed by technological advancement. And the cybersecurity industry / CISOs (aka fall people, giving someone responsibility without authority) are a poor pattern to use to effectively limit tail risk. At minimum, something more like financial auditing / engineer sign-off should be put in place around productionalized AI, preferably via a third-party.
- originalvichy 1y agoThis is the chilling part. When Mossad hunted down Eichmann, some of his defense could be summarized as the cliche of ”I was just following orders”. Now with this tooling, they are learning from the past, and centralizing the ”orders” into a non-human machine, which means only a few people (if ever) would stand trial if they were accused of crimes against humanity: minister of defense, some procurement guy, some technical lead. That’s it. Will our laws of war account for soldiers getting orders from algorithms? Will people get away with killing innocent civilians if a model paints them as a target?
- sitkack 1y agoWe were able to largely ban chemical weapons and so far have prevented the use of nuclear weapons a second time. Maybe we can do this with AI? Someday there will be a large shrine in Gaza to all the lives that made AI in war a morally reprehensible thing.
- ethbr1 1y agoThe reason we were able to ban chemical weapons and nuclear weapons, to the extent we did, is that it was in everyone's self-interest. In contrast, there's a reason major militaries haven't ratified the land mine treaty: they work. Unfortunately, AI driven weapons are going to effective in terms of decreased response time and increased automomy. Ergo, the best we can aim for should be enforced guidelines around its use. Especially with regards to human authority/responsibility.
- MotiBanana 1y ago> The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas) No such admission exists, and 20:1 is a made up number that can't be backed by any credible source.
- beloch 1y agohttps://archive.md/vvjIV https://archive.md/vvjIV
- therealpygon 1y ago[flagged]
- sitkack 1y agostop
- therealpygon 1y agoStop what, exactly? Or do your personal feelings obscure your ability to recognize a statement of facts that are unrelated to politics?
- balanced2 1y agoThe article is quite confusing for starting with examples of travel accidents which will have a strong human cause but tries to make a case for a AI-only disaster. There is still a strong human factor in every AI tool, for example the best answers come when the model can search for human content. Perhaps humans searching for an AI-only disaster is just a means to shed responsibility from the humans? That seems far more dangerous than anything AI can do.