3 ms·

AI-directed bombings of persons are already known to have occurred on a large scale, in Gaza. https://www.972mag.com/lavender-ai-israeli-army-gaza/ https://www

AI-directed bombings of persons are already known to have occurred on a large scale, in Gaza. https://www.972mag.com/lavender-ai-israeli-army-gaza/ https://www.972mag.com/lavender-ai-israeli-army-gaza/ One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target is male. This was despite knowing that the system makes what are regarded as “errors” in approximately 10 percent of cases, and is known to occasionally mark individuals who have merely a loose connection to militant groups, or no connection at all.

elzbardico 1y ago
But probably not LLM related, Israel intelligence have been building ML models for the military for a long time and probably they leverage some logic/symbolic AI in the mix.
- animal531 1y ago
  I'd be fairly certain at this point that someone has used LLM created code in something that has subsequently caused a death.
aprilthird2021 1y ago
[flagged]
- codr7 1y ago
  [flagged]
beloch 1y ago
This is a rare combination of awful factors, but most of them are all too human. Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because of the potential rewards. Note that civilians dying is usually part of the plan in such operations. The "Where's daddy?" tool is designed specifically to identify when targets are in their homes with their families so that they can be bombed along with their families. Lavender has allowed the IDF to perform the same kind of identification/location for anyone even peripherally involved with Hamas. The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas), and the real ratio is probably higher. If the AI fingers somebody, they're Hamas unless extraordinary evidence proves otherwise. Instead of scrutinizing what comes out of Lavendar, the IDF has shut journalists out of Gaza to reduce the odds of extraordinary evidence ever coming to public light. The real problem here is not that Lavendar makes mistakes. Humans doing the same job would too. The problem is that the IDF has so completely dehumanized Palestinians that, without questioning their digital oracle, they are willing to drop bombs on large numbers of civilians if there's a chance, according to Lavender, one of them might be involved with Hamas in some way. That's not an AI disaster. That's a human disaster.
- ethbr1 1y ago
  The IDF is using AI to whitewash ethics in the same way genai is using it to whitewash copyright. Historically, things have gone badly for society when ills were mostly constrained by potential culpability and prosecution of humans... and then that restriction was suddenly removed by technological advancement. And the cybersecurity industry / CISOs (aka fall people, giving someone responsibility without authority) are a poor pattern to use to effectively limit tail risk. At minimum, something more like financial auditing / engineer sign-off should be put in place around productionalized AI, preferably via a third-party.
  - originalvichy 1y ago
    This is the chilling part. When Mossad hunted down Eichmann, some of his defense could be summarized as the cliche of ”I was just following orders”. Now with this tooling, they are learning from the past, and centralizing the ”orders” into a non-human machine, which means only a few people (if ever) would stand trial if they were accused of crimes against humanity: minister of defense, some procurement guy, some technical lead. That’s it. Will our laws of war account for soldiers getting orders from algorithms? Will people get away with killing innocent civilians if a model paints them as a target?
    - sitkack 1y ago
      We were able to largely ban chemical weapons and so far have prevented the use of nuclear weapons a second time. Maybe we can do this with AI? Someday there will be a large shrine in Gaza to all the lives that made AI in war a morally reprehensible thing.
      - ethbr1 1y ago
        The reason we were able to ban chemical weapons and nuclear weapons, to the extent we did, is that it was in everyone's self-interest. In contrast, there's a reason major militaries haven't ratified the land mine treaty: they work. Unfortunately, AI driven weapons are going to effective in terms of decreased response time and increased automomy. Ergo, the best we can aim for should be enforced guidelines around its use. Especially with regards to human authority/responsibility.
directevolve 1y ago
The article’s not titled precisely - it’s specifically about LLM disasters, not AI disasters in general.
danaris 1y ago
While that is legitimately awful, I personally wouldn't characterize it as an "AI disaster". Israel is 100% ready to bomb people in Gaza based on all kinds of different bad reasons; the fact that, in these specific cases, the bad reason was AI is almost beside the point there. It kinda disappears into the noise of thousands and thousands of attacks either directly or indirectly injuring and killing civilians in Gaza.

elzbardico 1y ago

But probably not LLM related, Israel intelligence have been building ML models for the military for a long time and probably they leverage some logic/symbolic AI in the mix.

animal531 1y ago
I'd be fairly certain at this point that someone has used LLM created code in something that has subsequently caused a death.

aprilthird2021 1y ago

[flagged]

codr7 1y ago
[flagged]

beloch 1y ago

This is a rare combination of awful factors, but most of them are all too human. Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because of the potential rewards. Note that civilians dying is usually part of the plan in such operations. The "Where's daddy?" tool is designed specifically to identify when targets are in their homes with their families so that they can be bombed along with their families. Lavender has allowed the IDF to perform the same kind of identification/location for anyone even peripherally involved with Hamas. The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas), and the real ratio is probably higher. If the AI fingers somebody, they're Hamas unless extraordinary evidence proves otherwise. Instead of scrutinizing what comes out of Lavendar, the IDF has shut journalists out of Gaza to reduce the odds of extraordinary evidence ever coming to public light. The real problem here is not that Lavendar makes mistakes. Humans doing the same job would too. The problem is that the IDF has so completely dehumanized Palestinians that, without questioning their digital oracle, they are willing to drop bombs on large numbers of civilians if there's a chance, according to Lavender, one of them might be involved with Hamas in some way. That's not an AI disaster. That's a human disaster.

ethbr1 1y ago
The IDF is using AI to whitewash ethics in the same way genai is using it to whitewash copyright. Historically, things have gone badly for society when ills were mostly constrained by potential culpability and prosecution of humans... and then that restriction was suddenly removed by technological advancement. And the cybersecurity industry / CISOs (aka fall people, giving someone responsibility without authority) are a poor pattern to use to effectively limit tail risk. At minimum, something more like financial auditing / engineer sign-off should be put in place around productionalized AI, preferably via a third-party.
- originalvichy 1y ago
  This is the chilling part. When Mossad hunted down Eichmann, some of his defense could be summarized as the cliche of ”I was just following orders”. Now with this tooling, they are learning from the past, and centralizing the ”orders” into a non-human machine, which means only a few people (if ever) would stand trial if they were accused of crimes against humanity: minister of defense, some procurement guy, some technical lead. That’s it. Will our laws of war account for soldiers getting orders from algorithms? Will people get away with killing innocent civilians if a model paints them as a target?
  - sitkack 1y ago
    We were able to largely ban chemical weapons and so far have prevented the use of nuclear weapons a second time. Maybe we can do this with AI? Someday there will be a large shrine in Gaza to all the lives that made AI in war a morally reprehensible thing.
    - ethbr1 1y ago
      The reason we were able to ban chemical weapons and nuclear weapons, to the extent we did, is that it was in everyone's self-interest. In contrast, there's a reason major militaries haven't ratified the land mine treaty: they work. Unfortunately, AI driven weapons are going to effective in terms of decreased response time and increased automomy. Ergo, the best we can aim for should be enforced guidelines around its use. Especially with regards to human authority/responsibility.

directevolve 1y ago

The article’s not titled precisely - it’s specifically about LLM disasters, not AI disasters in general.

danaris 1y ago

While that is legitimately awful, I personally wouldn't characterize it as an "AI disaster". Israel is 100% ready to bomb people in Gaza based on all kinds of different bad reasons; the fact that, in these specific cases, the bad reason was AI is almost beside the point there. It kinda disappears into the noise of thousands and thousands of attacks either directly or indirectly injuring and killing civilians in Gaza.