12 ms·

> B., a senior officer who used Lavender, echoed to +972 and Local Call that in the current war, officers were not required to independently review the AI syste

> B., a senior officer who used Lavender, echoed to +972 and Local Call that in the current war, officers were not required to independently review the AI system’s assessments, in order to save time and enable the mass production of human targets without hindrances. > “Everything was statistical, everything was neat — it was very dry,” B. said. He noted that this lack of supervision was permitted despite internal checks showing that Lavender’s calculations were considered accurate only 90 percent of the time; in other words, it was known in advance that 10 percent of the human targets slated for assassination were not members of the Hamas military wing at all.``` So, there was no human sign-off. I guess the policy itself was ordered by someone, but all the ongoing targets that were selected for assassination were solely authorized by the AI system's predictions. This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances"

sequoia 2y ago
Hm OK, I read this a bit differently. I read these sections: > One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target is male. > According to the sources, the army knew that the minimal human supervision in place would not discover these faults. I took this to mean that a human did press the "approve" button on the computer's recommendation. Though they make clear they were basically "rubber stamping" the machine recommendation. But to my point: > “There was no ‘zero-error’ policy. Mistakes were treated statistically,” said a source who used Lavender. What is the "zero-error" alternative approach for dropping bombs in a war, or firing rockets for that matter? I don't understand the implicit comparison between this approach to targeting and a hypothetical approach that allows war to be waged without any innocents dying or buildings being destroyed. This system should be compared to whatever the real alternative is when it comes to target selection. Again I know nothing about military strategy, I'm hoping someone with more experience will speak up. To use an analogy: if we are talking about self-driving cars, the rates of collision or death should be compared the rates of collision or death in cars driven by humans. Comparing against some imaginary scenario where cars have no collisions and cause no deaths doesn't make sense.
- stonogo 2y ago
  The difference is between inaccuracy of a weapon hitting a target and inaccuracy of target selection in the first place. Remember the scene in Men In Black where the recruita do target practice? They were all accurate at hitting what they shot at but only Will Smith's character was accurate at selecting a target. This AI chooses targets; it does not fire weapons.
  - astockwell 2y ago
    Haha having recently rewatched MIB with my daughter after ~15 years, I don't think Will Smith correctly selected the target... :'D
    - ceejayoz 2y ago
      I think you very much missed the context of that scene. https://www.youtube.com/watch?v=ORHAP6duw9E https://www.youtube.com/watch?v=ORHAP6duw9E The job is not "shoot aliens". It's manage aliens, including Earth's population of legal resident aliens (like the taxi driver who he delivers a baby for). The Big Bad of the film is indeed posing as a human, and Smith's character runs into an endless procession of innocent (or at least non-capital-crime) aliens he should not shoot along the way. There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range.
      - YeGoblynQueenne 2y ago
        >> There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range. Yes, because he's Will Smith.
        mandmandam 2y ago
        No. There's an important point being made here. Not to be the tone police or anything, but a HN discussion of AI-powered mass murder really isn't the time to be glib.
jhallenworld 2y ago
I imagine you get to tune the probability window of "person is >90% likely a Hamas terrorist" and choose how many innocent people you kill. Who set the window? "Hamas terrorist" criteria: a male of fighting age, give higher weight to those congregating with others of fighting age. Basically take out a generation of Palestinian men and you're all set. Lovely. >This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances" Reminds me of similar industrial thinking of a certain previous fascist government.
- asadalt 2y ago
  ...and then target them at home along with their entire family.
  - Joker_vD 2y ago
    But you see, if you kill just them, then their family would very likely get radicalized because of that, and then you'd have to kill them too, only some time later so it's just more efficient to do it in one fell swoop while you have the chance. Of course it's perfectly ethical, why do you ask?
    - BriggyDwiggs42 2y ago
      See i found this old book by this machiavelli guy that sums up our approach perfectly. He was really onto something here.
      - logicchains 2y ago
        Chinese emperors were doing this looong before Machiavelli.
- cm2187 2y ago
  A war that would only kill 10% civilians would be a massive improvement over any recent conflict.
manquer 2y ago
90% is a BS number . Computed basis what ? What is the baseline how did they benchmark . Is there any data whatsoever to back this claim ? They just spout a high number that is not 100% (clearly civilians are being killed publicly undeniably ) claiming 100% would be too obviously ridiculous. More than half of 32,000+ (more under the rubble) killed are woman and children, Hamas is still quite able to fight, hardly any hostages has been recovered . Israel labels any sort of civilian organization as hamas including journalists, medical and aid staff. 200 UN staff and 100 journalists are dead so far . Israel’s argument is UNWRA terrorist aiding and journalists were also secretly Hamas and doing non journalistic stuff when killed so they include them in legitimate targets . If you consider everyone is Hamas unless otherwise proven then 90% is possible . There is no realistic way an algorithm was designed factoring in the level of destruction of infrastructure never seen in any real world data and also benchmarked accurately.
- davidf18 2y ago
  [dead]
nickpsecurity 2y ago
I don’t like Lavender. I think humans should always be in the loop. I’d like to see more care by analysts for kill orders. That said, any organization might do something if it’s 90% accurate. Assuming it even is (doubt it), I think any fair evaluation of such a technology must ask: What is the accuracy of inexperienced humans in the same position who are rushing through the review during a blitz invasion? If they have battle experience, what about them, too? (I’m assuming most won’t.) Is the system better than those humans or worse? How often? Do the strengths and weaknesses of the system allow confidence scores on predictions to know which need more review? Can we also increase reviews when the number of deaths will be high? That’s how I’d start a review of this tech. If anyone is building military AI, I also ask that you please include methods to highlight likely corner cases or high-stakes situations. Then, someone’s human instincts might kick in where they spot and resolve a problem even in the heat of war.
- NomDePlum 2y ago
  Lavender AI in it's current form is not a technology that should be considered at all for anything close to this. It's incompetent. Basically AI is being used as a wheel-of-death and nothing more. Anyone believing differently in my opinion is both delusional, and complicit
amenhotep 2y ago
It is very clear to me that that is a sentence reflecting the editorial interpretation of the paper rather than a direct quote. You might agree with the interpretation - I think I might - but that is very different from this specific sentiment being something Israeli leadership are openly saying.

sequoia 2y ago

Hm OK, I read this a bit differently. I read these sections: > One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target is male. > According to the sources, the army knew that the minimal human supervision in place would not discover these faults. I took this to mean that a human did press the "approve" button on the computer's recommendation. Though they make clear they were basically "rubber stamping" the machine recommendation. But to my point: > “There was no ‘zero-error’ policy. Mistakes were treated statistically,” said a source who used Lavender. What is the "zero-error" alternative approach for dropping bombs in a war, or firing rockets for that matter? I don't understand the implicit comparison between this approach to targeting and a hypothetical approach that allows war to be waged without any innocents dying or buildings being destroyed. This system should be compared to whatever the real alternative is when it comes to target selection. Again I know nothing about military strategy, I'm hoping someone with more experience will speak up. To use an analogy: if we are talking about self-driving cars, the rates of collision or death should be compared the rates of collision or death in cars driven by humans. Comparing against some imaginary scenario where cars have no collisions and cause no deaths doesn't make sense.

stonogo 2y ago
The difference is between inaccuracy of a weapon hitting a target and inaccuracy of target selection in the first place. Remember the scene in Men In Black where the recruita do target practice? They were all accurate at hitting what they shot at but only Will Smith's character was accurate at selecting a target. This AI chooses targets; it does not fire weapons.
- astockwell 2y ago
  Haha having recently rewatched MIB with my daughter after ~15 years, I don't think Will Smith correctly selected the target... :'D
  - ceejayoz 2y ago
    I think you very much missed the context of that scene. https://www.youtube.com/watch?v=ORHAP6duw9E https://www.youtube.com/watch?v=ORHAP6duw9E The job is not "shoot aliens". It's manage aliens, including Earth's population of legal resident aliens (like the taxi driver who he delivers a baby for). The Big Bad of the film is indeed posing as a human, and Smith's character runs into an endless procession of innocent (or at least non-capital-crime) aliens he should not shoot along the way. There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range.
    - YeGoblynQueenne 2y ago
      >> There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range. Yes, because he's Will Smith.
      - mandmandam 2y ago
        No. There's an important point being made here. Not to be the tone police or anything, but a HN discussion of AI-powered mass murder really isn't the time to be glib.

jhallenworld 2y ago

I imagine you get to tune the probability window of "person is >90% likely a Hamas terrorist" and choose how many innocent people you kill. Who set the window? "Hamas terrorist" criteria: a male of fighting age, give higher weight to those congregating with others of fighting age. Basically take out a generation of Palestinian men and you're all set. Lovely. >This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances" Reminds me of similar industrial thinking of a certain previous fascist government.

asadalt 2y ago
...and then target them at home along with their entire family.
- Joker_vD 2y ago
  But you see, if you kill just them, then their family would very likely get radicalized because of that, and then you'd have to kill them too, only some time later so it's just more efficient to do it in one fell swoop while you have the chance. Of course it's perfectly ethical, why do you ask?
  - BriggyDwiggs42 2y ago
    See i found this old book by this machiavelli guy that sums up our approach perfectly. He was really onto something here.
    - logicchains 2y ago
      Chinese emperors were doing this looong before Machiavelli.
cm2187 2y ago
A war that would only kill 10% civilians would be a massive improvement over any recent conflict.

manquer 2y ago

90% is a BS number . Computed basis what ? What is the baseline how did they benchmark . Is there any data whatsoever to back this claim ? They just spout a high number that is not 100% (clearly civilians are being killed publicly undeniably ) claiming 100% would be too obviously ridiculous. More than half of 32,000+ (more under the rubble) killed are woman and children, Hamas is still quite able to fight, hardly any hostages has been recovered . Israel labels any sort of civilian organization as hamas including journalists, medical and aid staff. 200 UN staff and 100 journalists are dead so far . Israel’s argument is UNWRA terrorist aiding and journalists were also secretly Hamas and doing non journalistic stuff when killed so they include them in legitimate targets . If you consider everyone is Hamas unless otherwise proven then 90% is possible . There is no realistic way an algorithm was designed factoring in the level of destruction of infrastructure never seen in any real world data and also benchmarked accurately.

davidf18 2y ago
[dead]

nickpsecurity 2y ago

I don’t like Lavender. I think humans should always be in the loop. I’d like to see more care by analysts for kill orders. That said, any organization might do something if it’s 90% accurate. Assuming it even is (doubt it), I think any fair evaluation of such a technology must ask: What is the accuracy of inexperienced humans in the same position who are rushing through the review during a blitz invasion? If they have battle experience, what about them, too? (I’m assuming most won’t.) Is the system better than those humans or worse? How often? Do the strengths and weaknesses of the system allow confidence scores on predictions to know which need more review? Can we also increase reviews when the number of deaths will be high? That’s how I’d start a review of this tech. If anyone is building military AI, I also ask that you please include methods to highlight likely corner cases or high-stakes situations. Then, someone’s human instincts might kick in where they spot and resolve a problem even in the heat of war.

NomDePlum 2y ago
Lavender AI in it's current form is not a technology that should be considered at all for anything close to this. It's incompetent. Basically AI is being used as a wheel-of-death and nothing more. Anyone believing differently in my opinion is both delusional, and complicit

amenhotep 2y ago

It is very clear to me that that is a sentence reflecting the editorial interpretation of the paper rather than a direct quote. You might agree with the interpretation - I think I might - but that is very different from this specific sentiment being something Israeli leadership are openly saying.