20 ms·

I'm disturbed by the idea that an AI could be used to make decisions that could proactively kill someone. (Presumably computer already make decisions that passi

I'm disturbed by the idea that an AI could be used to make decisions that could proactively kill someone. (Presumably computer already make decisions that passively kill people by, for example, navigating a self-driving car.) Though there was a human sign-off in this case, it seems one step away from people being killed by robots with zero human intervention which is about one step away from the plot of Terminator. I wonder what the alternative is in a case like this. I know very little about military strategy-- without the AI would Israel have been picking targets less, or more haphazardly? I think there may be some mis-reading of this article where people imagine that if Israel weren't using an AI they wouldn't drop any bombs at all, that's clearly unlikely given that there's a war on. Obviously people, including innocents, are killed in war, which is why we all loathe war and pray for the current one to end as quickly as possible.

readyplayeremma 3y ago
> B., a senior officer who used Lavender, echoed to +972 and Local Call that in the current war, officers were not required to independently review the AI system’s assessments, in order to save time and enable the mass production of human targets without hindrances. > “Everything was statistical, everything was neat — it was very dry,” B. said. He noted that this lack of supervision was permitted despite internal checks showing that Lavender’s calculations were considered accurate only 90 percent of the time; in other words, it was known in advance that 10 percent of the human targets slated for assassination were not members of the Hamas military wing at all.``` So, there was no human sign-off. I guess the policy itself was ordered by someone, but all the ongoing targets that were selected for assassination were solely authorized by the AI system's predictions. This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances"
- sequoia 3y ago
  Hm OK, I read this a bit differently. I read these sections: > One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target is male. > According to the sources, the army knew that the minimal human supervision in place would not discover these faults. I took this to mean that a human did press the "approve" button on the computer's recommendation. Though they make clear they were basically "rubber stamping" the machine recommendation. But to my point: > “There was no ‘zero-error’ policy. Mistakes were treated statistically,” said a source who used Lavender. What is the "zero-error" alternative approach for dropping bombs in a war, or firing rockets for that matter? I don't understand the implicit comparison between this approach to targeting and a hypothetical approach that allows war to be waged without any innocents dying or buildings being destroyed. This system should be compared to whatever the real alternative is when it comes to target selection. Again I know nothing about military strategy, I'm hoping someone with more experience will speak up. To use an analogy: if we are talking about self-driving cars, the rates of collision or death should be compared the rates of collision or death in cars driven by humans. Comparing against some imaginary scenario where cars have no collisions and cause no deaths doesn't make sense.
  - stonogo 3y ago
    The difference is between inaccuracy of a weapon hitting a target and inaccuracy of target selection in the first place. Remember the scene in Men In Black where the recruita do target practice? They were all accurate at hitting what they shot at but only Will Smith's character was accurate at selecting a target. This AI chooses targets; it does not fire weapons.
    - astockwell 3y ago
      Haha having recently rewatched MIB with my daughter after ~15 years, I don't think Will Smith correctly selected the target... :'D
      - ceejayoz 3y ago
        I think you very much missed the context of that scene. https://www.youtube.com/watch?v=ORHAP6duw9E https://www.youtube.com/watch?v=ORHAP6duw9E The job is not "shoot aliens". It's manage aliens, including Earth's population of legal resident aliens (like the taxi driver who he delivers a baby for). The Big Bad of the film is indeed posing as a human, and Smith's character runs into an endless procession of innocent (or at least non-capital-crime) aliens he should not shoot along the way. There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range.
        YeGoblynQueenne 3y ago
        >> There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range. Yes, because he's Will Smith.
        mandmandam 3y ago
        No. There's an important point being made here. Not to be the tone police or anything, but a HN discussion of AI-powered mass murder really isn't the time to be glib.
- jhallenworld 3y ago
  I imagine you get to tune the probability window of "person is >90% likely a Hamas terrorist" and choose how many innocent people you kill. Who set the window? "Hamas terrorist" criteria: a male of fighting age, give higher weight to those congregating with others of fighting age. Basically take out a generation of Palestinian men and you're all set. Lovely. >This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances" Reminds me of similar industrial thinking of a certain previous fascist government.
  - asadalt 3y ago
    ...and then target them at home along with their entire family.
    - Joker_vD 3y ago
      But you see, if you kill just them, then their family would very likely get radicalized because of that, and then you'd have to kill them too, only some time later so it's just more efficient to do it in one fell swoop while you have the chance. Of course it's perfectly ethical, why do you ask?
      - BriggyDwiggs42 3y ago
        See i found this old book by this machiavelli guy that sums up our approach perfectly. He was really onto something here.
        logicchains 3y ago
        Chinese emperors were doing this looong before Machiavelli.
  - cm2187 3y ago
    A war that would only kill 10% civilians would be a massive improvement over any recent conflict.
- manquer 3y ago
  90% is a BS number . Computed basis what ? What is the baseline how did they benchmark . Is there any data whatsoever to back this claim ? They just spout a high number that is not 100% (clearly civilians are being killed publicly undeniably ) claiming 100% would be too obviously ridiculous. More than half of 32,000+ (more under the rubble) killed are woman and children, Hamas is still quite able to fight, hardly any hostages has been recovered . Israel labels any sort of civilian organization as hamas including journalists, medical and aid staff. 200 UN staff and 100 journalists are dead so far . Israel’s argument is UNWRA terrorist aiding and journalists were also secretly Hamas and doing non journalistic stuff when killed so they include them in legitimate targets . If you consider everyone is Hamas unless otherwise proven then 90% is possible . There is no realistic way an algorithm was designed factoring in the level of destruction of infrastructure never seen in any real world data and also benchmarked accurately.
  - davidf18 3y ago
    [dead]
- nickpsecurity 3y ago
  I don’t like Lavender. I think humans should always be in the loop. I’d like to see more care by analysts for kill orders. That said, any organization might do something if it’s 90% accurate. Assuming it even is (doubt it), I think any fair evaluation of such a technology must ask: What is the accuracy of inexperienced humans in the same position who are rushing through the review during a blitz invasion? If they have battle experience, what about them, too? (I’m assuming most won’t.) Is the system better than those humans or worse? How often? Do the strengths and weaknesses of the system allow confidence scores on predictions to know which need more review? Can we also increase reviews when the number of deaths will be high? That’s how I’d start a review of this tech. If anyone is building military AI, I also ask that you please include methods to highlight likely corner cases or high-stakes situations. Then, someone’s human instincts might kick in where they spot and resolve a problem even in the heat of war.
  - NomDePlum 3y ago
    Lavender AI in it's current form is not a technology that should be considered at all for anything close to this. It's incompetent. Basically AI is being used as a wheel-of-death and nothing more. Anyone believing differently in my opinion is both delusional, and complicit
- amenhotep 3y ago
  It is very clear to me that that is a sentence reflecting the editorial interpretation of the paper rather than a direct quote. You might agree with the interpretation - I think I might - but that is very different from this specific sentiment being something Israeli leadership are openly saying.
yrro 3y ago
"A computer can never be held accountable, therefore a computer must never make a management decision" The IDF only read the first half of the classic IBM slide!
tmnvix 3y ago
> I wonder what the alternative is in a case like this. It seems obvious to me that the alternative would be a slower process for picking targets leading to fewer overall targets picked and the guarantee that a human conscience is involved in the process.
- ranger207 3y ago
  Or alternatively pressure from the top down on targeting specialists to get more and more targets selected resulting in less quality and effort spent on selecting targets and maybe leading to rubber-stamping proposed targets without adequate consideration. Which isn't to suggest that that would definitely make the AI better per se
  - cess11 3y ago
    It's an army too cowardly to have dismounted infantry protecting their tanks, so instead their conscripts burn alive in there when they get in contact with actual militants. It's an army incompetent enough to recreate the rubble of Stalingrad to help its enemy. How would they go about producing officers that could enact such pressure? How would they recognise the difference between a specialist and a charlatan whos family is good friends with the army rabbi?
    - jsmith99 3y ago
      The weirdest thing about this bizarre comment is the suggestion that rabbis have any influence on the Israeli army.
      - deleted 3y ago
        [deleted]
      - cess11 3y ago
        What do you mean, "bizarre"? It's well known that the IDF refuses to use dismounted infantry to protect their wagons, and that they've turned cities into rubble and given themselves some of the same kind of problems that the Nazis had in Stalingrad. https://www.idf.il/en/mini-sites/military-rabbinate/ https://www.idf.il/en/mini-sites/military-rabbinate/ You'll also find interesting stories in israeli papers. Rabbis are important to the IDF because the state it is part of is based on religious convictions, and quite often there is no other justification for what they do.
robbomacrae 3y ago
Disturbing indeed. I've been worried a push back in AI is coming and this sort of story could be a tipping point and certainly would justify a period of reflection. And your probably right that the alternatives maybe worse, the folks behind Lavender could probably even prove it with data.. but there should be a moral impetus to always have a human in the loop regardless. And any such attempt to justify it won't capture the publics attention like a sky-net doomsday happening over the civilians in Gaza.
- coffeebeqn 3y ago
  Pushback on AI will of course have a “National security” exception. If the industrial level facial recognition tech in Xinjiang was forgotten I doubt this will make a difference
- wruza 3y ago
  there should be a moral impetus to always have a human in the loop regardless I don’t understand how to come to this. War is crap, not a dinner party. There’s always a human on both sides who will drop a bomb and laugh on camera, with no responsibility. Go watch it (actually don’t, it’s NSFL). Reading this thread feels like everyone watched and believed in that movie where they tried to select and eliminate a target for 2 hours with futuristic hi-tech. A human hesitates to press the button before the war. When in it, he will only be concerned with things like ammunition saving and tactical nuances. There’s not much more morals in a human who usually sits there at the button than in AI automation.
  - outside1234 3y ago
    The thing that is different is now that human has an excuse: "The computer told me to put them in the oven."
  - robbomacrae 3y ago
    It's a good question. I also immediately found myself asking the same one of myself after posting that comment. I guess part of me just wants as many possible breakpoints along the process as possible. But also at least then you have someone who is liable when things go wrong. When its fully automated, like the other comment mentions, they can just shrug and blame the AI. Who gets sued when a self driving car kills someone by accident? I don't know. Perhaps a lack of ownership is excusable. But when a weapon deliberately kills someone I think we need to have ownership somewhere. Perhaps as a general rule the maker of the AI system should have liability for the AI up until someone else signs and accepts that responsibility. None of this "Company does not accept liability" crap. They have to make it clear that "customer accepts liability" or else it's them. That way they will be incentivized to make the military or whoever sign.
solardev 3y ago
Is having a human make those decisions really better? It was humans who ordered the Holocaust, My Lai, Wounded Knee, Rwanda, Tiananmen, etc. At least AI pretends to look at some data instead of just defaulting to tribal bloodlust... who's to say it can't be more ethical? It doesn't take much to beat our track record.
- cm2187 3y ago
  I think people are worried no one really understands how AI picks the target. Reminds me of that story from probably 5-7y ago. Someone wanted to use AI to classify photos of tanks as soviet vs US. So he went to a US tank museum and took lots of pictures of the tanks under every angle. Did the same in a soviet tank museum. The resulting model worked great on that training dataset. Then he tried on photos outside of the training dataset. Turned out that it was cloudy the day he visited the US museum and sunny for the soviet museum, and the model used the color of the sky to classify.
  - artificial 3y ago
    Seems like segmentation would be a better approach to identity objects in a photo rather than various other features.
  - ben_w 3y ago
    An eternal story; I heard the same thing at university 22 years ago, except then it was NATO taking nice crisp in-focus photos of their own tanks from close up, while the images of Soviet tanks were all blurry and grainy because they came from high-altitude spy planes. (This kind of human model hallucination is how and why I think Genesis got written and taken seriously). https://gwern.net/tank https://gwern.net/tank
  - solardev 3y ago
    > I think people are worried no one really understands how AI picks the target. Yeah, I mean, black-box murder is never really desirable... but is it fair to assume AI will never be able to elucidate its reasoning? And that also seems a bit of a double standard, when so many life-and-death decisions made by humans are also not entirely comprehensible or transparent, either to the general public or sometimes even to the other individuals closest to the decision-maker. Sometimes it's a snap judgment, sometimes it's a gut feeling, sometimes it's bad intel, sometimes it's just plain "because I said so"... not every kill list is the result of a reasoned, transparent, fair and ethical process. After all, how long have Israel and Hamas (or other groups) been at each other's throats, with cries of injustice and atrocities about either side, from observers all over the world? And it wasn't so long ago we destroyed Afghanistan and Iraq, and Russia is still going at it because of the desires of one man. AI doesn't have to be perfect to be better than us. If there's one thing humans are really, really bad at, it's letting objective data overrule our emotional states. It takes exceptional training and mental fortitude to be able to do that under pressure, especially life-and-death, us-vs-them pressure. Humans make mistakes, too, and friend-or-foe identification isn't easy for humans either, especially in the heat of battle or in poor visibility. Training for either humans or AI can always be improved, but probably will never reach 100% accuracy. Maybe we should start putting some hypothetical kill lists in front of both humans and AI, recording their decisions, and comparing them after a few years to see who did "better". I wouldn't necessarily bet on the humans...
cm2187 3y ago
We crossed the line of machines that automatically kill a long time ago. A heat seeking missile, or a shell that detects and target tanks [1] are effectively doing that. Software selects the target. The soldier only points in the general direction. AI is only a small technical increment. [1] https://en.wikipedia.org/wiki/SMArt_155 https://en.wikipedia.org/wiki/SMArt_155
- LordShredda 3y ago
  But you know soldiers are in a tank, and you know a pilot is in a plane. Who's in an apartment?
  - deleted 3y ago
    [deleted]
  - hyperliner 3y ago
    [dead]
  - solardev 3y ago
    It's never really that clear-cut, though. Human drone operators, pilots, etc. routinely send missiles into cars, buildings, weddings, etc. that cause collateral damage, killing or maiming innocents and passers-by. Sometimes it's an accident, but not always. And that's just when we even try to limit damage, vs indiscriminately firebombing or nuking entire cities. We shouldn't demand perfect accuracy of AI when we don't expect the same of humans. Long ago, we decided collateral damage in war is acceptable, especially when you end up winning the war and there's nobody left to prosecute you except historians =/
- robbomacrae 3y ago
  Difference is someone has to fire those weapons and that someone is responsible.
deleted 3y ago
[deleted]
underlipton 3y ago
>I wonder what the alternative is in a case like this. Don't Create The Torment Nexus I think that once you start from the viewpoint that you're not going to create the Torment Nexus, it becomes a lot easier to avoid creating the Torment Nexus.
0x457 3y ago
This system bassicaly just gave everyone a score from 1 to 100 of how luckely they are part the military wing of hamas. Another system would signal that target is at home and it's time to bomb. This system was using phone to geo-locate and due to nature of living in Gaza phones transfer hands often. Without Lavender they would have dropped less bombs IMO.
BriggyDwiggs42 3y ago
Look I know this is gonna sound cliche but the thing they should do is not engage in an offensive asymmetrical war and bomb a dense urban area full of innocents for basically no reason. Then they wouldn’t need the little ai.
- sequoia 3y ago
  This is obviously veering way off course of the topic of AI at this point, but I imagine the residents of kibbutz be'eri and the 100+ hostages still held in Gaza would disagree that Isreal is fighting for "basically no reason." I'm interested in analysis and criticism of Israel's use of AI in this case but suggesting Israel has no causus belli is absurd.
  - mandmandam 3y ago
    OP didn't say Israel is fighting for basically no reason. You're twisting their words, I'll assume out of a misreading. Read the comment again. They clearly said that there's no good reason to bomb Gaza the way that they have been doing, resulting in the murder of thousands and thousands and thousands and thousands of civilians.
    - flyinglizard 3y ago
      No one offered an alternative, either.
      - C6JEsQeQa5fCjE 3y ago
        Alternative to accomplish what exactly? To get hostages back? Hostage exchange. To dismantling Hamas and armed resistance from Palestinians? Stop ethnically cleansing the West Bank and remove the boot from their necks so that people in Gaza see that there is a better way and that going down fighting is not the only choice. Plenty of alternatives to death and destruction when those are not the actual goal itself. Of course those alternatives do not go hand in hand with the idea of an ethnically-cleansed Greater Israel, so here we are.
kromem 3y ago
There would have been slower target selection. A lot of news around the bombing called out the uniquely large scale and rapidity of the campaign. This was a preview of future conflicts. We're entering the WWI phase of new technology being brought without rules to conflicts where the abuses will be horrific until rules are finally put in place.
emadabdulrahim 3y ago
[flagged]
- flyinglizard 3y ago
  [flagged]
  - deleted 3y ago
    [deleted]

readyplayeremma 3y ago

> B., a senior officer who used Lavender, echoed to +972 and Local Call that in the current war, officers were not required to independently review the AI system’s assessments, in order to save time and enable the mass production of human targets without hindrances. > “Everything was statistical, everything was neat — it was very dry,” B. said. He noted that this lack of supervision was permitted despite internal checks showing that Lavender’s calculations were considered accurate only 90 percent of the time; in other words, it was known in advance that 10 percent of the human targets slated for assassination were not members of the Hamas military wing at all.``` So, there was no human sign-off. I guess the policy itself was ordered by someone, but all the ongoing targets that were selected for assassination were solely authorized by the AI system's predictions. This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances"

sequoia 3y ago
Hm OK, I read this a bit differently. I read these sections: > One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target is male. > According to the sources, the army knew that the minimal human supervision in place would not discover these faults. I took this to mean that a human did press the "approve" button on the computer's recommendation. Though they make clear they were basically "rubber stamping" the machine recommendation. But to my point: > “There was no ‘zero-error’ policy. Mistakes were treated statistically,” said a source who used Lavender. What is the "zero-error" alternative approach for dropping bombs in a war, or firing rockets for that matter? I don't understand the implicit comparison between this approach to targeting and a hypothetical approach that allows war to be waged without any innocents dying or buildings being destroyed. This system should be compared to whatever the real alternative is when it comes to target selection. Again I know nothing about military strategy, I'm hoping someone with more experience will speak up. To use an analogy: if we are talking about self-driving cars, the rates of collision or death should be compared the rates of collision or death in cars driven by humans. Comparing against some imaginary scenario where cars have no collisions and cause no deaths doesn't make sense.
- stonogo 3y ago
  The difference is between inaccuracy of a weapon hitting a target and inaccuracy of target selection in the first place. Remember the scene in Men In Black where the recruita do target practice? They were all accurate at hitting what they shot at but only Will Smith's character was accurate at selecting a target. This AI chooses targets; it does not fire weapons.
  - astockwell 3y ago
    Haha having recently rewatched MIB with my daughter after ~15 years, I don't think Will Smith correctly selected the target... :'D
    - ceejayoz 3y ago
      I think you very much missed the context of that scene. https://www.youtube.com/watch?v=ORHAP6duw9E https://www.youtube.com/watch?v=ORHAP6duw9E The job is not "shoot aliens". It's manage aliens, including Earth's population of legal resident aliens (like the taxi driver who he delivers a baby for). The Big Bad of the film is indeed posing as a human, and Smith's character runs into an endless procession of innocent (or at least non-capital-crime) aliens he should not shoot along the way. There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range.
      - YeGoblynQueenne 3y ago
        >> There's a reason he gets hired over all the military folks in the scene immediately blasting away at the aliens in the shooting range. Yes, because he's Will Smith.
        mandmandam 3y ago
        No. There's an important point being made here. Not to be the tone police or anything, but a HN discussion of AI-powered mass murder really isn't the time to be glib.
jhallenworld 3y ago
I imagine you get to tune the probability window of "person is >90% likely a Hamas terrorist" and choose how many innocent people you kill. Who set the window? "Hamas terrorist" criteria: a male of fighting age, give higher weight to those congregating with others of fighting age. Basically take out a generation of Palestinian men and you're all set. Lovely. >This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances" Reminds me of similar industrial thinking of a certain previous fascist government.
- asadalt 3y ago
  ...and then target them at home along with their entire family.
  - Joker_vD 3y ago
    But you see, if you kill just them, then their family would very likely get radicalized because of that, and then you'd have to kill them too, only some time later so it's just more efficient to do it in one fell swoop while you have the chance. Of course it's perfectly ethical, why do you ask?
    - BriggyDwiggs42 3y ago
      See i found this old book by this machiavelli guy that sums up our approach perfectly. He was really onto something here.
      - logicchains 3y ago
        Chinese emperors were doing this looong before Machiavelli.
- cm2187 3y ago
  A war that would only kill 10% civilians would be a massive improvement over any recent conflict.
manquer 3y ago
90% is a BS number . Computed basis what ? What is the baseline how did they benchmark . Is there any data whatsoever to back this claim ? They just spout a high number that is not 100% (clearly civilians are being killed publicly undeniably ) claiming 100% would be too obviously ridiculous. More than half of 32,000+ (more under the rubble) killed are woman and children, Hamas is still quite able to fight, hardly any hostages has been recovered . Israel labels any sort of civilian organization as hamas including journalists, medical and aid staff. 200 UN staff and 100 journalists are dead so far . Israel’s argument is UNWRA terrorist aiding and journalists were also secretly Hamas and doing non journalistic stuff when killed so they include them in legitimate targets . If you consider everyone is Hamas unless otherwise proven then 90% is possible . There is no realistic way an algorithm was designed factoring in the level of destruction of infrastructure never seen in any real world data and also benchmarked accurately.
- davidf18 3y ago
  [dead]
nickpsecurity 3y ago
I don’t like Lavender. I think humans should always be in the loop. I’d like to see more care by analysts for kill orders. That said, any organization might do something if it’s 90% accurate. Assuming it even is (doubt it), I think any fair evaluation of such a technology must ask: What is the accuracy of inexperienced humans in the same position who are rushing through the review during a blitz invasion? If they have battle experience, what about them, too? (I’m assuming most won’t.) Is the system better than those humans or worse? How often? Do the strengths and weaknesses of the system allow confidence scores on predictions to know which need more review? Can we also increase reviews when the number of deaths will be high? That’s how I’d start a review of this tech. If anyone is building military AI, I also ask that you please include methods to highlight likely corner cases or high-stakes situations. Then, someone’s human instincts might kick in where they spot and resolve a problem even in the heat of war.
- NomDePlum 3y ago
  Lavender AI in it's current form is not a technology that should be considered at all for anything close to this. It's incompetent. Basically AI is being used as a wheel-of-death and nothing more. Anyone believing differently in my opinion is both delusional, and complicit
amenhotep 3y ago
It is very clear to me that that is a sentence reflecting the editorial interpretation of the paper rather than a direct quote. You might agree with the interpretation - I think I might - but that is very different from this specific sentiment being something Israeli leadership are openly saying.

yrro 3y ago

"A computer can never be held accountable, therefore a computer must never make a management decision" The IDF only read the first half of the classic IBM slide!

tmnvix 3y ago

> I wonder what the alternative is in a case like this. It seems obvious to me that the alternative would be a slower process for picking targets leading to fewer overall targets picked and the guarantee that a human conscience is involved in the process.

ranger207 3y ago
Or alternatively pressure from the top down on targeting specialists to get more and more targets selected resulting in less quality and effort spent on selecting targets and maybe leading to rubber-stamping proposed targets without adequate consideration. Which isn't to suggest that that would definitely make the AI better per se
- cess11 3y ago
  It's an army too cowardly to have dismounted infantry protecting their tanks, so instead their conscripts burn alive in there when they get in contact with actual militants. It's an army incompetent enough to recreate the rubble of Stalingrad to help its enemy. How would they go about producing officers that could enact such pressure? How would they recognise the difference between a specialist and a charlatan whos family is good friends with the army rabbi?
  - jsmith99 3y ago
    The weirdest thing about this bizarre comment is the suggestion that rabbis have any influence on the Israeli army.
    - deleted 3y ago
      [deleted]
    - cess11 3y ago
      What do you mean, "bizarre"? It's well known that the IDF refuses to use dismounted infantry to protect their wagons, and that they've turned cities into rubble and given themselves some of the same kind of problems that the Nazis had in Stalingrad. https://www.idf.il/en/mini-sites/military-rabbinate/ https://www.idf.il/en/mini-sites/military-rabbinate/ You'll also find interesting stories in israeli papers. Rabbis are important to the IDF because the state it is part of is based on religious convictions, and quite often there is no other justification for what they do.

robbomacrae 3y ago

Disturbing indeed. I've been worried a push back in AI is coming and this sort of story could be a tipping point and certainly would justify a period of reflection. And your probably right that the alternatives maybe worse, the folks behind Lavender could probably even prove it with data.. but there should be a moral impetus to always have a human in the loop regardless. And any such attempt to justify it won't capture the publics attention like a sky-net doomsday happening over the civilians in Gaza.

coffeebeqn 3y ago
Pushback on AI will of course have a “National security” exception. If the industrial level facial recognition tech in Xinjiang was forgotten I doubt this will make a difference
wruza 3y ago
there should be a moral impetus to always have a human in the loop regardless I don’t understand how to come to this. War is crap, not a dinner party. There’s always a human on both sides who will drop a bomb and laugh on camera, with no responsibility. Go watch it (actually don’t, it’s NSFL). Reading this thread feels like everyone watched and believed in that movie where they tried to select and eliminate a target for 2 hours with futuristic hi-tech. A human hesitates to press the button before the war. When in it, he will only be concerned with things like ammunition saving and tactical nuances. There’s not much more morals in a human who usually sits there at the button than in AI automation.
- outside1234 3y ago
  The thing that is different is now that human has an excuse: "The computer told me to put them in the oven."
- robbomacrae 3y ago
  It's a good question. I also immediately found myself asking the same one of myself after posting that comment. I guess part of me just wants as many possible breakpoints along the process as possible. But also at least then you have someone who is liable when things go wrong. When its fully automated, like the other comment mentions, they can just shrug and blame the AI. Who gets sued when a self driving car kills someone by accident? I don't know. Perhaps a lack of ownership is excusable. But when a weapon deliberately kills someone I think we need to have ownership somewhere. Perhaps as a general rule the maker of the AI system should have liability for the AI up until someone else signs and accepts that responsibility. None of this "Company does not accept liability" crap. They have to make it clear that "customer accepts liability" or else it's them. That way they will be incentivized to make the military or whoever sign.

solardev 3y ago

Is having a human make those decisions really better? It was humans who ordered the Holocaust, My Lai, Wounded Knee, Rwanda, Tiananmen, etc. At least AI pretends to look at some data instead of just defaulting to tribal bloodlust... who's to say it can't be more ethical? It doesn't take much to beat our track record.

cm2187 3y ago
I think people are worried no one really understands how AI picks the target. Reminds me of that story from probably 5-7y ago. Someone wanted to use AI to classify photos of tanks as soviet vs US. So he went to a US tank museum and took lots of pictures of the tanks under every angle. Did the same in a soviet tank museum. The resulting model worked great on that training dataset. Then he tried on photos outside of the training dataset. Turned out that it was cloudy the day he visited the US museum and sunny for the soviet museum, and the model used the color of the sky to classify.
- artificial 3y ago
  Seems like segmentation would be a better approach to identity objects in a photo rather than various other features.
- ben_w 3y ago
  An eternal story; I heard the same thing at university 22 years ago, except then it was NATO taking nice crisp in-focus photos of their own tanks from close up, while the images of Soviet tanks were all blurry and grainy because they came from high-altitude spy planes. (This kind of human model hallucination is how and why I think Genesis got written and taken seriously). https://gwern.net/tank https://gwern.net/tank
- solardev 3y ago
  > I think people are worried no one really understands how AI picks the target. Yeah, I mean, black-box murder is never really desirable... but is it fair to assume AI will never be able to elucidate its reasoning? And that also seems a bit of a double standard, when so many life-and-death decisions made by humans are also not entirely comprehensible or transparent, either to the general public or sometimes even to the other individuals closest to the decision-maker. Sometimes it's a snap judgment, sometimes it's a gut feeling, sometimes it's bad intel, sometimes it's just plain "because I said so"... not every kill list is the result of a reasoned, transparent, fair and ethical process. After all, how long have Israel and Hamas (or other groups) been at each other's throats, with cries of injustice and atrocities about either side, from observers all over the world? And it wasn't so long ago we destroyed Afghanistan and Iraq, and Russia is still going at it because of the desires of one man. AI doesn't have to be perfect to be better than us. If there's one thing humans are really, really bad at, it's letting objective data overrule our emotional states. It takes exceptional training and mental fortitude to be able to do that under pressure, especially life-and-death, us-vs-them pressure. Humans make mistakes, too, and friend-or-foe identification isn't easy for humans either, especially in the heat of battle or in poor visibility. Training for either humans or AI can always be improved, but probably will never reach 100% accuracy. Maybe we should start putting some hypothetical kill lists in front of both humans and AI, recording their decisions, and comparing them after a few years to see who did "better". I wouldn't necessarily bet on the humans...

cm2187 3y ago

We crossed the line of machines that automatically kill a long time ago. A heat seeking missile, or a shell that detects and target tanks [1] are effectively doing that. Software selects the target. The soldier only points in the general direction. AI is only a small technical increment. [1] https://en.wikipedia.org/wiki/SMArt_155 https://en.wikipedia.org/wiki/SMArt_155

LordShredda 3y ago
But you know soldiers are in a tank, and you know a pilot is in a plane. Who's in an apartment?
- deleted 3y ago
  [deleted]
- hyperliner 3y ago
  [dead]
- solardev 3y ago
  It's never really that clear-cut, though. Human drone operators, pilots, etc. routinely send missiles into cars, buildings, weddings, etc. that cause collateral damage, killing or maiming innocents and passers-by. Sometimes it's an accident, but not always. And that's just when we even try to limit damage, vs indiscriminately firebombing or nuking entire cities. We shouldn't demand perfect accuracy of AI when we don't expect the same of humans. Long ago, we decided collateral damage in war is acceptable, especially when you end up winning the war and there's nobody left to prosecute you except historians =/
robbomacrae 3y ago
Difference is someone has to fire those weapons and that someone is responsible.

deleted 3y ago

[deleted]

underlipton 3y ago

>I wonder what the alternative is in a case like this. Don't Create The Torment Nexus I think that once you start from the viewpoint that you're not going to create the Torment Nexus, it becomes a lot easier to avoid creating the Torment Nexus.

0x457 3y ago

This system bassicaly just gave everyone a score from 1 to 100 of how luckely they are part the military wing of hamas. Another system would signal that target is at home and it's time to bomb. This system was using phone to geo-locate and due to nature of living in Gaza phones transfer hands often. Without Lavender they would have dropped less bombs IMO.

BriggyDwiggs42 3y ago

Look I know this is gonna sound cliche but the thing they should do is not engage in an offensive asymmetrical war and bomb a dense urban area full of innocents for basically no reason. Then they wouldn’t need the little ai.

sequoia 3y ago
This is obviously veering way off course of the topic of AI at this point, but I imagine the residents of kibbutz be'eri and the 100+ hostages still held in Gaza would disagree that Isreal is fighting for "basically no reason." I'm interested in analysis and criticism of Israel's use of AI in this case but suggesting Israel has no causus belli is absurd.
- mandmandam 3y ago
  OP didn't say Israel is fighting for basically no reason. You're twisting their words, I'll assume out of a misreading. Read the comment again. They clearly said that there's no good reason to bomb Gaza the way that they have been doing, resulting in the murder of thousands and thousands and thousands and thousands of civilians.
  - flyinglizard 3y ago
    No one offered an alternative, either.
    - C6JEsQeQa5fCjE 3y ago
      Alternative to accomplish what exactly? To get hostages back? Hostage exchange. To dismantling Hamas and armed resistance from Palestinians? Stop ethnically cleansing the West Bank and remove the boot from their necks so that people in Gaza see that there is a better way and that going down fighting is not the only choice. Plenty of alternatives to death and destruction when those are not the actual goal itself. Of course those alternatives do not go hand in hand with the idea of an ethnically-cleansed Greater Israel, so here we are.

kromem 3y ago

There would have been slower target selection. A lot of news around the bombing called out the uniquely large scale and rapidity of the campaign. This was a preview of future conflicts. We're entering the WWI phase of new technology being brought without rules to conflicts where the abuses will be horrific until rules are finally put in place.

emadabdulrahim 3y ago

[flagged]

flyinglizard 3y ago
[flagged]
- deleted 3y ago
  [deleted]