8 ms·

Opsec is hard. Also. No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. It cr

by lukev 3y ago

Opsec is hard. Also. No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. It creates the possibility of atrocity with no accountability. Although the fact that this guy was anonymous to start with definitely indicates that accountability isn't the goal here.

roughly 3y ago
The Guardian reported more on the IDF’s use of AI a couple days back, and it’s grim: https://www.theguardian.com/world/2024/apr/03/israel-gaza-ai-database-hamas-airstrikes https://www.theguardian.com/world/2024/apr/03/israel-gaza-ai... Although in that article, most of the horrors are coming from the human operators - there’s what comes off as a pretty strong disinterest in finding reasons to distrust the machine.
- 082349872349872 3y ago
  There must be a line somewhere between "pretty strong disinterest in finding reasons to distrust the machine" and "using the machine as plausible deniability for doing what you hoped to do anyway"; is there a relatively simple description of this line, or is it a fractal boundary?
  - roughly 3y ago
    Yeah, I phrased that as generously as possible because I didn’t really want to kick that particular hornets’ nest, but the article does a fine job making that point.
    - aebtebeten 3y ago
      Very charitable of you! (do we need an emoticon for charity, like we have for sarcasm? symmetry would suggest yes.) Upon further reflection, it occurs to me that these situations are exactly why the Geneva Conventions[0] require combatants to have (a) uniforms, and (b) a command structure: (a) prevents someone low from disingenuously stopping the buck by claiming "tweren't no orders; I'm acting on my own" (b) prevents someone high from arguing as Uber does: "you see, we're not a traditional army that gives orders to our fighters — we just provide a service that matches attackers and targets online, and really, anything bad that allegedly happened was only between those two parties"[1] [0] if we're not doing these anymore, just go ahead and tell me "the future is now, old man", and I'll take my mutterings to the pétanque piste. [1] > "Once the rockets are up, who cares where they come down? That's not my department," says Wernher von Braun —TAL
      - shermantanktop 3y ago
        Tinder for combat: swipe to kill.
- jakeinspace 3y ago
  Morality-washing was always going to be one of the first uses of AI, in hiring and in policing and in war (or ethnic cleansing).
  - Qem 3y ago
    > Morality-washing was always going to be one of the first uses of AI They even made a cruel joke by naming the automated killing system "Lavender". Lavender flowers were used in clothes washing since ancient times. The name of the flower cames from the Latin word for washing. Portuguese inherited it, for example, with the word for lavender, "lavanda", sounding similar to the verb to wash, "lavar".
- michelsedgh 3y ago
  Using one of the most biased and anti-israel as a source what could go wrong? Edit: A different and less biased title would be: the use of technology and AI to minimize civilian casualties in guerrilla warfare. No? How much civilians did US kill during Vietnam war? There was no AI then and the Vietnamese didnt murder and rape civilian Americans did they?Edit 2: downvote me until it gets removed and you enjoy your echo chamber of beliefs.
  - gambiting 3y ago
    >>Using one of the most biased and anti-israel as a source what could go wrong? Which one exactly? The guardian article quotes literally hundreds of sources - which ones are you bothered by? >>the use of technology and AI to minimize civilian casualties in guerrilla warfare. It achieves literally no such thing, it's dishonest to suggest otherwise. >>How much civilians did US kill during Vietnam war? Peak whataboutism. As if everyone on the internet is American or believes that we should be comparing everyone else to America.
  - catlover76 3y ago
    [dead]
  - roughly 3y ago
    > the use of technology and AI to minimize civilian casualties in guerrilla warfare The article makes it quite clear this is not what’s happening and not why the systems are in use. They’re being used to generate a target list faster than could be done by traditional methods, and are being used despite agreeing with traditional methods only 90% of the time. They’re being used to accelerate the war, not make it more accurate.
  - ClumsyPilot 3y ago
    > use of technology and AI to minimize civilian casualties In the year 2022 Russia invaded ukraine, flattened two (?) cities, showed no concern for civilian life, committed war crimes. And yet Israel has killed more children in 6 months than Russia did in 3 years. Russia killed 4 western aid workers in 2022, Israel already more than that in 1 week, we are suppose to believe these are accidents?
- ethbr1 3y ago
  The original(?) +972 article was also on HN: https://news.ycombinator.com/item?id=39918245 https://news.ycombinator.com/item?id=39918245 It was an excellent and detailed piece on the various systems, and their use by Israel in the Gaza conflict. >> In an unprecedented move, according to two of the sources, the army also decided during the first weeks of the war that, for every junior Hamas operative that Lavender marked, it was permissible to kill up to 15 or 20 civilians; in the past, the military did not authorize any “collateral damage” during assassinations of low-ranking militants. Needs no comment and basically sums up the war.
  - alchemist1e9 3y ago
    The names of the systems are “The Gospel” and “Lavender”! How is it that a Jewish army is using Christian references for their AI killing systems? I don’t think that is unintentional.
    - permo-w 3y ago
      is "Lavender" a Christian reference?
      - readyplayernull 3y ago
        Lavender's history has biblical roots. It is referred to as Spikenard in the Bible. Mary used it on the infant Jesus and anointed Jesus after the crucifixion when she was preparing him for burial. So it has a "life and death" connotation which makes a lot of sense when you think of the system as a roulette for those chosen to die.
        alchemist1e9 3y ago
        Exactly. There is no way that whomever named these systems didn’t do it intentionally. The question is why? what was the motivation?
        readyplayernull 3y ago
- hammock 3y ago
  Fully autonomous drones, from other countries, have been killing people since at least 2020. It's not just the IDF using AI. This was the earliest official report sourced by the UN, pegging Turkey: https://www.fox5ny.com/news/fully-autonomous-drone-was-used-to-hunt-down-soldiers-un-says https://www.fox5ny.com/news/fully-autonomous-drone-was-used-...
  - readthenotes1 3y ago
    The US has killed plenty of non comabatants using human guided drones for decades...
    - hammock 3y ago
      In question are fully autonomous, artificial intelligence-based, unmanned drones
      - 082349872349872 3y ago
        Can't make them too artificially intelligent or they might draw conclusions: https://news.ycombinator.com/item?id=38362711 https://news.ycombinator.com/item?id=38362711
        tialaramex 3y ago
        The link is, indirectly to Peter Watts' story "Malak", which, yeah, is exactly what I think of whenever people talk about this.
- klabb3 3y ago
  ”Machines never make the decision, they’re merely collecting data to assist human operatives” They’ll keep saying this forever, and there’s a shroud of plausible deniability to it! Not to mention it can soothe the conscience of the operatives who have to live with their decisions - they don’t even have to look through the family photos - the AI has already filtered out the suspicious parts. In reality, it’s a good ole recommendation engine, just like Netflix or TikTok. And when you’re tired, have too much work stacked up, you’ll be more likely to let autoplay take over – or approve the strike of someone’s home. To those that think this is harsh: please read the article. It’s that bad.
  - wahnfrieden 3y ago
    It's not a matter of being tired and having to catch up. The operators are explicitly instructed to treat the AI results as orders without questioning the results. In other words, operators are threatened with punishment if they take the time to inspect the results more closely before following orders. It's not even an option! > In order to speed up target elimination, *soldiers have been ordered to treat Lavender-generated targets as an order, rather than something to be independently checked*, the investigation found. Soldiers in charge of manually checking targets spent only seconds on each person, sources said, merely to make sure the target was a man. Children are also considered legitimate targets by Lavender.
    - klabb3 3y ago
      Right! The extreme point of laziness/stress is just pressing approve. At which point the machine is making the de-facto decision. In either case, the role of the recommendation engine is immensely impactful, as we already know from consumer products. But here the software engineers are directly involved in life-and-death decisions at scale. I really hope they know that.
      - wahnfrieden 3y ago
        They know that because it was designed to increase civilian casualties by unprecedented civilian:"combatant" ratios and confidence levels. These were design requirements, not faults. The notable element of this news is not that the decision-making was automated. It's that they set the civilian collateral death ratio to ~100 (and other details such as considering children to be valid combatant targets), regardless of whether the process that arrives at that was automated or not.
ziddoap 3y ago
>It creates the possibility of atrocity with no accountability. As gross and unfortunate as it is, this is a selling feature to some governments.
- westmeal 3y ago
  Hey at least Palantir says their death machine is ethical because it shows you a flowchart of a bunch of widgets. Do other weapons have widgets?!? I didn't think so!
- rany_ 3y ago
  Do you think this holds up in court? It gives me Nuremberg defense vibes ("I was just following orders"), except humans are following an AI's orders.
  - Sharlin 3y ago
    That's why you work to remove humans from the loop altogether.
    - postmodest 3y ago
      Man, Fuck Ted Faro.
    - rany_ 3y ago
      That's terrifying, I hate where this is all headed. I still hope that the people that ordered it setup and those that had it developed would be held accountable (I know, pipe dream).
    - salawat 3y ago
      ...I'm having trouble discerning whether you understand the abject stupidity of that suggestion, or if you're just being sarcastic. Poe's law strikes again.
      - rany_ 3y ago
        To be honest, I could see this happening. What's going on right now is absurd enough as it is. I pity The Onion.
Zambyte 3y ago
> It creates the possibility of atrocity with no accountability. No it doesn't. If you use any tool to kill people (ie a gun, bomb, computer) the person (or people) who used the tool are accountable.
- cmrdporcupine 3y ago
  What it does is create more levels of indirection and detachment.
  - Cthulhu_ 3y ago
    Indeed, they gave each hit 20 seconds - if that - to manually verify. That said, airstrikes, drones etc already add a layer of indirection and detachment. The thing that this "AI" tool allowed was to mark targets faster, or if you will, with less scrutiny, increasing the efficiency and speed of the killing machine.
    - cmrdporcupine 3y ago
      We've been creating indirection ever since the first homo ancestor carved/fashioned a spear tip or arrow, or the first metallurgist poured a bronze sword. But. There's matters of degrees. And this is just one or two steps away from full autonomous murder. Reminds me of this section Frank Herbert's God Emperor of Dune: ... She would carry with her forever afterward the clear sights and sounds and smells. The seeking machines would be there, the smell of blood and entrails, the cowering humans in their burrows, aware only that they could not escape . . . while all the time the mechanical movement approached, nearer and nearer and nearer ...louder...louder! ...like the terrible machines of that apocalyptic vision, the predator could follow any creature who left tracks.*
  - shermantanktop 3y ago
    Indirectly, the voters who put a warmongering leader into office are accountable…but when a war goes wrong, they never blame themselves. Cynically, it’s not who is actually responsible and by how much, it’s who is forced to take the blame.
michelsedgh 3y ago
[flagged]
- frob 3y ago
  Yes, we Americans committed many atrocities in our many wars and we will absolutely do so in the future. Please learn from us instead of following our path.
  - michelsedgh 3y ago
    They are not doing the same as America are they? They are not perfect but it just depends who you trust as a source in this war. All the stats either come from Hamas or IDF. People choose to believe the Hamas statistics all day and take a blind eye to what IDF is doing. Also big contrast is they are defending themselves. Im not talking only about Oct 7. Even before that Hamas sent bombs for many years before that. Imagine going to work and seeing bombs above ur head and all the world saying it is what it is you can’t attack them back or do anything about it.
    - frob 3y ago
      War crimes do not justify war crimes.
    - gambiting 3y ago
      >>Also big contrast is they are defending themselves. Yes, I'm sure Israel shooting all of those kids in Gaza is just self defense - after all, they can't grow up to become terrorists if they are dead already. https://www.theguardian.com/commentisfree/2024/mar/23/israel-killing-children-west-bank https://www.theguardian.com/commentisfree/2024/mar/23/israel... https://www.theguardian.com/world/2024/apr/02/gaza-palestinian-children-killed-idf-israel-war https://www.theguardian.com/world/2024/apr/02/gaza-palestini... Like, I know this is an extreme example, but this isn't anywhere near as black and white as you present. Israel has done a lot of shit to Gaza over many many many decades, and while it doesn't justify terrorist acts towards Israel, many people could argue that Palestinians are also defending themselves in many ways.
mlyle 3y ago
> it's horrifying and categorically unethical to use them to decide who to kill. Absolute statements are hard. I mean, I agree with the sentiment, but I can't get behind such a strong statement. If you have a 10 person group deciding who to kill, and reduce their workload with AI so that the 10 person group can take more time double checking-- that's "using AI to decide who to kill" but probably improves outcomes. The problem is, in practice, we know that people are just going to punt responsibility to the machine and shrug.
- ethbr1 3y ago
  The fundamental reality is that compressing decision timelines creates more effective military outcomes. Everything else flows from that. At a high-level, that decision compression will be an arms race between competitors for military advantage, where the argument to relax human-scale decisions will be "If we don't do it, we'll lose to someone who will." You saw the same thing in the nuclear arms race, as launch-to-impact time compression (as technology advanced) eroded available human decision time.
  - mlyle 3y ago
    > The fundamental reality is that compressing decision timelines creates more effective military outcomes. But popping off half-cocked often results in bad outcomes, so there's a tradeoff between decision quality and decision time. In theory, having AI in the loop could improve both. In practice, I suspect you're right.
    - ethbr1 3y ago
      Unfortunately, the negative impact of killing civilians currently seems to be... minimal. And maybe always has been, internationally-speaking? It would be nice to reform UN and supra-national rules of warfare in light of modern developments. Even a "this is an acceptable ratio of civilian collateral deaths, above which constitutes a war crime" would be helpful for curtailing nation's baser urges.
      - 3y ago
whyage 3y ago
Military technology has been helping people decide who to kill for ages. For example, long distance cameras and night vision devices use extensive image processing. They're designed to improve visibility but can undoubtedly lead to false positives in the heat of the battle.
Kalium 3y ago
> Also. No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. It creates the possibility of atrocity with no accountability. How much technology is it ethical to use in the decision-making loop for kinetic decisions? I don't think there's an easy answer here that actually engages meaningfully with the question.
kspacewalk2 3y ago
He didn't advocate for fully anonymous AI systems, but ones where humans will ultimately make the decision. Obviously these can be implemented in a variety of ways, from basically data mining software and some neat dashboards to help a bunch of humans decide on a target in a responsible way, to something where all the boring bits are hidden, a list of names is oh-so-helpfully drawn up and a rubber-stamp from an under-trained early-20s "intelligence officer" is all that's needed to kill someone, and their family, in their home.
tomp 3y ago
AI is a tool. It can be used to kill more people, and it can be used to kill less people. Imagine if the superweapon US developed during WWII was AI. They'd use it to deliver devastating destruction to military factories in Nagasaki and Hiroshima, saving 100k+ civilian lives.
thinkingtoilet 3y ago
>It creates the possibility of atrocity with no accountability. It most certainly does not. And comments and thinking like this enable that. There is a human who is responsible for deciding to use the AI, that human is responsible. If they can't handle the responsibility, then don't use AI.
nimbius 3y ago
its true. Whenever you're in a sensitive position and must practice opsec hygene, you need to be consistent and get it right every time. Your adversary only needs to get lucky once.
alephknoll 3y ago
> No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. Is it any more horrifying or unethical to decide without AI? > It creates the possibility of atrocity with no accountability. Who was held accountable for hiroshima? Or do you know how much atrocities were committed to create israel? You act like we had accountability before AI. We've never had accountability for atrocities. Just victor's 'justice'. AI can be better than humans when it comes to atrocities. It can be as bad as humans, But it can never be worse. Humans set the standard.
- uluyol 3y ago
  > Is it any more horrifying or unethical to decide without AI? It is harder to get 1000 people to participate in an unethical military campaign than it is to get 10 people. So if you can make those 10 people 100x as efficient...
  - alephknoll 3y ago
    > It is harder to get 1000 people to participate in an unethical military campaign than it is to get 10 people. No. It's easier to get 1000000 to participate than 1000. But that's not the point. We are talking about accountability. I'd say it's easier to hold 10 people accountable than a 1000.
deleted 3y ago
[deleted]
ghostway 3y ago
> Although the fact that this guy was anonymous to start with definitely indicates that accountability isn't the goal here. Well, it isn't. This is a strategic choice, and one pretty much every military took
namaria 3y ago
That's basically what AI as a concept is. Smoke as screens to hide responsibility and costs.
mapt 3y ago
> Also. No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. It creates the possibility of atrocity with no accountability. No it doesn't. That was always a possibility. Atrocity with no accountability long predates AI, and the ethics of "using them to decide who to kill" are isomorphic with the ethics of using a dice roll to decide who to kill. In this case, the people you could choose to hold accountable is everybody who planned, built, or activated the machine made of a Wacky Waving Inflatable Tube Man mounting a chaingun, designed to spray bullets indiscriminately in all directions. The fact that no human trigger finger is actually on the trigger does not AT ALL defuse ethical considerations, does not exculpate anybody. We kill with machines in every military, but it's human decisions all the way down no matter how long ago a human last physically touched the munition. The lack of accountability starts to occur when a journalist embraces headlines like "AI Made Decisions About Who To Target", rather than using scarequotes and agency - "IDF-run AI system 'Made Decisions' about who to target". The IDF are the actors in this construction, the AI system is just a tool. The lack of accountability continues in the political sphere where no war crime investigation takes place, no red line is instituted, no Geopolitical Consequences arise. It finalizes in the polity when no liberal / humanist vigilante takes revenge into their own hands, no 4chah/kiwifarms investigation doxes their family, no airline pilot diverts to the Hague, we just sort of embrace treating atrocity-committers humanely because we're lazy cowards.

by lukev 3y ago

roughly 3y ago

The Guardian reported more on the IDF’s use of AI a couple days back, and it’s grim: https://www.theguardian.com/world/2024/apr/03/israel-gaza-ai-database-hamas-airstrikes https://www.theguardian.com/world/2024/apr/03/israel-gaza-ai... Although in that article, most of the horrors are coming from the human operators - there’s what comes off as a pretty strong disinterest in finding reasons to distrust the machine.

082349872349872 3y ago
There must be a line somewhere between "pretty strong disinterest in finding reasons to distrust the machine" and "using the machine as plausible deniability for doing what you hoped to do anyway"; is there a relatively simple description of this line, or is it a fractal boundary?
- roughly 3y ago
  Yeah, I phrased that as generously as possible because I didn’t really want to kick that particular hornets’ nest, but the article does a fine job making that point.
  - aebtebeten 3y ago
    Very charitable of you! (do we need an emoticon for charity, like we have for sarcasm? symmetry would suggest yes.) Upon further reflection, it occurs to me that these situations are exactly why the Geneva Conventions[0] require combatants to have (a) uniforms, and (b) a command structure: (a) prevents someone low from disingenuously stopping the buck by claiming "tweren't no orders; I'm acting on my own" (b) prevents someone high from arguing as Uber does: "you see, we're not a traditional army that gives orders to our fighters — we just provide a service that matches attackers and targets online, and really, anything bad that allegedly happened was only between those two parties"[1] [0] if we're not doing these anymore, just go ahead and tell me "the future is now, old man", and I'll take my mutterings to the pétanque piste. [1] > "Once the rockets are up, who cares where they come down? That's not my department," says Wernher von Braun —TAL
    - shermantanktop 3y ago
      Tinder for combat: swipe to kill.
jakeinspace 3y ago
Morality-washing was always going to be one of the first uses of AI, in hiring and in policing and in war (or ethnic cleansing).
- Qem 3y ago
  > Morality-washing was always going to be one of the first uses of AI They even made a cruel joke by naming the automated killing system "Lavender". Lavender flowers were used in clothes washing since ancient times. The name of the flower cames from the Latin word for washing. Portuguese inherited it, for example, with the word for lavender, "lavanda", sounding similar to the verb to wash, "lavar".
michelsedgh 3y ago
Using one of the most biased and anti-israel as a source what could go wrong? Edit: A different and less biased title would be: the use of technology and AI to minimize civilian casualties in guerrilla warfare. No? How much civilians did US kill during Vietnam war? There was no AI then and the Vietnamese didnt murder and rape civilian Americans did they?Edit 2: downvote me until it gets removed and you enjoy your echo chamber of beliefs.
- gambiting 3y ago
  >>Using one of the most biased and anti-israel as a source what could go wrong? Which one exactly? The guardian article quotes literally hundreds of sources - which ones are you bothered by? >>the use of technology and AI to minimize civilian casualties in guerrilla warfare. It achieves literally no such thing, it's dishonest to suggest otherwise. >>How much civilians did US kill during Vietnam war? Peak whataboutism. As if everyone on the internet is American or believes that we should be comparing everyone else to America.
- catlover76 3y ago
  [dead]
- roughly 3y ago
  > the use of technology and AI to minimize civilian casualties in guerrilla warfare The article makes it quite clear this is not what’s happening and not why the systems are in use. They’re being used to generate a target list faster than could be done by traditional methods, and are being used despite agreeing with traditional methods only 90% of the time. They’re being used to accelerate the war, not make it more accurate.
- ClumsyPilot 3y ago
  > use of technology and AI to minimize civilian casualties In the year 2022 Russia invaded ukraine, flattened two (?) cities, showed no concern for civilian life, committed war crimes. And yet Israel has killed more children in 6 months than Russia did in 3 years. Russia killed 4 western aid workers in 2022, Israel already more than that in 1 week, we are suppose to believe these are accidents?
ethbr1 3y ago
The original(?) +972 article was also on HN: https://news.ycombinator.com/item?id=39918245 https://news.ycombinator.com/item?id=39918245 It was an excellent and detailed piece on the various systems, and their use by Israel in the Gaza conflict. >> In an unprecedented move, according to two of the sources, the army also decided during the first weeks of the war that, for every junior Hamas operative that Lavender marked, it was permissible to kill up to 15 or 20 civilians; in the past, the military did not authorize any “collateral damage” during assassinations of low-ranking militants. Needs no comment and basically sums up the war.
- alchemist1e9 3y ago
  The names of the systems are “The Gospel” and “Lavender”! How is it that a Jewish army is using Christian references for their AI killing systems? I don’t think that is unintentional.
  - permo-w 3y ago
    is "Lavender" a Christian reference?
    - readyplayernull 3y ago
      Lavender's history has biblical roots. It is referred to as Spikenard in the Bible. Mary used it on the infant Jesus and anointed Jesus after the crucifixion when she was preparing him for burial. So it has a "life and death" connotation which makes a lot of sense when you think of the system as a roulette for those chosen to die.
      - alchemist1e9 3y ago
        Exactly. There is no way that whomever named these systems didn’t do it intentionally. The question is why? what was the motivation?
        readyplayernull 3y ago
hammock 3y ago
Fully autonomous drones, from other countries, have been killing people since at least 2020. It's not just the IDF using AI. This was the earliest official report sourced by the UN, pegging Turkey: https://www.fox5ny.com/news/fully-autonomous-drone-was-used-to-hunt-down-soldiers-un-says https://www.fox5ny.com/news/fully-autonomous-drone-was-used-...
- readthenotes1 3y ago
  The US has killed plenty of non comabatants using human guided drones for decades...
  - hammock 3y ago
    In question are fully autonomous, artificial intelligence-based, unmanned drones
    - 082349872349872 3y ago
      Can't make them too artificially intelligent or they might draw conclusions: https://news.ycombinator.com/item?id=38362711 https://news.ycombinator.com/item?id=38362711
      - tialaramex 3y ago
        The link is, indirectly to Peter Watts' story "Malak", which, yeah, is exactly what I think of whenever people talk about this.
klabb3 3y ago
”Machines never make the decision, they’re merely collecting data to assist human operatives” They’ll keep saying this forever, and there’s a shroud of plausible deniability to it! Not to mention it can soothe the conscience of the operatives who have to live with their decisions - they don’t even have to look through the family photos - the AI has already filtered out the suspicious parts. In reality, it’s a good ole recommendation engine, just like Netflix or TikTok. And when you’re tired, have too much work stacked up, you’ll be more likely to let autoplay take over – or approve the strike of someone’s home. To those that think this is harsh: please read the article. It’s that bad.
- wahnfrieden 3y ago
  It's not a matter of being tired and having to catch up. The operators are explicitly instructed to treat the AI results as orders without questioning the results. In other words, operators are threatened with punishment if they take the time to inspect the results more closely before following orders. It's not even an option! > In order to speed up target elimination, *soldiers have been ordered to treat Lavender-generated targets as an order, rather than something to be independently checked*, the investigation found. Soldiers in charge of manually checking targets spent only seconds on each person, sources said, merely to make sure the target was a man. Children are also considered legitimate targets by Lavender.
  - klabb3 3y ago
    Right! The extreme point of laziness/stress is just pressing approve. At which point the machine is making the de-facto decision. In either case, the role of the recommendation engine is immensely impactful, as we already know from consumer products. But here the software engineers are directly involved in life-and-death decisions at scale. I really hope they know that.
    - wahnfrieden 3y ago
      They know that because it was designed to increase civilian casualties by unprecedented civilian:"combatant" ratios and confidence levels. These were design requirements, not faults. The notable element of this news is not that the decision-making was automated. It's that they set the civilian collateral death ratio to ~100 (and other details such as considering children to be valid combatant targets), regardless of whether the process that arrives at that was automated or not.

ziddoap 3y ago

>It creates the possibility of atrocity with no accountability. As gross and unfortunate as it is, this is a selling feature to some governments.

westmeal 3y ago
Hey at least Palantir says their death machine is ethical because it shows you a flowchart of a bunch of widgets. Do other weapons have widgets?!? I didn't think so!
rany_ 3y ago
Do you think this holds up in court? It gives me Nuremberg defense vibes ("I was just following orders"), except humans are following an AI's orders.
- Sharlin 3y ago
  That's why you work to remove humans from the loop altogether.
  - postmodest 3y ago
    Man, Fuck Ted Faro.
  - rany_ 3y ago
    That's terrifying, I hate where this is all headed. I still hope that the people that ordered it setup and those that had it developed would be held accountable (I know, pipe dream).
  - salawat 3y ago
    ...I'm having trouble discerning whether you understand the abject stupidity of that suggestion, or if you're just being sarcastic. Poe's law strikes again.
    - rany_ 3y ago
      To be honest, I could see this happening. What's going on right now is absurd enough as it is. I pity The Onion.

Zambyte 3y ago

> It creates the possibility of atrocity with no accountability. No it doesn't. If you use any tool to kill people (ie a gun, bomb, computer) the person (or people) who used the tool are accountable.

cmrdporcupine 3y ago
What it does is create more levels of indirection and detachment.
- Cthulhu_ 3y ago
  Indeed, they gave each hit 20 seconds - if that - to manually verify. That said, airstrikes, drones etc already add a layer of indirection and detachment. The thing that this "AI" tool allowed was to mark targets faster, or if you will, with less scrutiny, increasing the efficiency and speed of the killing machine.
  - cmrdporcupine 3y ago
    We've been creating indirection ever since the first homo ancestor carved/fashioned a spear tip or arrow, or the first metallurgist poured a bronze sword. But. There's matters of degrees. And this is just one or two steps away from full autonomous murder. Reminds me of this section Frank Herbert's God Emperor of Dune: ... She would carry with her forever afterward the clear sights and sounds and smells. The seeking machines would be there, the smell of blood and entrails, the cowering humans in their burrows, aware only that they could not escape . . . while all the time the mechanical movement approached, nearer and nearer and nearer ...louder...louder! ...like the terrible machines of that apocalyptic vision, the predator could follow any creature who left tracks.*
- shermantanktop 3y ago
  Indirectly, the voters who put a warmongering leader into office are accountable…but when a war goes wrong, they never blame themselves. Cynically, it’s not who is actually responsible and by how much, it’s who is forced to take the blame.

michelsedgh 3y ago

[flagged]

frob 3y ago
Yes, we Americans committed many atrocities in our many wars and we will absolutely do so in the future. Please learn from us instead of following our path.
- michelsedgh 3y ago
  They are not doing the same as America are they? They are not perfect but it just depends who you trust as a source in this war. All the stats either come from Hamas or IDF. People choose to believe the Hamas statistics all day and take a blind eye to what IDF is doing. Also big contrast is they are defending themselves. Im not talking only about Oct 7. Even before that Hamas sent bombs for many years before that. Imagine going to work and seeing bombs above ur head and all the world saying it is what it is you can’t attack them back or do anything about it.
  - frob 3y ago
    War crimes do not justify war crimes.
  - gambiting 3y ago
    >>Also big contrast is they are defending themselves. Yes, I'm sure Israel shooting all of those kids in Gaza is just self defense - after all, they can't grow up to become terrorists if they are dead already. https://www.theguardian.com/commentisfree/2024/mar/23/israel-killing-children-west-bank https://www.theguardian.com/commentisfree/2024/mar/23/israel... https://www.theguardian.com/world/2024/apr/02/gaza-palestinian-children-killed-idf-israel-war https://www.theguardian.com/world/2024/apr/02/gaza-palestini... Like, I know this is an extreme example, but this isn't anywhere near as black and white as you present. Israel has done a lot of shit to Gaza over many many many decades, and while it doesn't justify terrorist acts towards Israel, many people could argue that Palestinians are also defending themselves in many ways.

mlyle 3y ago

> it's horrifying and categorically unethical to use them to decide who to kill. Absolute statements are hard. I mean, I agree with the sentiment, but I can't get behind such a strong statement. If you have a 10 person group deciding who to kill, and reduce their workload with AI so that the 10 person group can take more time double checking-- that's "using AI to decide who to kill" but probably improves outcomes. The problem is, in practice, we know that people are just going to punt responsibility to the machine and shrug.

ethbr1 3y ago
The fundamental reality is that compressing decision timelines creates more effective military outcomes. Everything else flows from that. At a high-level, that decision compression will be an arms race between competitors for military advantage, where the argument to relax human-scale decisions will be "If we don't do it, we'll lose to someone who will." You saw the same thing in the nuclear arms race, as launch-to-impact time compression (as technology advanced) eroded available human decision time.
- mlyle 3y ago
  > The fundamental reality is that compressing decision timelines creates more effective military outcomes. But popping off half-cocked often results in bad outcomes, so there's a tradeoff between decision quality and decision time. In theory, having AI in the loop could improve both. In practice, I suspect you're right.
  - ethbr1 3y ago
    Unfortunately, the negative impact of killing civilians currently seems to be... minimal. And maybe always has been, internationally-speaking? It would be nice to reform UN and supra-national rules of warfare in light of modern developments. Even a "this is an acceptable ratio of civilian collateral deaths, above which constitutes a war crime" would be helpful for curtailing nation's baser urges.
    - 3y ago

whyage 3y ago

Military technology has been helping people decide who to kill for ages. For example, long distance cameras and night vision devices use extensive image processing. They're designed to improve visibility but can undoubtedly lead to false positives in the heat of the battle.

Kalium 3y ago

> Also. No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. It creates the possibility of atrocity with no accountability. How much technology is it ethical to use in the decision-making loop for kinetic decisions? I don't think there's an easy answer here that actually engages meaningfully with the question.

kspacewalk2 3y ago

He didn't advocate for fully anonymous AI systems, but ones where humans will ultimately make the decision. Obviously these can be implemented in a variety of ways, from basically data mining software and some neat dashboards to help a bunch of humans decide on a target in a responsible way, to something where all the boring bits are hidden, a list of names is oh-so-helpfully drawn up and a rubber-stamp from an under-trained early-20s "intelligence officer" is all that's needed to kill someone, and their family, in their home.

tomp 3y ago

AI is a tool. It can be used to kill more people, and it can be used to kill less people. Imagine if the superweapon US developed during WWII was AI. They'd use it to deliver devastating destruction to military factories in Nagasaki and Hiroshima, saving 100k+ civilian lives.

thinkingtoilet 3y ago

>It creates the possibility of atrocity with no accountability. It most certainly does not. And comments and thinking like this enable that. There is a human who is responsible for deciding to use the AI, that human is responsible. If they can't handle the responsibility, then don't use AI.

nimbius 3y ago

its true. Whenever you're in a sensitive position and must practice opsec hygene, you need to be consistent and get it right every time. Your adversary only needs to get lucky once.

alephknoll 3y ago

> No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. Is it any more horrifying or unethical to decide without AI? > It creates the possibility of atrocity with no accountability. Who was held accountable for hiroshima? Or do you know how much atrocities were committed to create israel? You act like we had accountability before AI. We've never had accountability for atrocities. Just victor's 'justice'. AI can be better than humans when it comes to atrocities. It can be as bad as humans, But it can never be worse. Humans set the standard.

uluyol 3y ago
> Is it any more horrifying or unethical to decide without AI? It is harder to get 1000 people to participate in an unethical military campaign than it is to get 10 people. So if you can make those 10 people 100x as efficient...
- alephknoll 3y ago
  > It is harder to get 1000 people to participate in an unethical military campaign than it is to get 10 people. No. It's easier to get 1000000 to participate than 1000. But that's not the point. We are talking about accountability. I'd say it's easier to hold 10 people accountable than a 1000.

deleted 3y ago

[deleted]

ghostway 3y ago

> Although the fact that this guy was anonymous to start with definitely indicates that accountability isn't the goal here. Well, it isn't. This is a strategic choice, and one pretty much every military took

namaria 3y ago

That's basically what AI as a concept is. Smoke as screens to hide responsibility and costs.

mapt 3y ago

> Also. No matter how bullish you are on the potential of AI, it's horrifying and categorically unethical to use them to decide who to kill. It creates the possibility of atrocity with no accountability. No it doesn't. That was always a possibility. Atrocity with no accountability long predates AI, and the ethics of "using them to decide who to kill" are isomorphic with the ethics of using a dice roll to decide who to kill. In this case, the people you could choose to hold accountable is everybody who planned, built, or activated the machine made of a Wacky Waving Inflatable Tube Man mounting a chaingun, designed to spray bullets indiscriminately in all directions. The fact that no human trigger finger is actually on the trigger does not AT ALL defuse ethical considerations, does not exculpate anybody. We kill with machines in every military, but it's human decisions all the way down no matter how long ago a human last physically touched the munition. The lack of accountability starts to occur when a journalist embraces headlines like "AI Made Decisions About Who To Target", rather than using scarequotes and agency - "IDF-run AI system 'Made Decisions' about who to target". The IDF are the actors in this construction, the AI system is just a tool. The lack of accountability continues in the political sphere where no war crime investigation takes place, no red line is instituted, no Geopolitical Consequences arise. It finalizes in the polity when no liberal / humanist vigilante takes revenge into their own hands, no 4chah/kiwifarms investigation doxes their family, no airline pilot diverts to the Hague, we just sort of embrace treating atrocity-committers humanely because we're lazy cowards.