6 ms·

I really don’t get all the fretting about drones making kill-no-kill decisions. You’re telling me that you’d rather this was done by a stressed, emotional, tir

• jjice 1y ago
  This issue is too complex for me to really have a good thought at the moment, but I will say that this reminds me of that classic story of the Soviet who didn't launch the nukes during a threat came in from the US. The threat ended up being a false alarm, and that human judgement saved a lot of lives. A machine, assuming it would have launched when seeing that signal, would've ended differently.
  • nthingtohide 1y ago
    Imagine an AI system is programmed to poison the well incase of losing scenario. So it takes advantage of Kessler System to make sure no one else has advantage if it doesn't too poisoning the whole of earth orbits for centuries to come in which satellites could reside.
  • perihelions 1y ago
    In 1962, the US sent out a valid nuclear launch order with correct launch codes, by mistake. https://www.vice.com/en/article/former-us-airman-tells-un-an-accidental-nuclear-war-was-narrowly-avoided-in-1962/ https://www.vice.com/en/article/former-us-airman-tells-un-an... ("Former US Airman Tells UN an Accidental Nuclear War Was Narrowly Avoided in 1962") https://news.ycombinator.com/item?id=10452983 https://news.ycombinator.com/item?id=10452983 ("The Okinawa missiles of October 1962 (thebulletin.org)", 71 comments)
  • netsharc 1y ago
    https://en.wikipedia.org/wiki/Stanislav_Petrov https://en.wikipedia.org/wiki/Stanislav_Petrov
• sorcerer-mar 1y ago
  The thing you're missing is that the decision isn't actually made by a machine. It's made by a person much much higher up across a much broader effective surface area. That is, historically to wage war you had to ultimately convince/coerce millions of people to wage that war for you -- continuously for years -- along with millions more to support the effort politically and economically. In the highly automated world, you literally only need a lot of money (which also tends to concentrate outside of any semblance of democratic control).
  • orbital-decay 1y ago
    The amount of autonomy and decision complexity matters a lot. Autonomously making its way into a known area, recognizing the target and self-correcting to it is basically how every modern cruise missile works.
    • netsharc 1y ago
      And now we know making an ersatz cruise missile in the garage is possible...
      • AnimalMuppet 1y ago
        True, though (so far) range, velocity, and payload are far smaller than what we expect for a "cruise missile".
        • falcor84 1y ago
          Well, this week proves that range and velocity limitation can be overcome if you ship them near to their destination in small boxes ahead of time. This wasn't really an option with cruise missiles.
          • orbital-decay 1y ago
            https://en.wikipedia.org/wiki/Kalibr_(missile_family)#Club-K https://en.wikipedia.org/wiki/Kalibr_(missile_family)#Club-K
• user568439 1y ago
  The stressed private might still have a bit of empathy and humanity. Meanwhile millions of drones can be programmed (or hacked) to kill millions of people without excluding civilians or anyone
  • yaris 1y ago
    My understanding is that a drone will make decisions that are reproducible (same data - same decision), so if anything goes wrong then it should be possible to investigate (to some extent) and fix. A stressed private is in this sense ”undebuggable” because much more not-easily-reproducible factors influence decisions. Also I’m afraid that stressed and tired privates at war tend to err towards ”just kill them all” because it looks much more like a videogame.
    • lukan 1y ago
      The data is never the same. Every situation in war (or reality in general) is unique.
  • alkonaut 1y ago
    We have had weapons which are autonomous for decades. You launch them consciously then you know that it will find and destroy weapons based on some "intelligence" (A homing missile with a radar you know is likely to hit the thing that reflects the radar waves, whatever that is. There are artillery shells which home in on vehicles and so on). The launch decision by the human means "I'm responsible for this thing hitting and the thing that it finds". The kill/no-kill decision is made at launch time. An AA missile might hit a civilan jet, but there is no way the operator will make a new kill/no-kill decision once it reaches the jet. You made the decision at launch. That's the same with these drones. The smarter they get, the further away the human goes. Today it might be simple to create autonomous weapons who are instructed to kill vehicles matching various known appearances. That too already exists. The strike on the Russian bombers was reportedly carried out manually, but it would have been pretty easy to have that autonomous, since the targets are huge, stationary, easily recognizable and easy to navigate to in the geography. If you launch a quadcopter and instruct it to kill any adult human it finds, then that's the same thing. You wouldn't launch it into an area where there is a remote possibility of being any civilians. No difference from firing an artillery shell. If there is a civilian, or a soldier waving a white flag or whatever - there is no cancel button for your artillery shell. The decision to kill whatever is in the other end was made when you fired it. There is literally no difference between firing a million drones and firing a million artillery shells down range. It's your human responsibility and your human consciousness when you make the decision. I don't think we have had widespread use of autonomous human-targeting drones yet, but it's by no means science fiction today. Just a matter of time. We'll see their use in this conflict.
• hoseja 1y ago
  This is flawed thinking. Human decisions scale worse - this is a feature.
• bell-cot 1y ago
  To be fair, that's a relatively buried and brief aspect in the article. Which is otherwise heavy on real-world military realities. Personally, I see anyone fretting about imperfect drones making those decisions as either (1) one of today's 10K, or (2) a performer covering a trending topic. Land mines were deployed at giga-scale over a century ago (WWI). Spring-guns were enough of a problem in England to be outlawed two centuries ago.
  • lukan 1y ago
    A) land mines are banned in most countries for that reason B) mines don't move by themself, but stay where you deployed them A moving autonomous killer drone has a potentially bigger effect if unleashed on the wrong area. Besides, war is usually fought in civilian areas. If you send the drone to the frontline, but its GPS is jammed, it might move somewhere close by.
    • bell-cot 1y ago
      A) Most countries also have laws/treaties banning invasions. But when those happen to them anyway, most countries start ignoring the bans on landmines. Which are quick/cheap/easy to produce, even with century-old tech. B) https://en.wikipedia.org/wiki/Naval_mine https://en.wikipedia.org/wiki/Naval_mine Not that anybody's doing Environmental Assessments on land, to determine whether the mines might be washed downstream after a heavy rain, or moved downslope by a landslide, or ... The problem you describe sounds very much like poorly-directed mortar/artillery fire & strategic bombing - which have been killing civilians at scale since at least WWI.
• basisword 1y ago
  >> I really don’t get all the fretting about drones making kill-no-kill decisions. >> You’re telling me that you’d rather this was done by a stressed, emotional, tired front-line private? I want somebody that can be held accountable when it goes wrong. With automated systems we don't have that.
  • eviks 1y ago
    That's not an issue, you can always hold someone accountable, from those who own to those who program.
    • kelipso 1y ago
      Is the person who presses a button that releases an autonomous kill-no-kill drone into a country going to jail when that drone razes an entire town of civilians? I doubt it. “Mistakes happen” etc. The visceral reaction is not there that would motivate the legal system to send the group that started a cascade of autonomous decisions by a robot to kill a bunch of civilians.
      • eviks 1y ago
        That depends on those two imaginary countries. Also, are you unaware of the many instances where people faced no accountability even when it wasn't a mistake, but deliberate?
        • kelipso 1y ago
          This would make the accountability situation much worse.
    • basisword 1y ago
      We can't even figure this out for self-driving nevermind warfare.
• JohnBooty 1y ago
  Can we think slightly more broadly? Targeting a stationary Tu-95 bomber with no protective measures in place is probably the easiest possible identification task for a drone. A lot of kill/no-kill decisions are more subtle, or involve unknowns, possible nearby civilians, etc. Or look at this crazy FPV piloting job. You think AI could do this? Pilot maneuvers through an absolute maze of anti-drone nets to hit a moving truck. (essentially SFW; video terminates slightly before impact) https://www.reddit.com/r/CombatFootage/comments/1l2aqxp/ukrainian_fiber_optic_drone_flew_through_a_gap_in/ https://www.reddit.com/r/CombatFootage/comments/1l2aqxp/ukra... stressed, emotional, tired front-line private Drone operators are generally not on the front lines, and sometimes they are literally on the other side of the world sitting in a cubicle. They are usually specialists of some sort, not untrained privates.
  • CamperBob2 1y ago
    Why couldn't AI do that? It's better than any human at this stuff.
    • GuinansEyebrows 1y ago
      Sometimes I feel like the entire AI field just never watched The Terminator.
      • CamperBob2 1y ago
        Whether it should is a different question. It certainly can.
        • deleted 1y ago
          [deleted]
      • ThrowawayR2 1y ago
        Campy, low budget action movies are not a sound basis for forming public policy.
    • JohnBooty 1y ago
• bad_haircut72 1y ago
  I was once talking to an F35 pilot about what its like and during the conversation he said something, "If its red, its dead" referring to how the computer classifies things as friend/foe for you. I realised it doesnt mean much to have a human in the loop when theyre making their decisions based completely on info the computer serves up anyway.
  • dralley 1y ago
    Indeed - in a hostile situation you're never going to get an eyeball on most hostile air targets.
• cheeseomlit 1y ago
  It doesn't take much imagination to see what's so disturbing about it, they're swarms of autonomous killing machines. It can't be bargained with. It can't be reasoned with. It doesn't feel pity or remorse, or fear. And it absolutely will not stop, ever, until you are dead
• scott_w 1y ago
  The issue is that software making these decisions can't possibly have the full context when making the decision because it was coded long before the decision point occurred. What does that mean in practice? Have you ever saw a flash in the corner of your eye, looked for a second, then realised it was just a reflection? For that split second, your brain identified a potential threat and was trying to quickly decide whether to hide, fight or run. But underneath that was a nagging of "it's probably nothing," that caused you to delay pulling out your gun and opening fire on a passing car. A computer programme may or may not have this coded in. A machine designed for an active war-zone, or a car assuming it'll only ever auto-pilot on a motorway, is used in a suburban area. That same programme that's fine in its original intended location suddenly opens fire on a grandma with a shiny stroller, or swerves into a pedestrian because "pedestrians don't walk on motorways." That's what we worry about.
  • pjc50 1y ago
    > Have you ever saw a flash in the corner of your eye, looked for a second, then realised it was just a reflection? For that split second, your brain identified a potential threat and was trying to quickly decide whether to hide, fight or run. But underneath that was a nagging of "it's probably nothing," that caused you to delay pulling out your gun and opening fire on a passing car. .. or vice-versa. The average war has plenty of stressed people firing semi blindly at half-identified shapes. And if we want to be literal, there's the case of Lee Clegg and the exact circumstances in which it is legal or not to open fire on a passing car and kill a teenage girl.
    • scott_w 1y ago
      I’m not denying that awful shit like this happens with humans. I’m pointing out the moral hazard in turning it into a preprogrammed system. We’ve seen software bugs empty bank accounts simply because running that faulty system for 5 minutes iterates 100,000 times. Now give the power to decide who lives and dies and it’s only limited by the ammo it holds.
• _verandaguy 1y ago
  I think this oversimplifies the matter. At a micro level, sure, you could argue that kill/no-kill decisions are being made by frontline enlisted soldiers, but the stress, the exhaustion, and being able to see your enemy downrange add a degree of humanization and discretion. Wars are brutal. Some warring factions are incredibly brutal. But taking a human life can't be reduced to a `KILL? [Y/N]` decision made by some MAJ somewhere. Killing enemy soldiers is a last resort (formally, this is the concept of _military necessity_), and normalization of it as anything else is a mistake. I also think it's a fallacy to compare this to the self-driving car trolley problem; the timeline of decisions, length of the decision chain, and ultimate goal of the decision-making process are _aggressively_ different. My overarching points are, I guess: War is a tragedy, obviously, but I don't think there's a way to avoid it right now. In absence of some way to stop war forever (hah), we can't trivialize taking human lives. It's not lost on me that this has been happening for a while at this point, and that it's getting worse. I still oppose it, and I think we all have a responsibility to be more critical of this regime of warfare. I don't know how to counter the argument of "well, if we don't do it, _someone else_ will, eventually." This is some really fucked up, self-justifying inductive reasoning that can't easily be countered by calling out the moral bankruptcy of the premise. In the past, mutual disarmament treaties have been a down-the-line bandaid for this kind of thought process, but the nuclear rearmament we're seeing in the world right now shows it's not a panacea.
  • XorNot 1y ago
    You could start by not inventing vague hypotheticals to argue against, and instead engaging with observable, measurable strategic and tactical reality? War is studied. There are journals, papers and research on war fighting at all possible levels. In the most recent action by Ukraine you can observe actual reality: what did they attack? Military equipment of the enemy. Why did they attack it? To degrade the enemy's ability to sustain and rotate their forces attacking them. What was it for? Well for one thing it will hopefully considerably reduce their ability to bomb civilian targets.
    • _verandaguy 1y ago
      In this specific case, I agree that it was fine -- using drones with limited decision-making ability to strike targets like parked aircraft is okay, as long as there's an overwhelming likelihood of the drones not getting false positives from invalid targets. My response was more aimed at the parent comment to my previous one, which seemed to paint delegating kill/no-kill decisions with a brush of "I don't know why this is such a big deal."
• cpascal 1y ago
  Machines have been making kill-or-no-kill decisions for decades, and they were a lot more indeterminant. Heat-seeking missiles largely kill whatever is hot. Proximity fuses in WW2 detonated whenever they got near something. Anti-personal mines kill whenever pressure is applied. A CIWS will target things that get too close to it that don't identify IFF. Naval mines kill if something magnetic is near them. At the end of the day, it's still humans deploying these weapon systems and accepting the risk that they might cause unintended casualties.
• __MatrixMan__ 1y ago
  When a stressed person mistakenly does something they shouldn't have, they typically stop doing it after only a small handful of mistakes. We get tired, or bored, or we wise up to our mistake. When a misconfigured computer does something wrong, it frequently does it over and over and over again until it is prevented from doing so by an external intervention. No tired private is going to mistakenly rampage through a populated area mowing down civilians. But a confused drone swarm might.
  • amelius 1y ago
    "To err is human, but to really foul things up you need a computer."
  • Nasrudith 1y ago
    Very good principle but horrifyingly a drugged soldier might. Look up Larium, an anti-malaria drug potentially implicated in wartime rampages. Morally and ethically it raises some weird questions. Sure you can balance the probabilities of death from taking a drug vs not. All drugs have a lethal dose. But most drugs don't kill people who haven't taken them.
• lawlessone 1y ago
  >It’s not an easy decision but I’d rather a machine made it than a stressed person! A machine also can't be held responsible. Who do the surviving relatives of the crush pedestrian sue? No longer the driver i guess
  • sfn42 1y ago
    The thing that gets me about this autonomous car thing is that it shouldn't really happen. When a car has to swerve to avoid an accident it is typically because the driver was speeding and/or not paying attention. Autonomous cars wouldn't speed and they are always paying attention. Maybe you could argue that it happened because the other driver was drunk or for some other reason swerved into your lane but this is still an incredibly niche situation and if it does happen just sue that guy. The vast majority of car-to-car accidents happen when both drivers are irresponsible. Autonomous cars will significantly reduce accident rates just by following the rules of traffic and being constantly aware. The whole who should be sued thing is asinine, just have a national insurance fund for it or whatever. Cheaper than all the accidents it prevents.
    • pests 1y ago
      I’ve read that self driving cars almost always try to slow down /brake opposed to swerving for the simple reason of removing energy from any potential collision. There was an article here serveral weeks ago about how in some self driving cars the brakes can be pre-loaded and can be used much quicker than a human could, reaction times aside.
  • SoftTalker 1y ago
    Normally the owner of the vehicle, at least if the driver was using it with their permission. Think about this before you let someone borrow your car.
    • lawlessone 1y ago
      How can you old the owner responsible for the AI's errors?
• Sporktacular 1y ago
  Yes. The private knows he can be punished for getting it wrong.
• ARandomerDude 1y ago
  > tired front-line private It's hard to imagine there is a military in the world that would put something so important in the hands of a private. Yes the drones in this war are relatively low cost but they are also so vital to both sides' war effort that there is inevitably more command and control over their use.