6 ms·

> if we ever let it get as far as giving LLM completions direct agentic access to real world infra, then we've failed I see no reason to believe this is not al

• justlikereddit 2y ago
  Israel use such a system to decide who and where should be be bombed to death, is that direct enough control of weapons to qualify?
  • throwaway290 2y ago
    On the other hand indiscriminately throwing rockets and targeting civilians like Hamas did for decades is loads better!
    • staunton 2y ago
      You're comparing the actions of what most people here view as a democratic state (parlamentary republic) and an opaquely run terrorist organization. We're talking about potential consequences of giving AIs influence on military decisions. To that point, I'm not sure what your comment is saying. Is it perhaps: "we're still just indiscriminately killing civilians just as always, so giving AI control is fine"?
      • throwaway290 2y ago
        > we're still just indiscriminately killing civilians just as always, so giving AI control is fine I don't even want to respond because the "we" and "as always" here is doing a lot. I don't have it in me to have an extended discussion to address how indiscriminately killing civilians was never accepted practice in modern warfare. Anyways. There are two conditions in which I see this argument(?) is useful. If you assume their goal is indiscriminately killing civilians and ML helps them, or if you assume that their ML tools cause less precise targeting of militants that causes more civilians being dead contrary to intent. Which one is it? Cards on the table. > We're talking about potential consequences of giving AIs influence on military decisions No, I replied to a comment that was talking about a specific example.
        • staunton 2y ago
          > I don't have it in me to have an extended discussion to address how indiscriminately killing civilians was never accepted practice in modern warfare. I did not claim it is/was accepted practice. I was asking if "doing it with AI is just the same so what's the big deal" was your position on the general issue (of AI making decisions in war), which I thought was a possible interpretation of your previous comment. > No, I replied to a comment that was talking about a specific example. OK. That means the two of us were/are just talking past each other and won't be having an interesting discussion.
          • throwaway290 2y ago
            > doing it with AI is just the same so what's the big deal Is that implying fewer civilian deaths is NOT a big deal? I think the parents and children of people who were killed would disagree with you.
    • deleted 2y ago
      [deleted]
    • javcasas 2y ago
      I'm sure you can understand that both of them are awful, and one does not justify the other (feel free to choose which is the "one" and which is the "other").
      • throwaway290 2y ago
        Oh, totally. If there were two sides indiscriminately killing each other for no reason I couldn't say one justifies the other. But back to the topic, if one side is using ML to ultimately kill fewer civilians then this is a bad example against using ML.
        • cruffle_duffle 2y ago
          > But back to the topic, if one side is using ML to ultimately kill fewer civilians then this is a bad example against using ML. Depends on how that ML was trained and how well its engineers can explain and understand how its outputs are derived from its inputs. LLM’s are notoriously hard to trace and explain.
          • throwaway290 2y ago
            I don't disagree.
            • javcasas 2y ago
              Also the military is notorious for ignoring requests by scientists, for example to not use the nuclear bomb as a weapon of war. https://en.m.wikipedia.org/wiki/Szil%C3%A1rd_petition https://en.m.wikipedia.org/wiki/Szil%C3%A1rd_petition So the developers may program the AI to be careful, but the military has the final word on deciding if the AI is set on safety or agressiveness.
  • rjagH 2y ago
    For the downvoters: https://www.972mag.com/lavender-ai-israeli-army-gaza/ https://www.972mag.com/lavender-ai-israeli-army-gaza/ It is so sad that mainstream narratives are upvoted and do not require sources, whereas heterodoxy is always downvoted. People would have downvoted Giordano Bruno here.
    • selimthegrim 2y ago
      Giordano Bruno would have to show off his viral memory palace tricks on TikTok before he got a look in.
    • mhotchen 2y ago
      This is awful
    • int_19h 2y ago
      It's mainstream enough to have a Wikipedia article. https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_Gaza_Strip https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G...
  • deleted 2y ago
    [deleted]
• semi-extrinsic 2y ago
  > because they are a good scape goat and not because they save money. Exactly the point - there are humans in control who filter the AI outputs before they are applied to the real world. We don't give them direct access to the HR platform or the targeting computer, there is always a human in the loop. If the AI's output is to fire the CEO or bomb an allied air base, you ignore what it says. And if it keeps making too many such mistakes, you simply decommission it.
  • dartos 2y ago
    I think you’re missing OC’s point. There’s always a human in the loop, but instead of stopping an immoral decision, they’ll just keep that decision and blame the AI if there’s any pushback. It’s what United Healthcare was doing.
    • semi-extrinsic 2y ago
      No, this part I agree 100% with. But in this scenario there is no grandiose danger due to lack of "alignment". Either the AI says what the MBA wants it to say, or it gets asked again with a modified prompt. You can replace "AI" with "McKinsey consultant" and everything in the whole scenario is exactly the same.
      • int_19h 2y ago
        Consider the case of AI designating targets for Israeli strikes in Gaza [1] which get only a cursory review by humans. One could argue that it's still the case of AI saying what humans want it to say ("give us something to bomb"), but the specific target that it picks still matters a great deal. [1] https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_Gaza_Strip https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G...
        • HappMacDonald 2y ago
          Replace AI with dart board..
  • 2y ago
• Grimblewald 2y ago
  hell, AI systems order human deaths, and it is followed without humans double checking the order, and no one bats an eyelid. Granted this is israel and they're a violent entity trying to create and ethno-state by perpetrating a genocide so a system that is at best 90% accurate in identifying Hamas (using Israels insultingly broad definition) is probably fine for them, but it doesn't change the fact we allow AI systems to order human executions and these orders are not double checked, and are followed by humans. Don't believe me? read up on the "lavender" system Israel uses.
• internet_points 2y ago
  > They are surely in direct control of weapons somewhere. https://spectrum.ieee.org/jailbreak-llm https://spectrum.ieee.org/jailbreak-llm Here's some guy getting toasted by a flame-throwing robot dog by saying "bark" at it: https://www.youtube.com/clip/UgkxmKAEK_BnLIMjyRL7l6j_ECwNEms33TIb https://www.youtube.com/clip/UgkxmKAEK_BnLIMjyRL7l6j_ECwNEms... (The Thermonator: https://throwflame.com/products/thermonator-robodog/ https://throwflame.com/products/thermonator-robodog/ they also sell flame-throwing drones because who wouldn't want that in the wrong hands)
• jerf 2y ago
  "They are firing people and they are deciding who gets their insurance claims covered." AI != LLM. "AI" has been deciding those things for a while, especially insurance claims, since before LLMs were practical. LLMs being hooked up to insurance claims is a highly questionable decision for lots of reasons, including the inability to "explain" its decisions. But this is not a characteristic of all AI systems, and there are plenty of pre-LLM systems that were called AI that are capable of explaining themselves and/or having their decisions explained reasonably. They can also have reasonably characterizable behaviors that can be largely understood. I doubt LLMs are, at this moment, hooked up to too many direct actions like that, but that is certainly rapidly changing. This is the time for the community engineering with them to take a moment to look to see if this is actually a good idea before rolling it out. I would think someone in an insurance company looking at hooking up LLMs to their system should be shaken by an article like this. They don't want a system that is sitting there and considering these sorts of factors in their decision. It isn't even just that they'd hate to have an AI that decided it had a concept of "mercy" and decided that this person, while they don't conform to the insurance company policies it has been taught, should still be approved. It goes in all directions; the AI is as likely to have an unhealthy dose of misanthropy and accidentally infer that it is supposed to be pursuing the interests of the insurance company and start rejecting claims way too much, and any number of other errors in any number of other directions. The insurance companies want an automated representation of their own interests without any human emotions involved; an automated Bob Parr is not appealing to them: https://www.youtube.com/watch?v=O_VMXa9k5KU https://www.youtube.com/watch?v=O_VMXa9k5KU (which is The Incredibles insurance scene where Mr. Incredible hacks the system on behalf of a sob story)
  • dragonwriter 2y ago
    > “AI” has been deciding those things for a while, especially insurance claims, since before LLMs were practical. Yeah, but no one thinks of rules engines as “AI” any more. AI is a buzzword whose applicability to any particular technology fades with the novelty of that technology.
    • jerf 2y ago
      My point is the equivocation is not logically valid. If you want to operate on the definition that AI is strictly the "new" stuff we don't understand yet, you must be sure that you do not slip in the old stuff under the new definition and start doing logic on it. I'm actually not making fun of that definition, either. YouTube has been trying to get me to watch https://www.youtube.com/watch?v=UZDiGooFs54 https://www.youtube.com/watch?v=UZDiGooFs54 , "The moment we stopped understanding AI [AlexNet]", but I'm pretty sure I can guess the content of the entire video from the thumbnail. I would consider it a reasonable 2040s definition of "AI" as "any algorithm humans can not deeply understand"; it may not be what people think of now, but that definition would certainly capture a very, very important distinction between algorith types. It'll leave some stuff at the fringes, but eh, all definitions have that if you look hard enough.