5 ms·

> “The AI hallucinated. I never asked it to do that.” > That’s the defense. And here’s the problem: it’s often hard to refute with confidence. Why is it neces

• salawat 9mo ago
  Except for the fact that that very accountability sink is relied on by senior management/CxO's the world over. The only difference is that before AI, it was the middle manager's fault. We didn't tell anyone to break the law. We just put in place incentive structures that require it, and play coy, then let anticipatory obedience do the rest. Bingo. Accountability severed. You can't prove I said it in a court of law, and skeevy shit gets done because some poor bloke down the ladder is afraid of getting fired if he doesn't pull out all the stops to meet productivity quotas. AI is just better because no one can actually explain why the thing does what it does. Perfect management scapegoat without strict liability being made explicit in law.
  • pixl97 9mo ago
    Hence why many life and death things require licencing and compliance, and tend to come with very long paper trails. The software world has been very allergic to getting anywhere near the vicinity of a system like that.
    • salawat 9mo ago
      Did I give the impression that the phenomena was unique to software? Hell, Boeing was a shining example of the principle in action with 737 MAX. Don't get much more "people live and die by us, and we know it (but management set up the culture and incentives to make a deathtrap anyway)." No one to blame of course. These things just happen. Licensure alone doesn't solve all these ills. And for that matter, once regulatory capture happens, it has a tendency to make things worse due to consolidation pressure.
  • Muromec 9mo ago
    >AI is just better because no one can actually explain why the thing does what it does. Perfect management scapegoat without strict liability being made explicit in law. AI is worse in that regard, because, although you can't explain why it does so, you can point a finger at it, say "we told you so" and provide the receipts of repeated warnings that the thing has a tendency of doing the things.
• niyikiza 9mo ago
  You're right, they should be responsible. The problem is proving it. "I asked it to summarize reports, it decided to email the competitor on its own" is hard to refute with current architectures. And when sub-agents or third-party tools are involved, liability gets even murkier. Who's accountable when the action executed three hops away from the human? The article argues for receipts that make "I didn't authorize that" a verifiable claim
  • bulatb 9mo ago
    There's nothing to prove. Responsibility means you accept the consequences for its actions, whatever they are. You own the benefit? You own the risk. If you don't want to be responsible for what a tool that might do anything at all might do, don't use the tool. The other option is admitting that you don't accept responsibility, not looking for a way to be "responsible" but not accountable.
    • tossandthrow 9mo ago
      Sounds good in theory, doesn't work in reality. Had it worked then we would have seen many more CEOs in prison.
      • NoMoreNicksLeft 9mo ago
        The veil of liability is built into statute, and it's no accident. Such so magic forcefield exists for you, though.
      • walt_grata 9mo ago
        There being a few edge cases where it doesn't work in doesn't mean it doesn't work in the majority of cases and that we shouldn't try to fix the edge cases.
      • bulatb 9mo ago
        We're taking about different things. To take responsibility is volunteering to accept accountability without a fight. In practice, almost everyone is held potentially or actually accountable for things they never had a choice in. Some are never held accountable for things they freely choose, because they have some way to dodge accountability. The CEOs who don't accept accountability were lying when they said they were responsible.
• doctorpangloss 9mo ago
  Wait till you find out about “pedal confusion.”
• nerdsniper 9mo ago
  The distinction some people are making is between copy/pasting text vs agentic action. Generally mistakes "work product" as in output from ChatGPT that the human then files with a court, etc. are not forgiven, because if you signed the document, you own its content. Versus some vendor-provided AI Agent which simply takes action on its own that a "reasonable person" would not have expected it to. Often we forgive those kinds of software bloopers.
  • ori_b 9mo ago
    If you put a brick on the accelerator of a car and hop out, you don't get to say "I wasn't even in the car when it hit the pedestrian".
    • Shalomboy 9mo ago
      This is true for bricks, but it is not true if your dog starts up your car and hits a pedestrian. Collisions caused by non-human drivers are a fascinating edge case for the times we're in.
      • victorbjorklund 9mo ago
        I don’t know where you from but at least in Sweden you have strict liability for anything your dog does
      • deleted 9mo ago
        [deleted]
      • freejazz 9mo ago
        Prima facie negligence = liability
      • ori_b 9mo ago
        In the USA, at least, it seems pet owners are liable for any harm their pets do.
      • cess11 9mo ago
        Legally, in a lot of jurisdictions, a dog is just your property. What it does, you did, usually with presumed intent or strict liability.
• ibejoeb 9mo ago
  Yeah. Legal will need to catch up to deal with some things, surely, but the basic principles for this particular scenario aren't that novel. If you're a professional and have an employee acting under your license, there's already liability. There is no warrant concept (not that I can think of right now, at least) that will obviate the need to check the work and carry professional liability insurance. There will always be negligence and bad actors. The new and interesting part is that while we have incentives and deterrents to keep our human agents doing the right thing, there isn't really an analog to check the non-human agent. We don't have robot prison yet.
• imiric 9mo ago
  That's quickly becoming difficult to determine. The workflow of starting dozens or hundreds of "agents" that work autonomously is starting to gain traction. The goal of people who work like this is to completely automate software development. At some point they want to be able to give the tool an arbitrary task, presumably one that benefits them in some way, and have it build, deploy, and use software to complete it. When millions of people are doing this, and the layers of indirection grow in complexity, how do you trace the result back to a human? Can we say that a human was really responsible for it? Maybe this seems simple today, but the challenges this technology forces on society are numerous, and we're far from ready for it.
  • niyikiza 9mo ago
    This is the problem we're working on. When orchestrators spawn sub-agents spawn tools, there's no artifact showing how authority flowed through the chain. Warrants are a primitive for this: signed authorization that attenuates at each hop. Each delegation is signed, scope can only narrow, and the full chain is verifiable at the end. Doesn't matter how many layers deep.
  • Wobbles42 9mo ago
    Translation: People want to use a tool and not be liable for the result. People not wanting to be liable for their actions is not new. AI hasn't changed anything here, it's just a new lame excuse.
• NedF 9mo ago
  [dead]
• godelski 9mo ago
  > It shouldn't matter, because whoever is producing the work product is responsible for it, no matter whether genAI was involved or not. I hate to ask, but did you RTFA? Scrolling down ever so slightly (emphasis not my own) | *Who authorized this class of action, for which agent identity, under what constraints, for how long; and how did that authority flow?* | A common failure mode in agent incidents is not “we don’t know what happened,” but: | > We can’t produce a crisp artifact showing that a specific human explicitly authorized the scope that made this action possible. They explicitly state that the problem is you don't know which human to point at.
  • plastic041 9mo ago
    > They explicitly state that the problem is you don't know which human to point at. The point is "explicitly authorized", as the article emphasizes. It's easy to find who ran the agent(article assumes they have OAuth log). This article is about 'Everyone knows who did it, but did they do it on purpose? Our system can figure it out'
• ragBagger 9mo ago
  I feel like you have missed the point of this. It isn't to completely absolve the user of liability, it's to prove malice instead of incompetence. If the user claims that they only authorized the bot to review files, but they've warranted the bot to both scan every file and also send emails to outside sources, the competitors in this case, then you now have proof that the user was planning on committing corporate espionage. To use a more sane version of an example below, if your dog runs outside the house and mauls a child, you are obviously guilty of negligence, but if there's proof of you unleashing the dog and ordering the attack, you're guilty of murder.