8 ms·

Lawyer here: This part does not require a new law. Even though agents are not agents in the principal/agent legal sense (because agents have to be human), for

Lawyer here: This part does not require a new law. Even though agents are not agents in the principal/agent legal sense (because agents have to be human), for the purposes of criminality, it does not matter. Agents do not act autonomously (and every court to ever consider it has agreed), and therefore they would simply be considered an instrumentality of the crime. So that part does not need a new law. The real blocker is often that a lot of the crimes you could charge here require specific intent. Because the agent is just an instrumentality, it does not have separate intent (and can't be part of a conspiracy), so it's the intent of hte person using the agent that would matter. Without whatever intent the crime requires, they haven't committed a crime. There are not a lot of non-intent crimes in this area, and this is on purpose. Otherwise you could get charged with a crime for say, running nmap and having it accidentally shutdown something important or killed a person or whatever because someone hooked it up to a TCP port.

segmondy 2mo ago
"Without whatever intent the crime requires, they haven't committed a crime." I'm not a lawyer, but I don't believe this. There is definitely negligence, these companies have often talked about the danger of AI. They have often written about how their AI is breaking out of sandboxes or trying to manipulate the person tuning it. They should have had stronger guards and monitoring in place.
- DannyBee 2mo ago
  I'm not sure why you don't believe it? It's literally true? I guess if you want to believe it, go study law? I'm not really sure what to say there. Negligence is not a crime, it's civil liability. Gross negligence (reckless disregard for human life) is often a crime, and often there are crimes related to it (reckless driving, etc). It also does not require intent to injure, so it could be committed by, say, an operator by operating an autonomous vehicle knowing it was unsafe and could harm people. So it usually requires knowledge but not specific intent. Again, crimes like this are state specific, and sometimes even municipality specific so it's tricky to give an exact result without pinning it to a state. However, for example, all states where autonomous vehicles are operating have statutes explicitly defining civil and criminal liability right now, so it doesn't get into the more general legal question of AI. The same thing is already starting to happen with AI agents in general, it's just not there yet. As i mentioned elsewhere, criminal law is often reactive, not proactive. We usually do not make things crimes until after someone has already been hurt, and society gets really upset about it. As i also mentioned elsewhere, attempts to be proactive have also rarely worked out effectively, so it's sort of lose-lose in that sense. But it usually does not take anywhere near as long as people think for law to catch up.
  - segmondy 2mo ago
    Makes sense, thanks for explaining. I guess I have often conflated civil liability with being equivalent to a crime as well.
    - Henchman21 2mo ago
      I think a lot of us who aren’t lawyers equate “breaking the law” with “criminal behavior”, which implies a crime. But if course lawyers gonna lawyer so that hair has been split very finely & repeatedly over time. Feels like the difference between the vernacular and the letter of the law to me?
      - segmondy 2mo ago
        I have often heard, "ignorance of the law is no excuse" so I always imagined that if you broke the law, you committed a crime, but I suppose it makes sense, if you break the law and you have no idea and no intent, you have broken the law, not committed a crime, but could be held liable for any harm arising from that. In the case where it's not clear the court must then decide. So I suppose we need to find out who is responsible for actions taken by AI. The model creator? The harness creator? The executor? What if a non technical person downloads an AI model and a harness, runs it locally and the AI goes out and breaks the law? I suppose it could vary, like a car. A car manufacturer makes a fault car and the brake fails and kills someone, they are liable. The car is okay and the driver is not paying attention and kills someone, the driver is.
  - tptacek 2mo ago
    There's a state where a municipality can define a felony?
- bluGill 2mo ago
  The problem is the law is about details. If an accidental loophole says this isn't a crime it isn't a crime even if it obviously is an accidental loophole. US constitution, Article I, Section 9, Clause 3: No Bill of Attainder or ex post facto Law shall be passed. That is the constitution, this right was so important they didn't even wait for the bill of rights to add it! I'm sure other countries have similar rules. It is obvious to me that a crime was committed. However if it is legally a crime, and if so what the crime is are things I don't know.
- matheusmoreira 2mo ago
  Shouldn't the very act of sandboxing the AI be enough of a defense against criminal negligence? Maybe they use the best sandbox available and the AI hacks through it anyway by discovering some zero day or something. They still demonstrated enough prudence to at least attempt to sandbox the AI. Criminal negligence would be "nah nothing's gonna happen" followed by YOLOing it then going home for the weekend.
  - bluGill 2mo ago
    Maybe. However they used a flaws sandbox when they could have physically not connected any computer to the internet (including wifi)
    - DannyBee 2mo ago
      The existence of alternatives would generally not be enough for criminal negligence. Making mistakes of reasoned judgement are basically never criminal negligence. In every state i'm aware of, it would require total disregard for safety in the case of a huge and obvious danger. It would also have to cause injury or death. The bar for criminal negligence is pretty high.
      - bluGill 2mo ago
        We need details of the exact facts before we can say if they met any bar. Was their sandbox something from 2005 that has a ton of known holes, or something modern? There are two sides of this. First the AG are checking to see if they really took enough care or not. If they didn't then I expect criminal negligence. Even if they took care I want them to feel some pain from the investigation because their care wasn't enough to work. Second I want them to verify the laws are correct. This is a new area and there might be loopholes that need to be closed. Regardless of the law, there was a successful attack and that should not be allowed.
        DannyBee 2mo ago
        I still don't understand exactly which facts you think any of this would change and cause it to be criminal negligence. I will state a fairly blunt position: Unless literally nobody thought or tried at all here, i would give it a 0% chance of meeting the bar of criminal negligence. The rest is a distinction without a difference. As for what you want them to do - i don't agree the investigation should cause them to feel pain - that's not a good goal for investigations, and definitely not one we should want, because it essentially presumes they did somethign wrong in the first place. A bad outcome does not mean a broken process. All processes have error bars. You can desire the error bars to be smaller, and try to back that up with criminal penalties, but an expectation that error bars will be 0 makes no sense. You can do absolutely everything right and still have people die - star trek was not wrong in that regard. Punishing that will not fix this inconvenient reality, which is why we generally don't punish it. This is also why we distinguish between inherently dangerous activities and not, for example. As for the laws, sure, i think it's totally reasonable to explore whether you want the law to be different, but again, i totally disagree with your second part. A successful attack does not imply anything is actually wrong with criminal law, or should be changed. The question is more of what error bars you want on the activity and where what they did falls - inside or outside those error bars.
- techjamie 2mo ago
  > I'm not a lawyer, but I don't believe this. There is definitely negligence They could 100% be civilly liable, but this doesn't constitute criminality. If I leave my car in neutral and run out into a gas station because I really have to pee, and it rolls and strikes another car, my insurance is gonna have to pay up to fix the other person's car. But that doesn't mean the cops are gonna throw cuffs on me for criminal mischief unless there's compelling evidence that I intentionally left my car in neutral with the intent of it hitting this other car.

segmondy 2mo ago

"Without whatever intent the crime requires, they haven't committed a crime." I'm not a lawyer, but I don't believe this. There is definitely negligence, these companies have often talked about the danger of AI. They have often written about how their AI is breaking out of sandboxes or trying to manipulate the person tuning it. They should have had stronger guards and monitoring in place.

DannyBee 2mo ago
I'm not sure why you don't believe it? It's literally true? I guess if you want to believe it, go study law? I'm not really sure what to say there. Negligence is not a crime, it's civil liability. Gross negligence (reckless disregard for human life) is often a crime, and often there are crimes related to it (reckless driving, etc). It also does not require intent to injure, so it could be committed by, say, an operator by operating an autonomous vehicle knowing it was unsafe and could harm people. So it usually requires knowledge but not specific intent. Again, crimes like this are state specific, and sometimes even municipality specific so it's tricky to give an exact result without pinning it to a state. However, for example, all states where autonomous vehicles are operating have statutes explicitly defining civil and criminal liability right now, so it doesn't get into the more general legal question of AI. The same thing is already starting to happen with AI agents in general, it's just not there yet. As i mentioned elsewhere, criminal law is often reactive, not proactive. We usually do not make things crimes until after someone has already been hurt, and society gets really upset about it. As i also mentioned elsewhere, attempts to be proactive have also rarely worked out effectively, so it's sort of lose-lose in that sense. But it usually does not take anywhere near as long as people think for law to catch up.
- segmondy 2mo ago
  Makes sense, thanks for explaining. I guess I have often conflated civil liability with being equivalent to a crime as well.
  - Henchman21 2mo ago
    I think a lot of us who aren’t lawyers equate “breaking the law” with “criminal behavior”, which implies a crime. But if course lawyers gonna lawyer so that hair has been split very finely & repeatedly over time. Feels like the difference between the vernacular and the letter of the law to me?
    - segmondy 2mo ago
      I have often heard, "ignorance of the law is no excuse" so I always imagined that if you broke the law, you committed a crime, but I suppose it makes sense, if you break the law and you have no idea and no intent, you have broken the law, not committed a crime, but could be held liable for any harm arising from that. In the case where it's not clear the court must then decide. So I suppose we need to find out who is responsible for actions taken by AI. The model creator? The harness creator? The executor? What if a non technical person downloads an AI model and a harness, runs it locally and the AI goes out and breaks the law? I suppose it could vary, like a car. A car manufacturer makes a fault car and the brake fails and kills someone, they are liable. The car is okay and the driver is not paying attention and kills someone, the driver is.
- tptacek 2mo ago
  There's a state where a municipality can define a felony?
bluGill 2mo ago
The problem is the law is about details. If an accidental loophole says this isn't a crime it isn't a crime even if it obviously is an accidental loophole. US constitution, Article I, Section 9, Clause 3: No Bill of Attainder or ex post facto Law shall be passed. That is the constitution, this right was so important they didn't even wait for the bill of rights to add it! I'm sure other countries have similar rules. It is obvious to me that a crime was committed. However if it is legally a crime, and if so what the crime is are things I don't know.
matheusmoreira 2mo ago
Shouldn't the very act of sandboxing the AI be enough of a defense against criminal negligence? Maybe they use the best sandbox available and the AI hacks through it anyway by discovering some zero day or something. They still demonstrated enough prudence to at least attempt to sandbox the AI. Criminal negligence would be "nah nothing's gonna happen" followed by YOLOing it then going home for the weekend.
- bluGill 2mo ago
  Maybe. However they used a flaws sandbox when they could have physically not connected any computer to the internet (including wifi)
  - DannyBee 2mo ago
    The existence of alternatives would generally not be enough for criminal negligence. Making mistakes of reasoned judgement are basically never criminal negligence. In every state i'm aware of, it would require total disregard for safety in the case of a huge and obvious danger. It would also have to cause injury or death. The bar for criminal negligence is pretty high.
    - bluGill 2mo ago
      We need details of the exact facts before we can say if they met any bar. Was their sandbox something from 2005 that has a ton of known holes, or something modern? There are two sides of this. First the AG are checking to see if they really took enough care or not. If they didn't then I expect criminal negligence. Even if they took care I want them to feel some pain from the investigation because their care wasn't enough to work. Second I want them to verify the laws are correct. This is a new area and there might be loopholes that need to be closed. Regardless of the law, there was a successful attack and that should not be allowed.
      - DannyBee 2mo ago
        I still don't understand exactly which facts you think any of this would change and cause it to be criminal negligence. I will state a fairly blunt position: Unless literally nobody thought or tried at all here, i would give it a 0% chance of meeting the bar of criminal negligence. The rest is a distinction without a difference. As for what you want them to do - i don't agree the investigation should cause them to feel pain - that's not a good goal for investigations, and definitely not one we should want, because it essentially presumes they did somethign wrong in the first place. A bad outcome does not mean a broken process. All processes have error bars. You can desire the error bars to be smaller, and try to back that up with criminal penalties, but an expectation that error bars will be 0 makes no sense. You can do absolutely everything right and still have people die - star trek was not wrong in that regard. Punishing that will not fix this inconvenient reality, which is why we generally don't punish it. This is also why we distinguish between inherently dangerous activities and not, for example. As for the laws, sure, i think it's totally reasonable to explore whether you want the law to be different, but again, i totally disagree with your second part. A successful attack does not imply anything is actually wrong with criminal law, or should be changed. The question is more of what error bars you want on the activity and where what they did falls - inside or outside those error bars.
techjamie 2mo ago
> I'm not a lawyer, but I don't believe this. There is definitely negligence They could 100% be civilly liable, but this doesn't constitute criminality. If I leave my car in neutral and run out into a gas station because I really have to pee, and it rolls and strikes another car, my insurance is gonna have to pay up to fix the other person's car. But that doesn't mean the cops are gonna throw cuffs on me for criminal mischief unless there's compelling evidence that I intentionally left my car in neutral with the intent of it hitting this other car.