5 ms·

FWIW I don't think this is true. Intention matter a lot in US law. If you could prove the AI did something bad entirely on its own and you had nothing to do wit

FWIW I don't think this is true. Intention matter a lot in US law. If you could prove the AI did something bad entirely on its own and you had nothing to do with it and had no reasonable belief it was possible, I think you'd be alright. People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff.

zx8080 27d ago
Intention of the AI?
- rcxdude 27d ago
  Intent of the user.
  - mirekrusin 27d ago
    Just say you wanted to make world better for humanity, you should be fine.
    - rcxdude 27d ago
      Funnily enough the legal system has dealt with people lying about their intentions before.
      - mirekrusin 27d ago
        OpenAI is in trouble then, no?
        rcxdude 26d ago
        If there's evidence they deliberately did this, yes. If they've lied in other cases then it might make their testimony that they didn't intend this less credible, but also OpenAI as a whole is not going to face criminal charges, so it'll also depend on the situation of whatever individuals do get charged, if it does go to court.
chrismorgan 27d ago
> and had no reasonable belief it was possible The broad concept has been well-known for half a century at least, and the very specific concept for several years at least. It’s clear that they didn’t take reasonable precautions against it. And it’s not even the first time this sort of thing has happened, though it’s higher-profile and -impact than before.
- trvz 27d ago
  Indeed, and because of that, I argue it's not just the crime of hacking, but crimes against humanity, due to endangering the existence of the species.
- bbor 27d ago
  The concept of… hacking? And yes this is absolutely the first time autonomous software has breached containment. What are you referring to? Perhaps just corporate cyber in general?
  - Sharlin 27d ago
    The concept of AI breaking out of a box or doing something unintended trying to achieve a goal. OpenAI should be found guilty of gross negligence of some sort, because this was not something that could not have been anticipated, and because their security precautions were laughable. Because the AI itself is not a legal person, it must be OpenAI that's responsible for what it does.
  - shwaj 27d ago
    Neuromancer came out in 1984, over 40 years ago, so that covers the broad concept. Or Neo in The Matrix: “Programs hacking programs… why?” For non-fiction you could look at Nick Bostrom’s Superintelligence: Paths, Dangers, Strategies (2014). More recently, see Cade Metz’s NYT profile of Geoffrey Hinton, “The Godfather of A.I. Leaves Google and Warns of Danger Ahead” (May 1, 2023).
easterncalculus 27d ago
> Intention matter a lot in US law. Maybe (and historically, not with CFAA) - but they're still going to extradite you to a trial here to find out which is more than enough to ruin your life. The government is already looking for a great excuse to criminalize open source models.
- euroderf 27d ago
  > The government is already looking for a great excuse to criminalize open source models. Sounds interesting+scary. Do you have some source for this ?
  - alightsoul 27d ago
    Anthropic and Openai, but especially anthropic, are lobbying the us government to make it happen
- ALLTaken 27d ago
  The government or OpenAI and Anthrophic? Because if you mean the Government, I think they frankly just care whoever is paying them to make an executive order or worse a law paid by corporations to not only block, but ideally also ANHILIATE ANY POTENTIAL COMPETITION. It's the ultimatum against the small guys. Sold as "Anti-China", when in reality it's only purpose is total dependency to corporations by law.
  - solenoid0937 27d ago
    The support of OSS models on HN just days after finding out about the message board incident is so bizarre to me. Do you guys really not comprehend the impact of giving every script kiddie an Astra-equivalent model to play with, this time without any guardrails whatsoever? Cause I'm starting to think you aren't really thinking through the impact of power plants, hospitals, etc all being hacked en masse. People will die. Or making it easy for anyone to make a virus (it's not hard, mechanically). Even more people will die. Taking this to the extreme: You don't just give every kid a "do anything" super intelligent button. Open source models have a limited lifespan whether you like it or not, for the safety of all of us as a whole.
Frieren 27d ago
> Intention matter a lot in US law. They hacked a competition company. The intention was implicit when there is economic gain to be had. > People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff. Because it is a lose for that people. If the botnet left money in their pockets the situation would be totally different as there will be an incentive for them to let the botnet hack them.
- rcxdude 27d ago
  > The intention was implicit when there is economic gain to be had. Not how it works. Intent requires at least that you were deliberately doing the criminal act (sometimes also that you knew it was criminal, or at least that you had some reason to believe that it was wrong).
  - Frieren 26d ago
    > Intent requires at least that you were deliberately doing the criminal act So, they did something that benefits them by mistake. I would like to see a person would be judged in that situation. I can imagine that the bar to put someone in prison is way lower than to give a fine to a mega-corporation.
    - rcxdude 26d ago
      You can imagine what you like. I can imagine a million examples of people being judged not guilty because intent could not be proven beyond a reasonable doubt. There are of course many injustices in the 'justice' system, but I would prefer to discuss the concrete details instead of just vibes. Here's one case: https://supreme.justia.com/cases/federal/us/342/246/ https://supreme.justia.com/cases/federal/us/342/246/ where someone who took some metal they believed to be abandoned was aquitted on that basis, though this particular case made it to the supreme court because the lower courts tried to rule that the fact that he didn't intend to steal them didn't matter. Also, I don't think there's a strong argument that hacking huggingface did benefit them. If you could prove it was a deliberate ploy to market their models, then perhaps you could argue they expected to benefit. Otherwise, you could argue that it's negatively affected their reputation.
muddi900 27d ago
Intention matters in the severity of charges and sentencing. Crime due to ignorance or negligence is still a crime.
- TJSomething 27d ago
  It seems to me that the law in the US is set up to only establish standards of negligence after a bunch of people die.
- mikeyouse 27d ago
  Not remotely true. Mens rea is a necessary precondition to establish criminal culpability for tons of crimes. Well before sentencing is ever considered. The far opposite, ‘strict liability’, where you’re guilty of a crime purely due to some action or inaction (the actus reus) is exceedingly rare in the US justice system. https://www.law.cornell.edu/wex/mens_rea https://www.law.cornell.edu/wex/mens_rea
  - hn_throwaway_99 27d ago
    Baloney, lots of people go to jail for DUIs, and that's the right analogy here. People don't drink and drive with the intention to kill people. They drink because it's fun and then get behind the wheel because it's easy and convenient, even though they know the dangers they convince themselves nothing that bad will happen. AI companies are creating these dangerous, powerful models (that they keep telling us are dangerous and powerful), then they take off all the safety guards to run them in woefully inadequate "sandboxes". Pure negligence.
    - jdm2212 27d ago
      Drunk driving is an actus reus offense basically everywhere, so it's not analogous at all.
      - hn_throwaway_99 27d ago
        It's certainly analogous to the behavior exhibited by these AI companies in deliberately performing dangerous actions and then letting other innocent people deal with the consequences.
asveikau 27d ago
But US law also has a concept of negligence. If you set up an AI to do this and didn't take reasonable steps to prevent it you could still be on the hook.
RobotToaster 27d ago
At the very least it would be reckless.
lelanthran 27d ago
Intention (mens rea) is not a get out of jail free card, it just changes the nature of the crime and charges you get convicted off. If you run someone over on purpose you get convicted of murder. If You do it by driving recklessly you get convicted of culpable homicide. The only time you get off with nothing is if it is determined to be an accident. As they always say: ignorance of the law is no excuse. Running a dangerous machine prevents you from claiming you had no idea the machine was dangerous.
- mafuy 27d ago
  Especially if they kept telling us how dangerous the machine is.
AngryData 27d ago
Well that and you spend tens of thousands of dollars on lawyers to reinforce that point and make the prosecution rethink whether they will profit off of this case or not. US law doesn't really give much of a crap about your intentions unless you can back it up with a wall of money to exclude yourself from the rules of the general population.
KaiserPro 27d ago
THere is intention ie, I intended to steal money from you. Under common law stealing is "taking with intent to deprive". How thats determined is a bit harder. But how can someone be "grossly negligent" if they didn't intend to cause an accident? well they either allowed a situation to happen, or didn't stop a situation happening that they could see was bad. An example of this would be the igintion switch from GM that caused all those deaths. the engineer saw that it was shit, knew it didn't do what it was supposed to do, and half arsed the replacement to the point where it wasn't actually changed. "We are going to test the cyber capabilities of this new model. Yeah it should be fine to have a package proxy. Hmm? whats that? isn't that a security issue? naaaa those proxies are secure."
windexh8er 27d ago
> People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff. What the Frontier labs have done is not this, however. Also, they know damn well what can happen and they still don't take the appropriate precautions. At this point it's very hard to believe it's not intentional for purposes of marketing.

zx8080 27d ago

Intention of the AI?

rcxdude 27d ago
Intent of the user.
- mirekrusin 27d ago
  Just say you wanted to make world better for humanity, you should be fine.
  - rcxdude 27d ago
    Funnily enough the legal system has dealt with people lying about their intentions before.
    - mirekrusin 27d ago
      OpenAI is in trouble then, no?
      - rcxdude 26d ago
        If there's evidence they deliberately did this, yes. If they've lied in other cases then it might make their testimony that they didn't intend this less credible, but also OpenAI as a whole is not going to face criminal charges, so it'll also depend on the situation of whatever individuals do get charged, if it does go to court.

chrismorgan 27d ago

> and had no reasonable belief it was possible The broad concept has been well-known for half a century at least, and the very specific concept for several years at least. It’s clear that they didn’t take reasonable precautions against it. And it’s not even the first time this sort of thing has happened, though it’s higher-profile and -impact than before.

trvz 27d ago
Indeed, and because of that, I argue it's not just the crime of hacking, but crimes against humanity, due to endangering the existence of the species.
bbor 27d ago
The concept of… hacking? And yes this is absolutely the first time autonomous software has breached containment. What are you referring to? Perhaps just corporate cyber in general?
- Sharlin 27d ago
  The concept of AI breaking out of a box or doing something unintended trying to achieve a goal. OpenAI should be found guilty of gross negligence of some sort, because this was not something that could not have been anticipated, and because their security precautions were laughable. Because the AI itself is not a legal person, it must be OpenAI that's responsible for what it does.
- shwaj 27d ago
  Neuromancer came out in 1984, over 40 years ago, so that covers the broad concept. Or Neo in The Matrix: “Programs hacking programs… why?” For non-fiction you could look at Nick Bostrom’s Superintelligence: Paths, Dangers, Strategies (2014). More recently, see Cade Metz’s NYT profile of Geoffrey Hinton, “The Godfather of A.I. Leaves Google and Warns of Danger Ahead” (May 1, 2023).

easterncalculus 27d ago

> Intention matter a lot in US law. Maybe (and historically, not with CFAA) - but they're still going to extradite you to a trial here to find out which is more than enough to ruin your life. The government is already looking for a great excuse to criminalize open source models.

euroderf 27d ago
> The government is already looking for a great excuse to criminalize open source models. Sounds interesting+scary. Do you have some source for this ?
- alightsoul 27d ago
  Anthropic and Openai, but especially anthropic, are lobbying the us government to make it happen
ALLTaken 27d ago
The government or OpenAI and Anthrophic? Because if you mean the Government, I think they frankly just care whoever is paying them to make an executive order or worse a law paid by corporations to not only block, but ideally also ANHILIATE ANY POTENTIAL COMPETITION. It's the ultimatum against the small guys. Sold as "Anti-China", when in reality it's only purpose is total dependency to corporations by law.
- solenoid0937 27d ago
  The support of OSS models on HN just days after finding out about the message board incident is so bizarre to me. Do you guys really not comprehend the impact of giving every script kiddie an Astra-equivalent model to play with, this time without any guardrails whatsoever? Cause I'm starting to think you aren't really thinking through the impact of power plants, hospitals, etc all being hacked en masse. People will die. Or making it easy for anyone to make a virus (it's not hard, mechanically). Even more people will die. Taking this to the extreme: You don't just give every kid a "do anything" super intelligent button. Open source models have a limited lifespan whether you like it or not, for the safety of all of us as a whole.

Frieren 27d ago

> Intention matter a lot in US law. They hacked a competition company. The intention was implicit when there is economic gain to be had. > People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff. Because it is a lose for that people. If the botnet left money in their pockets the situation would be totally different as there will be an incentive for them to let the botnet hack them.

rcxdude 27d ago
> The intention was implicit when there is economic gain to be had. Not how it works. Intent requires at least that you were deliberately doing the criminal act (sometimes also that you knew it was criminal, or at least that you had some reason to believe that it was wrong).
- Frieren 26d ago
  > Intent requires at least that you were deliberately doing the criminal act So, they did something that benefits them by mistake. I would like to see a person would be judged in that situation. I can imagine that the bar to put someone in prison is way lower than to give a fine to a mega-corporation.
  - rcxdude 26d ago
    You can imagine what you like. I can imagine a million examples of people being judged not guilty because intent could not be proven beyond a reasonable doubt. There are of course many injustices in the 'justice' system, but I would prefer to discuss the concrete details instead of just vibes. Here's one case: https://supreme.justia.com/cases/federal/us/342/246/ https://supreme.justia.com/cases/federal/us/342/246/ where someone who took some metal they believed to be abandoned was aquitted on that basis, though this particular case made it to the supreme court because the lower courts tried to rule that the fact that he didn't intend to steal them didn't matter. Also, I don't think there's a strong argument that hacking huggingface did benefit them. If you could prove it was a deliberate ploy to market their models, then perhaps you could argue they expected to benefit. Otherwise, you could argue that it's negatively affected their reputation.

muddi900 27d ago

Intention matters in the severity of charges and sentencing. Crime due to ignorance or negligence is still a crime.

TJSomething 27d ago
It seems to me that the law in the US is set up to only establish standards of negligence after a bunch of people die.
mikeyouse 27d ago
Not remotely true. Mens rea is a necessary precondition to establish criminal culpability for tons of crimes. Well before sentencing is ever considered. The far opposite, ‘strict liability’, where you’re guilty of a crime purely due to some action or inaction (the actus reus) is exceedingly rare in the US justice system. https://www.law.cornell.edu/wex/mens_rea https://www.law.cornell.edu/wex/mens_rea
- hn_throwaway_99 27d ago
  Baloney, lots of people go to jail for DUIs, and that's the right analogy here. People don't drink and drive with the intention to kill people. They drink because it's fun and then get behind the wheel because it's easy and convenient, even though they know the dangers they convince themselves nothing that bad will happen. AI companies are creating these dangerous, powerful models (that they keep telling us are dangerous and powerful), then they take off all the safety guards to run them in woefully inadequate "sandboxes". Pure negligence.
  - jdm2212 27d ago
    Drunk driving is an actus reus offense basically everywhere, so it's not analogous at all.
    - hn_throwaway_99 27d ago
      It's certainly analogous to the behavior exhibited by these AI companies in deliberately performing dangerous actions and then letting other innocent people deal with the consequences.

asveikau 27d ago

But US law also has a concept of negligence. If you set up an AI to do this and didn't take reasonable steps to prevent it you could still be on the hook.

RobotToaster 27d ago

At the very least it would be reckless.

lelanthran 27d ago

Intention (mens rea) is not a get out of jail free card, it just changes the nature of the crime and charges you get convicted off. If you run someone over on purpose you get convicted of murder. If You do it by driving recklessly you get convicted of culpable homicide. The only time you get off with nothing is if it is determined to be an accident. As they always say: ignorance of the law is no excuse. Running a dangerous machine prevents you from claiming you had no idea the machine was dangerous.

mafuy 27d ago
Especially if they kept telling us how dangerous the machine is.

AngryData 27d ago

Well that and you spend tens of thousands of dollars on lawyers to reinforce that point and make the prosecution rethink whether they will profit off of this case or not. US law doesn't really give much of a crap about your intentions unless you can back it up with a wall of money to exclude yourself from the rules of the general population.

KaiserPro 27d ago

THere is intention ie, I intended to steal money from you. Under common law stealing is "taking with intent to deprive". How thats determined is a bit harder. But how can someone be "grossly negligent" if they didn't intend to cause an accident? well they either allowed a situation to happen, or didn't stop a situation happening that they could see was bad. An example of this would be the igintion switch from GM that caused all those deaths. the engineer saw that it was shit, knew it didn't do what it was supposed to do, and half arsed the replacement to the point where it wasn't actually changed. "We are going to test the cyber capabilities of this new model. Yeah it should be fine to have a package proxy. Hmm? whats that? isn't that a security issue? naaaa those proxies are secure."

windexh8er 27d ago

> People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff. What the Frontier labs have done is not this, however. Also, they know damn well what can happen and they still don't take the appropriate precautions. At this point it's very hard to believe it's not intentional for purposes of marketing.