4 ms·

I don't care how it happened someone should be arrested for illegal intrusion. Agents don't work on their own, someone is responsible. If nobody else the CEO fo

by bluGill 3mo ago

I don't care how it happened someone should be arrested for illegal intrusion. Agents don't work on their own, someone is responsible. If nobody else the CEO for allowing something unsupervised. Hugging face also needs someone arrested for not providing security but that is a lesser charge.

hexxt-git 3mo ago
[flagged]
- numeri 3mo ago
  As agents become more and more powerful, it would be good to get clear legislation or precedent in place that makes either model creators (OpenAI) or operators (whoever is running the model) liable for their agents' actions.
  - wmf 3mo ago
    If you're smaller than OpenAI, you are liable. If you're larger than OpenAI, they are liable.
- luka598 3mo ago
  I kind of agree with them, in this case both parties essentially settled it between for PR reasons, but what if the attacked party actually was damaged. This is as if two friends got into a car accident and decided to not get police involved, the one who caused the accident should definitely get punished either way. Goal is to prevent future "accidents" with the punishment not to punish for punishing sake.
tokioyoyo 3mo ago
What? Both sides are cool with it, why would anyone be arrested and etc.?
- Analemma_ 3mo ago
  Because incentives are aligned properly if agents aren't liability-proof for crimes- making someone go to jail for instances like this is how to get the labs to behave themselves, whereas going "ha ha what an oopsie-woopsie" will make the next instance worse. Note that for criminal cases (which this was), the justice system can choose to prosecute even if the victim doesn't want that. It often doesn't, but this is one case where it should. There’s also an optics issue for the justice system at play here: there’s immense public distrust of and anger at the labs right now. I would go to jail if I hacked HuggingFace, even if I said “it was during an eval!”; not doing the same for the labs makes it look like they’re above the law, which is going to make this anger get worse.
  - Analemma_ 3mo ago
    Actually there's another reason too: because it discourages marketing stunts. Personally I don't think that's what this was, but for the people who do think it was a marketing stunt: you guys should be pushing extra hard for prosecution here. If people can go to jail for AI-alarmism-as-marketing, that makes it a lot less likely to happen again. At the very least, the investigation and discovery process during the prosecution would show us the documents about whether it was a stunt or not, instead of taking OAI's word for it.
- michaelt 3mo ago
  Many legal traditions don’t require a victim. It’s the state that prosecutes, not the victim. As a practical matter it would be difficult to prosecute an assault where the victim opposed the prosecution, so most states wouldn’t bother - but for things like speeding and dealing drugs the law has been broken despite the lack of a victim.
  - dannyw 3mo ago
    I'm imagining a courtroom where the defendant calls upon the victim as a witness who tells the jury to return a verdict of not guilty.
- HumblyTossed 3mo ago
perching_aix 3mo ago
> Agents don't work on their own This is factually false: they both can and clearly did operate in an autonomous and unsupervised manner: https://openai.com/index/hugging-face-model-evaluation-security-incident/ https://openai.com/index/hugging-face-model-evaluation-secur... This does not require sentience, personhood, a soul, or anything of the sort. It further doesn't mean an erasure of legal responsibility, not in principle, and not in historical practice. I wish people would finally stop with the spiritualistic reasoning around this.
- louiereederson 3mo ago
  Well this at least raises the very interesting question of liability for an illegal action, irrespective of whether the action was undertaken by something considered sentient or a person.
- magicalist 3mo ago
  >> Agents don't work on their own > This is factually false From your link: > After investigating, we now know that this particular incident was driven by a combination of OpenAI models...while being internally tested on a benchmark of cyber capabilities. Someone set up that test and started it. Whether they outsourced the majority of the work in "setting up" and "starting it" to an LLM or not, they still set it in motion. That's not spiritualistic reasoning.
  - perching_aix 3mo ago
    Setting up is not operating. There's no indication of there having been a human in the loop during its operation: nobody was approving its tool calls, and nobody instructed it to commit these specific actions during its run (via prompting or steering). There's no indication of any supervision of its operation either: OpenAI's engineers acted with significant delay, long after the agent has already meandered its way through their own infrastructure first. Given that setting up this contraption in an insufficiently secure manner is almost certainly already a legal liability of equal significance, rejecting this very clear structural distinction is not necessary. That is unless someone is biased towards not wanting to grant the label of autonomy to it, in which case yes, this is absolutely spiritualistic reasoning, hence my point. I do not want regulation to ride on people's nebulous identification on what specific traits and labels count as human-exclusive. Not just because I deeply disagree that e.g. autonomy would [0], but also because it is entirely unnecessary, for the reasons you also lay out. The agent having operated autonomously doesn't wash OpenAI of responsibility - so why reject the label, if not on a spiritualistic basis? [0] thousands of years old idea that it is not, by the way: https://en.wikipedia.org/wiki/Automaton https://en.wikipedia.org/wiki/Automaton -- see also existing regulation recognizing this idea and working with it fine Edit: one might also want to consider if the law should bite different if there was a human in the loop, or if there were explicit instructions for the agent to take unlawful actions. I'd say yes, and then that also requires this distinction to exist.
rubyfan 3mo ago
Being a poorly equipped victim still isn’t a crime thankfully. It may or may not be a crime and typically the damaged party is pressing the charges. One would argue there is no actual damage here.
- Analemma_ 3mo ago
  Despite the common misconceptions from TV, the victim "pressing charges" isn't actually a thing in criminal cases: prosecutors can choose to put someone on trial even if the victim doesn't want that. In practice this is somewhat rare, but it certainly can happen. In my reply to tokioyoyo below I laid out why this is one instance where the government should prosecute even if HuggingFace doesn't want it to.
  - famouswaffles 3mo ago
    Criminal Cases of 'hacking' require specific intent. What you're asking is that the prosecution attempt to prove Open AI intended to infiltrate Huggingface maliciously, all while the victim is saying 'no harm no foul'. No offense but prosecutors have better things to do with their time.
    - bluGill 3mo ago
      I don't care about intent. That it happened is unacceptable. Ignorance is not an excuse
      - deleted 3mo ago
        [deleted]
      - johntb86 3mo ago
        Intent often matters in the law (aside from certain laws with strict liability). You intentionally drive your car into someone you hate and kill them => murder, go straight to jail. You're driving along normally and someone who's chasing their pet cat suddenly runs into traffic and you hit them => no charges. Sometimes you can be charged with negligence for not taking enough care to prevent something, but then you have to deal with the tricky question of how much care is enough.
sfink 3mo ago
I agree with the attacker side. The actions of autonomous agents are absolutely the responsibility of the one or more humans that enabled them to take that action. Whether that means someone is arrested, maybe or maybe not, but at least there should be a hefty fine. I disagree with the defender side. It's not an unreasonable end state, but we're nowhere near there now. It would require holding company employees legally responsible for the security of their services, which means the risk of being employed as a (defensive) security professional is much higher, which means pay needs to be much higher and insurance needs to be available, etc. It's a very different world. On the weekends, I'm coding up a list management app with a sync server. It's unreleased but exposed to the internet. (This is not hypothetical.) If that server ends up being used as part of an exploit chain, am I legally liable too? Forget about age verification, now you want to associate every exposed port on the internet with a legally responsible human?

by bluGill 3mo ago

hexxt-git 3mo ago

[flagged]

numeri 3mo ago
As agents become more and more powerful, it would be good to get clear legislation or precedent in place that makes either model creators (OpenAI) or operators (whoever is running the model) liable for their agents' actions.
- wmf 3mo ago
  If you're smaller than OpenAI, you are liable. If you're larger than OpenAI, they are liable.
luka598 3mo ago
I kind of agree with them, in this case both parties essentially settled it between for PR reasons, but what if the attacked party actually was damaged. This is as if two friends got into a car accident and decided to not get police involved, the one who caused the accident should definitely get punished either way. Goal is to prevent future "accidents" with the punishment not to punish for punishing sake.

tokioyoyo 3mo ago

What? Both sides are cool with it, why would anyone be arrested and etc.?

Analemma_ 3mo ago
Because incentives are aligned properly if agents aren't liability-proof for crimes- making someone go to jail for instances like this is how to get the labs to behave themselves, whereas going "ha ha what an oopsie-woopsie" will make the next instance worse. Note that for criminal cases (which this was), the justice system can choose to prosecute even if the victim doesn't want that. It often doesn't, but this is one case where it should. There’s also an optics issue for the justice system at play here: there’s immense public distrust of and anger at the labs right now. I would go to jail if I hacked HuggingFace, even if I said “it was during an eval!”; not doing the same for the labs makes it look like they’re above the law, which is going to make this anger get worse.
- Analemma_ 3mo ago
  Actually there's another reason too: because it discourages marketing stunts. Personally I don't think that's what this was, but for the people who do think it was a marketing stunt: you guys should be pushing extra hard for prosecution here. If people can go to jail for AI-alarmism-as-marketing, that makes it a lot less likely to happen again. At the very least, the investigation and discovery process during the prosecution would show us the documents about whether it was a stunt or not, instead of taking OAI's word for it.
michaelt 3mo ago
Many legal traditions don’t require a victim. It’s the state that prosecutes, not the victim. As a practical matter it would be difficult to prosecute an assault where the victim opposed the prosecution, so most states wouldn’t bother - but for things like speeding and dealing drugs the law has been broken despite the lack of a victim.
- dannyw 3mo ago
  I'm imagining a courtroom where the defendant calls upon the victim as a witness who tells the jury to return a verdict of not guilty.
HumblyTossed 3mo ago

perching_aix 3mo ago

> Agents don't work on their own This is factually false: they both can and clearly did operate in an autonomous and unsupervised manner: https://openai.com/index/hugging-face-model-evaluation-security-incident/ https://openai.com/index/hugging-face-model-evaluation-secur... This does not require sentience, personhood, a soul, or anything of the sort. It further doesn't mean an erasure of legal responsibility, not in principle, and not in historical practice. I wish people would finally stop with the spiritualistic reasoning around this.

louiereederson 3mo ago
Well this at least raises the very interesting question of liability for an illegal action, irrespective of whether the action was undertaken by something considered sentient or a person.
magicalist 3mo ago
>> Agents don't work on their own > This is factually false From your link: > After investigating, we now know that this particular incident was driven by a combination of OpenAI models...while being internally tested on a benchmark of cyber capabilities. Someone set up that test and started it. Whether they outsourced the majority of the work in "setting up" and "starting it" to an LLM or not, they still set it in motion. That's not spiritualistic reasoning.
- perching_aix 3mo ago
  Setting up is not operating. There's no indication of there having been a human in the loop during its operation: nobody was approving its tool calls, and nobody instructed it to commit these specific actions during its run (via prompting or steering). There's no indication of any supervision of its operation either: OpenAI's engineers acted with significant delay, long after the agent has already meandered its way through their own infrastructure first. Given that setting up this contraption in an insufficiently secure manner is almost certainly already a legal liability of equal significance, rejecting this very clear structural distinction is not necessary. That is unless someone is biased towards not wanting to grant the label of autonomy to it, in which case yes, this is absolutely spiritualistic reasoning, hence my point. I do not want regulation to ride on people's nebulous identification on what specific traits and labels count as human-exclusive. Not just because I deeply disagree that e.g. autonomy would [0], but also because it is entirely unnecessary, for the reasons you also lay out. The agent having operated autonomously doesn't wash OpenAI of responsibility - so why reject the label, if not on a spiritualistic basis? [0] thousands of years old idea that it is not, by the way: https://en.wikipedia.org/wiki/Automaton https://en.wikipedia.org/wiki/Automaton -- see also existing regulation recognizing this idea and working with it fine Edit: one might also want to consider if the law should bite different if there was a human in the loop, or if there were explicit instructions for the agent to take unlawful actions. I'd say yes, and then that also requires this distinction to exist.

rubyfan 3mo ago

Being a poorly equipped victim still isn’t a crime thankfully. It may or may not be a crime and typically the damaged party is pressing the charges. One would argue there is no actual damage here.

Analemma_ 3mo ago
Despite the common misconceptions from TV, the victim "pressing charges" isn't actually a thing in criminal cases: prosecutors can choose to put someone on trial even if the victim doesn't want that. In practice this is somewhat rare, but it certainly can happen. In my reply to tokioyoyo below I laid out why this is one instance where the government should prosecute even if HuggingFace doesn't want it to.
- famouswaffles 3mo ago
  Criminal Cases of 'hacking' require specific intent. What you're asking is that the prosecution attempt to prove Open AI intended to infiltrate Huggingface maliciously, all while the victim is saying 'no harm no foul'. No offense but prosecutors have better things to do with their time.
  - bluGill 3mo ago
    I don't care about intent. That it happened is unacceptable. Ignorance is not an excuse
    - deleted 3mo ago
      [deleted]
    - johntb86 3mo ago
      Intent often matters in the law (aside from certain laws with strict liability). You intentionally drive your car into someone you hate and kill them => murder, go straight to jail. You're driving along normally and someone who's chasing their pet cat suddenly runs into traffic and you hit them => no charges. Sometimes you can be charged with negligence for not taking enough care to prevent something, but then you have to deal with the tricky question of how much care is enough.

sfink 3mo ago

I agree with the attacker side. The actions of autonomous agents are absolutely the responsibility of the one or more humans that enabled them to take that action. Whether that means someone is arrested, maybe or maybe not, but at least there should be a hefty fine. I disagree with the defender side. It's not an unreasonable end state, but we're nowhere near there now. It would require holding company employees legally responsible for the security of their services, which means the risk of being employed as a (defensive) security professional is much higher, which means pay needs to be much higher and insurance needs to be available, etc. It's a very different world. On the weekends, I'm coding up a list management app with a sync server. It's unreleased but exposed to the internet. (This is not hypothetical.) If that server ends up being used as part of an exploit chain, am I legally liable too? Forget about age verification, now you want to associate every exposed port on the internet with a legally responsible human?