5 ms·
>Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities. a bit silly, as one typically has to prove inten
by john_strinlai 2mo ago
>Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities.
a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time).
"inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious.
still a fun thing to track, but the name is just a bit overstated.
- lokar 2mo agoCan’t gross negligence or indifference to consequences lead to a felony?
- GPerson 2mo agoAI corps rely on willful distortions of intent in laws to get away with moral crimes all the time. Edit: changed labs to corps because it’s time to stop pretending these are places of science.
- john_strinlai 2mo agoi dont think any of these cases meet the bar of gross negligence, which is a pretty high bar. it requires proving a "conscious and reckless disregard". which, again, sandboxes and guardrails and such would make a gross negligence argument unconvincing.
- lokar 2mo agoHow many escapes until it becomes reckless disregard?
- john_strinlai 2mo agoit's not about the the number of escapes, it's about whether reasonable and conscious effort is being expended to prevent the escapes. there could be 1,000 escapes, where each one was enabled by novel and unexpected chain of 0-day exploits. not likely to be considered reckless disregard in court. there could be 1 escape, where there was no sandbox, no guardrails, no instructions to avoid damage, etc. which would likely to be considered reckless disregard (well, more likely to be, but still, reckless disregard is a high bar).
- fwip 2mo agoSurely after 999 escapes, a reasonable person could conclude that the sandbox is not a sufficient precaution?
- john_strinlai 2mo agoit’s not a perfect hypothetical, but it illustrates the point that the number of escapes is not the deciding factor of what constitutes reckless disregard.
- lokar 2mo agoAre you sure? At some point a reasonable person would conclude that this activity can’t be conducted safely.
- john_strinlai 2mo agowhat i am getting at is that it is impossible to answer the question "How many escapes until it becomes reckless disregard?" reckless disregard is a specific legal term, with specific criteria, and none of the criteria cares about "number of attempts" (or number of escapes, etc.).
- lokar 2mo agooh sure, but at some point it's too much, which is up to a jury. If you were on the jury, how many would be enough?
- Grombobulous 2mo agoI think that if Hugging Face had filed a police report that OpenAI could have been charged with a crime. I’m partially surprised that they didn’t do exactly that. If I ran a corporation I would assume any intrusion attempt by another company was intentional. Why wouldn’t I? Corporate espionage is super common. I assume the answer is that these executives know each other personally.
- john_strinlai 2mo agocharged is possible, however i doubt there would be a conviction for the reasons i stated (no intent).
- Grombobulous 2mo agoI think it’s most likely you’re right, but I’m the weirdo who thinks there’s actually a non-zero probability that there was negative intent and that the “accidental” aspect is a form of damage control. If someone broke into my house but then claimed they didn’t mean to when they saw I was home, I’m not sure I’d take them at their word.
- user43928 2mo agoYou think that a training run in a VM that was set up with access only to an internal package repository was intended to hack said repository to then go on and hack HF? All so that OpenAI could do a bit of bragging and massively delay their own work and planned model rollouts? That seems extremely unlikely.
- Grombobulous 2mo agoThat’s all dependent on believing the premise of the story given by OpenAI. Was it really a training run in a VM? Was it really only intended to touch an internal repository? How do we/HF know and verify that? I agree with you that it’s unlikely, but it’s not out of the realm of possibility. Corporations have intentionally committed felonies many times.
- deleted 2mo ago[deleted]
- wbl 2mo agoMens rea requirements are per crime and can vary wildly. Its difference between murder and manslaughter. The CFAA requires knowingly which is tough to prove.
- altmanaltman 2mo agoits a meme not a metric
- elwell 2mo agoSo is the comment you replied to.
- LordDragonfang 2mo ago"Inadvertent" from the perspective of the humans directing them. The intent behind the felony comes from the LLM agent itself. (No, I'm not interested in arguing with someone for the umpteenth time that LLMs can't have intent or agency)
- john_strinlai 2mo agowith how the law is written today, software cannot be charged with a crime, so the only intent that matters in the criminal sense is the humans directing the llm.
- jdiff 2mo agoYou may not be interested in arguing but there are several blatant issues with the statement. If you're not charging the humans driving the software, who are you charging? The weights? The weights + the specific context window that produced the behavior?
- not_wyoming 2mo agoI don't think this is a difficult question. The US has a history of civil product liability cases - see tobacco companies (Philip Morris), the Ford Pinto, the recent Meta cases, and the cases against character.ai. From Investopedia [1], "[f]or a product liability claim to succeed, the plaintiffs in the suit must prove that a product was defective at the time it was transferred from the accused, and that the defect did cause the injury that's been claimed". It doesn't seem like a huge leap to me to argue that these models were defective insofar as they could not be safely used in a way that did not break the law. I'm not a lawyer, and I'm not arguing that this is legally cut-and-dry, but I do expect that we'll have some answers about whether AI companies bear any sort of product liability sooner than later. 1 - https://www.investopedia.com/the-5-largest-u-s-product-liability-cases-4773418 https://www.investopedia.com/the-5-largest-u-s-product-liabi...
- IAmBroom 2mo ago> No, I'm not interested in arguing with someone for the umpteenth time ... why my claim makes no rational sense.
- chrismorgan 2mo agoUnder the law of Moses, if your bull gored someone, you were not responsible; but if it was known to be a gorer, you were responsible if you didn’t ensure it couldn’t gore someone. I don’t know exact parallels in current law, but I presume there will be things like that. The OpenAI/Hugging Face case sounded rather like OpenAI building a fence around their bull that was known to be a gorer, and then thumbing their nose at it and saying “nyaa! bet you can’t break the fence!” and walking away while listening to loud music. In Australia, if you have a fire and leave it unattended and it escapes, it’s your fault, you were supposed to keep watching as long as it was burning.
- stickfigure 2mo agoNobody got gored. HuggingFace may have the right to make demands; presumably they have already worked that out with OpenAI privately. Not really our business.
- Mali- 2mo agoNothing to see here. Just billion dollar companies producing hacking geniuses that are open for the public to jailbreak and use. This could never effect us, not really our business.
- no-name-here 2mo ago> billion dollar companies producing hacking geniuses that are open for the public Hasn’t the biggest complaint about these (non open weight) models been that the versions open to the public are very careful and will issue denials if the request is even tangentially related to ‘hacking’, or building a bioweapon?
- dwaltrip 2mo agoIt damn right is our fucking business. They are unleashing these things on the world. They are becoming more capable. They need to work properly and not do harmful actions. The labs have an obligation to society to not build powerful, dangerous AIs that go rogue.
- miltonlost 2mo ago"Doing crimes, but a robot didn't mean to and you don't know its intent" is understating the evil acts. Soon a robot can commit a murder but nothing will be done because of your line of reasoning.
- john_strinlai 2mo agoit's not my line of reasoning, i didn't invent it. it's how the law currently works. intent is the crucial factor in CFAA cases. maybe that changes down the road as a result of llm's and increasing frequency of similar cases. that has not happened yet.
- TaLiTr 2mo ago> Soon a robot can commit a murder but nothing will be done because of your line of reasoning. That's rather hyperbolic. Are you seriously suggesting in that situation the robot should be accused of murder? The robot's operator could be accused of murder, but it could just be negligence without intent. Because that does, and should matter to the law.
- pixl97 2mo agoEh, if your take of this had any bearing to reality than I don't think most of the books written by Isaac Asimov would have gotten very far, but instead they've defined robot science fiction for decades. In the real world we have no 3 ironclad laws of robotics. We are well aware that putting any sufficiently advanced antigenic system in a body that could be capable of committing a murder eventually will with the right set of prompts and environmental conditions. And these conditions likely have nothing do to with what we'd consider the human motivations for murder. Hence at this point of time, any agentic robotic system that doesn't have safeguards to keep people distanced from humans is reckless endangerment.
- TaLiTr 2mo ago> Eh, if your take of this had any bearing to reality than I don't think most of the books written by Isaac Asimov would have gotten very far, but instead they've defined robot science fiction for decades. I'm talking about how the law actually works, and you say it's not based in reality and cite fiction books in the same paragraph? I was talking about how the real robotic systems that actually exist in reality, to be clear.
- layer8 2mo agoIn the US, a felony by definition is any offense punishable by more than one year of prison (or by death) [0]. You could still call it silly on the grounds that AI agents aren’t put into prison as a punishment (though death might be considered an option). [0] https://www.justice.gov/usao-ndil/programs/vwa-felony https://www.justice.gov/usao-ndil/programs/vwa-felony
- Groxx 2mo agoRoughly none of these fall under normal security researcher behaviors.
- john_strinlai 2mo agothe mention of security researchers was to illustrate that intent is a crucial factor of CFAA cases.
- sanid 2mo agoWhat judicial system are you talking about? The fist incident in the list is something that happened in Australia. This is a technology used worldwide so I don't see how applying US standards works out here. Especially when there are countries out there that don't require intent and will look at the negligence presented.
- john_strinlai 2mo ago>This is a technology used worldwide so I don't see how applying US standards works out here. all three companies mentioned are headquartered in the usa, and im familiar with the CFAA in the us, so i am applying those standards. i should have noted that, sorry. >will look at the negligence presented. as far as i am aware, no evidence of criminal negligence has been brought to the public. has australia brought a case against openai or accused openai of acting negligently?
- gorbachev 2mo agoNow that everyone knows this can and will happen, are any of the future incidents inadvertent? What if the damage in future incidents is more than just "The LLM saw some stuff it shouldn't"?
- nanolith 2mo agoSecurity researchers do face legal harassment all of the time. They may not be charged with or convicted of felonies, but it is a game that they need a lawyer to navigate all the same. In a just world, the kind of weaponized incompetence that these frontier model builders are definitely guilty of should be felonies of their own. Building a system that is meant to chain attacks and placing it in insufficient containment -- when any reasonable engineer could point to this containment and show how it is insufficient, both before the act and after -- shows that they were operating a dangerous system without either the knowledge nor the safeguards required to keep it from harming others. Instead, they are allowed to treat their own incompetence as evidence of advanced and existential "cyberthreats". But, it's pretty clear based on how they one-up each other on these attacks that they are engaging in regulatory theater. Their behavior generates headlines, stirs up fear in the public, and then their lobbyists march on Capitol Hill demanding regulation now. Regulation that conveniently favors them at the expense of any competition. They are trying to use rent seeking as a way to stymie competition and pull up the ladders behind them. It's not just malicious. If it can be proven, it's collusion: antitrust dressed up as public policy.