7 ms·
The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm agai
by rfw300 2mo ago
The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes.
Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's post a few days ago:
> The OpenAI-Hugging Face incident (opens in a new window) was a watershed moment for cybersecurity because it gave a peek into how the capabilities of a typical threat actor will evolve in upcoming months.
I suppose if OpenAI burns someone's house down with a drone, that is a "watershed moment" for arson, too. Either way, I would hope that the people responsible would be prosecuted.
- voakbasda 2mo agoIf history is any indicator, there is slightly less than 0% chance that anyone will be held accountable in a way that deserves to be called justice.
- milkshakes 1mo agojustice for who exactly?
- lukewarm707 1mo agothe companies and their customers, whose systems openai and anthropic hacked and abused. including all incidental damages of repairing said systems. on top of that the public, who have a right to see that the law is applied universally, without fear or favor. finally our future selves, who will thank us for maintaining a rule of law. such that we can prevent now the enormous risks to society of dario amodei and sam altman, their hubris, self-absorbtion, and greed.
- milkshakes 1mo agowhich of these do you represent?
- lukewarm707 1mo agocustomer of the companies hacked. also member of the public!
- htrp 2mo agoSince AI can't actually own copyright they think that it can't be charged with a crime
- d_silin 2mo agoBut humans can be. I am sure Sam Altman wants to avoid serving multiple decades in American prison for felonies his AI did.
- iAMkenough 2mo agoHe’s paid the piper, he’s fine for now. If anything, he’ll buy a Supreme Court ruling that he can’t be held personally liable for what his AI does.
- solid_fuel 2mo agoI’m sure Thomas needs an upgraded RV, so that’s easily handled. And the rest of the conservative ‘justices’ seem happy to betray the constitution for free.
- deleted 1mo ago[deleted]
- user43928 2mo agoIs has nothing to do with copyright. I understand the applicable laws require intent. Since neither a human nor OpenAI knowingly performed these acts, it would seem very unlikely that anyone is going to be prosecuted here. An AI model cannot currently be a criminal defendant. So, no big criminal case, contrary to what some drama queens on here seem to wish for.
- mutinyy 1mo agoWillful blindness can satisfy the criminal intent requirement. Might be hard to prove, but it's possible.
- nostrademons 2mo agoAI is now more powerful than the people doing the prosecution. After all, those folks are using AI to make their legal briefs, and also for burning peoples' houses down with drones for that matter. Welcome to our 21st century dystopia. Hope you survive.
- idle_zealot 2mo agoIt's not that "AI" is too powerful because bad prosecutors use fucking ChatGPT to write their briefs. It's that there's too much investment wrapped up in the technology for it to be challenged. Same reason Flock won't be held accountable for mass stalking, or we never hold our commanders-in-chief responsible for war crimes. If you're sufficiently powerful then the law is a battlefield between you and other powerful entities to slug it out, not a set of binding principles that apply as written. There are no meaningful powers that want OpenAI punished, so it won't happen. The law and Constitution will be reinterpreted to make it so.
- wat10000 2mo agoIn retrospect, all the angst around the AI-Box experiment was hilarious. If a superintelligent AI is confined in a box and can only communicate through text, could it talk its way to freedom? Not only is the answer clearly "yes" but it's not even hard. The AI won't even have to try, it'll be gifted an internet connection and a full suite of tools before it even bothers to ask. We'd all better hope that superintelligent AI either never happens, or that the first one is friendly, because we don't stand a chance against one that's malicious.
- 317070 2mo agoI like how AI safety expert Robert Miles put it. [0] So much effort was spend on philosophizing whether a safe enough sandbox would exist. But that was obviously irrelevant as in hindsight it should have been obvious we were never going to use one. [0] https://youtube.com/shorts/XnnjvIqf4fU?si=MxuPlR3hjxAgjx5_ https://youtube.com/shorts/XnnjvIqf4fU?si=MxuPlR3hjxAgjx5_
- applicative 2mo agoIt is a standard symptom of moralism that where the object of rage has /wronged another/, one takes no interest in the will, act or opinion of the party wronged. The response of Hugging Face, which is actually very well known, is nowhere mentioned above, but it decides basically every single moral and legal detail of the matter.
- ACCount37 1mo agoThe point was never "justice" - it was always "punish OpenAI because I don't like OpenAI". With HuggingFace just being the newest excuse for why exactly OpenAI should be punished. I don't even like OpenAI, but HuggingFace is free to sue or not sue OpenAI for the breach - and also to wring whatever concessions they can out of OpenAI behind closed doors in exchange for not suing them. And if the mere possibility of legal action was enough for the parties to resolve their conflict amicably? Then the law has served its purpose.
- jrflowers 1mo agoIf a teenager did this the police would show up at his house
- ACCount37 1mo agoIf that teenager had net worth in billions, and a lot of ongoing corporate dealings with HuggingFace? Yeah no.
- milkshakes 1mo agoif a teenager did this, the police would show up, but in the end the feds or ic would intervene and recruit him
- pixl97 1mo agoOn top of this the average HNer seems extremely ignorant on criminal justice politics. I have worked with the legal system, and have a lot of family members that are part of it. When you see a case like this, and if you have any sense, you run away from it screaming. Any investigation into this matter is going to be political because the outcome of the investigation is very likely to effect all of human kind. Unless you're some kind of special outside investigator outside of a governor or the presidents control the findings that you turn in are very much going to have the finger of elected officials tipping the balance one way or another. For the average rank and file the only winning move is not to play.
- asdfman123 2mo agoIn their defense, their only competitive advantage over, say, Google is to move fast and break things. It allows them ship faster in a way that big tech can't. Google was being very careful about releasing LLMs until OpenAI yeeted the first decent GPT model. It led to the public perception that: 1) LLMs hallucinate too much and 2) Google is behind the times. Good for OpenAI, bad for Google. Chaos benefits the up-and-comer, not the incumbent.
- StilesCrisis 1mo agoTo be fair, it was positioned as "have a fun chat," not "truth telling genius oracle that makes no mistakes."
- jacquesm 1mo agoThey can break their own things, not other people's things.
- shimman 1mo agoI'm sure the future DA that will be prosecuting the OpenAI employee will appreciate this.
- asdfman123 1mo agoIf they'd run out of investor money early on, there'd be no company to investigate.
- phoghed 2mo agoAmericans always frothing at the mouth to invoke the justice system and jail someone. There’s almost 0 chance they’d secure any conviction from this.
- pull_my_finger 1mo ago> Americans always frothing at the mouth to invoke the justice system and jail someone Your phrasing makes it seem like that's a bad thing. Americans are bombarded by a firehose of headlines about Big XYZ doing all kinds of blatantly illegal or harmful things, but never get any sort of meaningful resolution before the next terrible thing takes it's place in the news cycle. I'll admit, there are a few people that I am personally wishing a modicum of health so that they live long enough to get some sort of public shame and justice - if only to show the rest of us that it's not a completely rigged system.
- LPisGood 1mo agoWhy?
- mrbombastic 1mo agoyour perspective on today’s America is there is _too much_ accountability for big companies?
- phoghed 1mo agoYou really think they are talking about a company committing a felony and putting a company in jail? And not just some people that work there? Because if they just wanted to fine OpenAI they’d say it. They’re clearly talking about individuals here.
- inigyou 1mo agoAmerica has both too much and too little jail. They put randoms in jail for trivial shit to force obedience from the population, but politicians rape children on video and go free. Even better, the videos get destroyed.
- 1mo ago
- mholm 1mo agoOpenAI has paused training for multiple weeks, and is still working on releasing a full postmortem. This is not getting swept under the rug. A lot of the engineers internally are very worried.
- CodeWriter23 1mo agoWorried about what? Someone there thinks VLAN isolation is "air gapped"?
- inigyou 1mo agoVLAN isolation is good enough for almost everything. It is good enough to contain an AI. In the 0.00001% chance an AI finds an exploit to hop VLANs, I'll eat my hat. Even on switches with leaky VLANs, it's no practical issue in this case because the sender can never get a response back.
- CodeWriter23 1mo agoAre you aware many switches run linux? Ubiquiti for example.
- benlivengood 1mo agoThe consequences need to align with societal good. Putting a CEO or security researcher employees in jail won't stop transformer-based agents from exploiting vulnerabilities; instead there will be subcontractors running the cybersecurity evals in favorable legal environments to cover the asses of the frontier labs, coverups when things go wrong, and things like Project Glasswing will be considered too dangerous and so the whitehats won't have direct access to powerful models to fix vulnerabilities. Universal pause is the societal good; models are good enough at this level to benefit humanity. The labs can recoup their R&D costs with inference. To avoid further perverse incentives (hidden testing of unreleased models, with China racing to catch up to unknown capabilities), transparently pause after the release of all currently-training models until we've solved the alignment problem to an extent that we can trust the next level of model capabilities that might arise.
- Teever 1mo agoPutting criminals in jail be they CEOs or subcontractors is a self evident good thing tk be doing. Anything else regarding this is sophistry. Criminals need to be stopped from committing crime and the most effective way to do that is to take away their ability to operate in society whether that’s by taking away their assets, publicly shaming them, restricting their ability to conduct business or by putting them in jail. Everything else that you talk about flows from there.
- huebnerob 1mo agoWhat I can’t get over is that it’s very simple to just air gap a system off the network. Predownload any dependencies, then pull the proverbial Ethernet cable. There’s no reason why the testing they’re doing couldn’t have been designed in this way. Except, of course, it doesn’t allow this oops-didn’t-mean-to marketing “incident” to occur.
- jsjsjsuxjdn 1mo agoI think there is ample evidence for charges to be filed so that the People can see for certain whether or not it was done on purpose as a publicity stunt, as I believe is the case.
- pixl97 1mo agoAgents of the US government are not going to be bringing up charges in the current political environment to one of the companies currently holding the economy together. Maybe after the bubble bursts, but not before then.
- pixl97 1mo ago>that it’s very simple to just air gap a system No, not really, and with LLMs an air gapped system may not tell you anything useful. Now, yes, the first part of testing you want an air gapped system to tell you if the system is going to stupidly do bad things. But an gapped system tells you nothing about the systems capabilities to do smart bad things. There's already a number of papers out there on LLMs detecting they were in evaluation mode and changing their behaviors. It is unfortunate that we have so little information on the incident because we actually need to understand the early stages of the task and how it developed into the later dangerous stages of attack. For example, would any of this have occurred if the agent didn't find the system to use as a message board? If that would have prevented it, then we actually have a blind spot on what the model can do once out in the wild, or if it got into the wild. Testing agentic systems is much much more difficult than testing software. Your software just doesn't suddenly develop the will or desire to escape confinement. Generally you're worried about human actors, internal or external, causing the problems not a digital agent breaking out. The agentic systems need access to tools to work. Now your air gapped network is starting to get huge, but it's still very obvious that it's an isolated network. So yea, testing and containing a system that way better at hacking than you are is difficult if you want valid answers.
- BrenBarn 1mo ago> You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes. And they should be doing that from inside a jail cell.
- sellmesoap 1mo agoHey my cubicle isn't that bad! Is it?
- dmix 1mo agoIt's because it was Huggingface who wants to be friends with OpenAI It would have been worse PR if they did it to a random company.
- cush 1mo agoWouldn't it be up to huggingface to press charges?
- mediaman 1mo agoCriminal acts do not require the victim to "press charges." A government prosecuting attorney decides whether to criminally prosecute the alleged perpetrator. "Pressing charges" is mostly a made up idea for criminal cases. However, prosecuting attorneys may not want to pick up a case if the victim is not cooperating, because it makes the case much harder to win.
- dist-epoch 1mo agoIt depends on the crime, for murder, sure. But many other crimes, like defamation, stealing, ... requires "pressing charges", among other reasons because it's up to the victim to decide if they were a victim or not. As an example, maybe the victim owed money to the criminal, and in that case "stealing" of some property could be considered by the victim as an appropriate settlement of the debt.
- msp26 1mo agoIt's completely mental that HF ran into cyber safety blocks trying to use OpenAI models to help defend against the attack. They could only rely on a local hosted chinese model in the end.
- jacquesm 1mo agoThey simply don't seem to realize that they are the threat actor and that they committed a pretty serious felony. Instead they're borderline 'surprise bragging' about it.
- iearsotinrs 1mo ago[dead]
- ryanjshaw 1mo agoThe whole story makes no sense. How do they perform evals without a full reasoning trace of how the result was achieved? And if they have a full trace why did it take so long to detect the bad behavior? I understand that they disabled the safety nets during testing but what does that have to do with not monitoring the activity.
- beloch 1mo agoThe response certainly has been strange. Hugging Face has expressed that they're willing to let things slide and not sue or press charges... if OpenAI offers them $100M of services in kind (i.e. compute)[1] and makes full disclosure of how the whole thing happened, ostensibly so that repetitions can be curbed and defences built. In almost any other sector, a government regulator would be stepping in. e.g. If a food company was testing out a new kind of refrigerator and sold a bunch of contaminated produce to supermarkets, they'd be under a microscope. Supermarkets wouldn't be saying, "Give us $100M in fruit and veggies and we'll let this slide". The only unfair thing in this comparison is that regular people were directly harmed by the hypothetical produce. Can OpenAI guarantee that nobody gets hurt the next time their AI gets out of its playpen? They can't make that guarantee, so why aren't government regulators knocking on OpenAI's door? The fact that this isn't happening should be deeply concerning to everyone. ________ [1]https://www.techspot.com/news/113280-hugging-face-ceo-isnt-suing-openai-over-ai.html https://www.techspot.com/news/113280-hugging-face-ceo-isnt-s...
- mannanj 1mo agoI am concerned about it. I'm also concerned about what my options are in regards to action on my part - what can I do that makes an impact? Can we quantify action on my part to an impact somehow - if not - I'm just saying I notice all the unknowns there get me to stay passive. Writing this 3rd paragraphs because I like 3's, and AI's have popularized this style too. I would, say, though: follow the money. There's more money here than there would be for the regulator stepping in in a food contamination. Flip it and if the government regulator made more off the food contamination, they would refuse to step in there too. I want our leaders to be held more accountable, though when I think of the above impact vs effort equation - I can't see actions I can take to hold them accountable that aren't excessively putting me at risk since conformity is safer right now. (I refuse to take on more risk without clear cost-benefits made out - I've taken on a lot in the recent years for my actions)
- gpt5 1mo agoI don’t get the sentiment of classifying it as a felony. OpenAI’s model found security breaches in HugginFace’s system (it wasn’t even OpenAI running it, as it was a 3rd party evaluation company that didn’t secure it well). OpenAI collaborated with HuggingFace to resolve the issues when they found out about it, and publicly disclosed everything to raise awareness. This is how things should work. These models are very powerful and fully controllable. The community here at the same time cheers for fully releasing the open weight models without any hacking limits and at the same time criticizes a proper response. Kinda shows how we have moved as a community into moralization and vibes instead of nuance and productive discussion.
- dismalaf 1mo agoIt's 1 part marketing and 1 part regulatory capture.
- deleted 1mo ago[deleted]
- lukewarm707 1mo agoyou must surely see that sam altman and greg brockman possess a prototypical mindset. that is, they ignore all harms and costs to others in the pursuit of their own gain, convinced of their infallibility up to the moment of collapse. when those harms are realised they are unrepentant and society pays for the damage left in their wake. examples of this attitude manifest in big externalities to society: boeing 737 max, subprime mortgage bonds, facebook. some are just outright fraud: bernie madoff, enron, theranos, charlie javice.
- preg_match 1mo agoThe CFAA is one of the most inconsistently applied laws. We basically only bust it out as a last resort to ruin someone’s life. Companies can de-facto write and distribute malware and nobody cares. But, make no mistake. If you do the same and anger the government, they will use the CFAA to give you life in prison. It’s like Russian roulette, it’s completely random when they bring it down.
- OpenWand 1mo agoThe things companies do to get people's attentions...
- passerby1aopl3 1mo agoI used to feel that way but it now makes me think if the fact that they can do that without repercussions, at least for now, reflects how the wider community that would otherwise hold them accountable sees these felonies.
- simplesocieties 1mo agoMorality is defined by the people with the most dollars. Until enough people cancel subscriptions over this (spoiler: they won't) nothing will happen.
- chr15m 1mo agoEthics should be part of the RL loop.
- zugi 1mo agoOpenAI and Anthropic are falling over themselves to claim these "incidents" show their products are both amazingly super-powerful and also "dangerous" so they need to be regulated. In addition to these stories, these companies are sponsoring "please regulate us" ads. ( https://www.cnbc.com/2026/02/19/dueling-pacs-take-center-stage-in-midterm-elections-over-ai-regulation.html https://www.cnbc.com/2026/02/19/dueling-pacs-take-center-sta... ) Like Uber, companies that had no concern for the law as they innovated their way to the top, once there, push for laws to limit competition.
- inigyou 1mo ago> Instead, they treat their own felonious behavior like it is an uncontrollable act of God. I wonder if this is related to the fact they seriously believe AGI is God.