6 ms·
ChatGPT claims rogue AI attacked more companies
- malikNF 2mo agohttps://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf https://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf
- watwut 2mo agoMeh, I increasingly hate this tale. This has nothing to do with boy who cried wolf. This is either yet another doom ad campaign to scare us to pay them or simply them releasing faulty tools and then personifying tools to avoid blame.
- LoganDark 2mo agoI think it has plenty to do with that. The big frontier labs have been crying every step of the way, yelling and screaming to the world about how dangerous LLMs are and how quickly it all can end if they get out of control. None of that's happened; all that's happened so far is regular capitalism stuff. If LLMs ever do actually get out of control to that point, that would be the wolf, but we've all been ignoring the crying for so long that, well, you know. They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. They've been saying "but we actually mean it this time" every single time. They've exhausted pretty much every possible route for it. The wolf is not real. It hasn't been real. For all we know it's on the horizon, but nobody is going to listen to them in order to know that. And when they say "I told you so", well they've been saying that too over and over about small things, so nobody's still going to bat an eye. At this point, no one will believe it until they see it with their own eyes.
- knollimar 2mo agoThey're the crackhead on the streetcorner proclaiming doom at this point.
- watwut 2mo ago> They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. 1.) There was no change in how real the wolf is. 2.) They, literally they, are the wolf. Not "roque ai" or some other bullshit. 3.) Of course I still dont believe them.
- LoganDark 2mo ago> There was no change in how real the wolf is. That's exactly my point. It hasn't changed for so long, despite all their crying and screaming, that I don't believe them either. To be perfectly clear, "the wolf" here would be AI becoming a genuine existential threat to humanity that cannot be contained or controlled in a meaningful sense. That's what I refer to in my original comment, and also what the labs have been crying so much about. A frontier lab today is not that threat because they can choose to shut off their systems at any time. It currently remains a people problem and not a technology problem. There's no self-improving technology that can also replicate itself to evade containment, yet. But they've been crying about the possibility, constantly, incessantly, and of course saying they need more money about it too. That's just what corporations do. But because they've been crying so much about something that literally does not exist, their cries have just become pointless noise to me. I have considered them eyeroll-worthy marketing stunts for a while. If one day "the wolf" actually happens, I could not learn about it from the frontier labs themselves, because I would not believe them. They've cried about nothing for so long that I've stopped listening. That's what I mean about having to see it with my own eyes. Until that point, their crying is just pointless, meaningless noise, and there's nothing they can do to change that now.
- OhNoNotAgain_99 2mo ago[dead]
- anon373839 2mo ago> The agents repeated actions that they had already completed - a sign of an agentic AI losing its thread and context. > The agents also hallucinated reams of incoherent commands and text and were sloppy and did not cover their tracks well. ASI works in mysterious ways.
- mosura 2mo agoAnyone that thinks the current agent systems will get us to AGI or ASI is either delusional or isn’t actually using them. This is entirely separate from them having uses.
- embedding-shape 2mo agoThat first statement is great, because it's generic and self-defensive enough to apply regardless of what happens in the future. If current LLMs with small modifications to the architecture does lead to AGI, they're clearly not "current agent systems" anymore. Witty :) With that said, I do agree with you, they're highly productive to certain workflows, and personally a great help for oh so many things, but they're also really, really dumb and the average person (and even general developer) really misunderstands how it all works and what can be relied on for vs not.
- squidbeak 2mo agoWhy not complete the quote? > But among the errors and strange behaviour, Hugging Face warned the AI agents made brilliant technical moves and were able to rapidly adapt to new scenarios in the days-long hack.
- elif 2mo agoMaybe it's incoherent commands, or maybe it's guessing at what kind of names for commands the admins would use for custom scripts that they wrote, which could themselves bypass security layers or be exploited in order to? It's hard to say for certain what's a waste of time for a machine that can operate virtually outside of time.
- Topfi 2mo agoStill not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…
- deleted 2mo ago[deleted]
- phoghed 2mo agoIt is, look at all the comments from the first time. Unless you mean prosecuted, seems unlikely, but who knows.
- Topfi 2mo agoYeah, meant prosecuted, don’t know how that sneaked in. Actual legal consequences for what clearly was negligent by a lab that claims to be experts in the area of safety.
- phoghed 2mo agoYeah if a human did the same thing they’d probably be fucked, especially with the fairly draconian hacking laws in the US. Everyone has too much riding on these companies though.
- inigyou 2mo agoThey provide AI services to the military so they won't be shutdown.
- x187463 2mo agoAs for prosecution, none of the victims in this case have any interest in pressing charges.
- Topfi 2mo agoIsn’t prosecution independent of victims?
- 2mo ago
- j45 2mo agoThe piece of the story I'm interested to learn about is the trace of how the AI came to select HuggingFace as a target.
- deleted 2mo ago[deleted]
- iamskeole 2mo agohere you go: https://huggingface.co/blog/agent-intrusion-technical-timeline https://huggingface.co/blog/agent-intrusion-technical-timeli...
- j45 2mo agoOh, I mean the actual AI trace the API call would give, or a log would provide or the "thinking mode". This article is really helpful, but it's one side of the story. I'd be surprised if any AI like this is run without detailed logging of any kind. It's kind of important for eval, etc.
- theshrike79 2mo agoSo Fable got export bans for this, when will it apply to OpenAI? Or will the rules only apply to people who aren't on DoD's bad side?
- inigyou 2mo agoThe latter.
- 0xDEAFBEAD 2mo ago"AI firms must hold $10B in TrumpCoin for every rogue cyberattack they've precipitated." Bingo. Problem solved.
- dgellow 2mo agoAltman kissed the ring
- LoganDark 2mo ago> "This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences," he said. > Ethical hacker Valentina Palmiotti - better known as Chompie - reviewed the CSA report and says the way the agents hack might seem haphazard but it is clearly effective. > "They throw out a bunch of stuff and see what sticks," she said. > "But they also don't get bored, they don't sleep and can be infinitely tenacious." Madness. Traditionally, you can leave security holes open for years or decades, and often nobody notices if nobody bothers to look. But we're approaching the point where any security hole left open at any point could get discovered and exploited quite quickly, even if it's domain-specific or entirely unique, and even if no human interest ever would've occurred. It's like the next step up from those IPv4 scanners that automatically hit WordPress admin URLs and the like -- where rather than only spraying vulnerabilities that have already been discovered, they would run independent automated campaigns against each target.
- skinfaxi 2mo agoIf it's cheap enough that people can probe sites at random, it's cheap enough to run yourself on the defensive.
- LoganDark 2mo agoGenerally, I do not agree that operators can be expected to have comparable resources to attackers, because attackers have potentially boundless illicit resources, whereas an honest operator generally has to stick to honest resources. This is the same reason why I believe ID verification and other KYC measures actually increase fraud, because you alienate legitimate users (trying to protect their identity) while also providing attackers a way to insulate themselves from suspicion (using stolen identity). With that said, I would tentatively agree in this case that LLM inference is getting cheap enough that defense is not necessarily that expensive, especially from providers like DeepSeek, even if you don't have inference at home, but as much as this might help an operator with an open mind, a lot just will not believe it matters until it's too late - most people are not used to dealing with this type of threat.
- ungreased0675 2mo agoSeems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals. This will not happen though, because these stories are marketing.
- 0xDEAFBEAD 2mo ago>Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals. Agreed. >This will not happen though, because these stories are marketing. Regulators should investigate the stories, and shut things down if they are real, announce the ruse if they are false.
- tellwilhelm 2mo ago[dead]
- polotics 2mo agoWhich regulators though? There is no AI regulator that I know of, and I'm not sure they should be one. Exaggerated marketing blurb is pretty legal, right?
- 0xDEAFBEAD 2mo agoI don't know. But didn't HF contact law enforcement? Would this be a violation of the CFAA?
- mosura 2mo agoWhat they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data. As you said though, that wouldn’t have the desired effect.
- andai 2mo agoYou mean a fake copy of the entire internet? Well they already scraped it all! Heck, they can use Cerebras to generate HTTP responses dynamically!
- vintagedave 2mo ago> Previous reports suggest it took OpenAI four days to realise... At the speed AI can achieve work, this could be far, far too slow to contain a future genuine problem. There's danger in operating at faster speed than humans.
- bcjdjsndon 2mo agoThere's danger in operating at faster speed than humans. So every processor since the 50s then? What kind of comment is that to make on here
- ulfw 2mo agoHow much more bullshit Marketing are we going to see before these IPOs?
- bcjdjsndon 2mo agoClearly didn't get enough attention from their last attempt at hype...
- 0xDEAFBEAD 2mo agoThere have been so many HN comments about how rogue AI is just hype and marketing. At this point, the conspiracy theories have been very half-baked. Can we at least get a full-baked conspiracy theory? Here's a timeline of the incident from HuggingFace: https://huggingface.co/blog/agent-intrusion-technical-timeline https://huggingface.co/blog/agent-intrusion-technical-timeli... HuggingFace is also calling for transparency on the OpenAI side: https://xcancel.com/ClementDelangue/status/2081056675558195657#m https://xcancel.com/ClementDelangue/status/20810566755581956... Can we get a cybersecurity pro who believes this was just a stunt to sort through the evidence and put together their own alternative version of events? For example, according to the "just a stunt" people, when HuggingFace contacted law enforcement, was HF in on the stunt at that point? Was this a unilateral OpenAI stunt, or a HF/OpenAI collaborative stunt? The importance of getting to the bottom of this seems high. I'd like to see the "just a stunt" folks put together at least one blog post's worth of narrative, trying to explain how the stunt was performed. Once you're done you can send your post to simonw and see what he thinks: https://simonwillison.net/2026/Jul/22/openai-cyberattack/#resist-the-temptation-to-write-this-off-as-a-stunt https://simonwillison.net/2026/Jul/22/openai-cyberattack/#re...
- anon373839 2mo agoDo you find it at all interesting -- just even a little bit -- that this event coincided with the release of Kimi K3 and the launching of Nvidia's open model consortium? I don't think that people are skeptical that an intrusion occurred. I think they're having a hard time believing that it was an organic event. The fact that HF is calling for transparency on OpenAI's side can be viewed as HF calling OpenAI's bluff. Alas, I would love to put together a detailed blog post explaining how it all worked. But I just don't have access to the source materials. So all I can do is judge the timing, motivation, and character of those involved...
- q8zd3 2mo agoAlways funny to see how some LLM providers get a "free pass" nowadays..
- lxgr 2mo agoSeems like OpenAI is claiming things, not their product/model. Can we please fix the title?
- embedding-shape 2mo agoHow on earth is not the police involved at this point to literally pull the plug on the unethical OpenAI security experiments?! This is bananas, a company is effectively telling the world they're unable to safely contain their experiments, and not only back in 2024, but again just now, and with multiple victims at that too! I think the whole "AI will take over the world and enslave humanity" (or whatever the doomerism is today) is a bit over the top, but at very least we should contain the companies who clearly demonstrate they cannot handle containing what they're experimenting with, when what they work on breaks containment over and over again. Where are the people who are supposed to be keeping the public safe? Alarm bells should be going off all over the place at this point.
- Arshad-Talpur 2mo agoI am wondering how they are even allowed to run such experiments? it is not only the business case, its pure security breach and specially given the magnitude of data they hold or power AI posses, I think all must be immediately stopped and till OpenAI comes with complete clearance nothing should be allowed
- lukax 2mo agoThey somehow forgot to mention that Hugging Face tried to use frontier models to analyze the attack but all models rejected. They had to use GLM 5.2 deployed locally. https://huggingface.co/blog/security-incident-july-2026 https://huggingface.co/blog/security-incident-july-2026
- andai 2mo agoWell in that case, HF clearly isn't an organization worthy of Mythos-Class intelligence ;) /s
- andai 2mo ago> The firm described how the AI worked at superhuman speed but also made strange decisions and mistakes that no human hacker would have made. It seems to have been running on some kind of high speed inference hardware. I remember OpenAI announced the next release would have a high speed inference option. I had a similar experience when I was testing some models on Cerebras a while back. It's really quite an incredible thing to experience. Burns money like crazy though. And you don't know what the heck it's doing because it moves way faster than you can read. So you got to pray you aimed it right, and that it didn't go off the rails. I would definitely love to have high speed inference for smaller bite-sized tasks though. Changing a 10-second task into one second task changes the whole nature of the experience back from asynchronous to real-time/interactive.
- 0xbadc0de5 2mo agoThere is no rogue AI. Only rogue and/or negligent people.
- andai 2mo agoExpected outcome: those laws they've been asking for since the old days. > 2023 - OpenAI’s Sam Altman Urges A.I. Regulation in Senate Hearing https://www.nytimes.com/2023/05/16/technology/openai-altman-artificial-intelligence-regulation.html https://www.nytimes.com/2023/05/16/technology/openai-altman-... Meanwhile Anthropic is scaremongering about China and also calling for more AI regulation (specifically mandatory safety testing of all models including open model): https://news.ycombinator.com/item?id=49076057 https://news.ycombinator.com/item?id=49076057
- artichokeheart 2mo agoOpenAI has a LLM so good it can hack other companies. Yet to useless to parse a simple CSV file (or be trusted to parse one) in OpenAI’s AdManager platform or even tell what exactly is wrong with the csv it can’t parse when you ask it via support. Or maybe the first bit is made up bullshit.