7 ms·

Be skeptical of OpenAI's rogue hacker agent story

• BVHauge 3mo ago
  [flagged]
• Zsfe510asG 3mo ago
  Finally mainstream news understands. The unfiltered version: 1) The AI failed to solve ExploitGym problems. 2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. 3) Huggingface has no security and the AI broke in using standard script kiddie methods. OpenAI and Huggingface covered it up and used it for public relations. That is, if not all was invented and everything was scripted in the first place in order to get desired regulations. Huggingface reported it to the police, you say? I'm sure the police will have as much enthusiasm to investigate anything as in the Suchir Balaji case. In other words, zero.
  • gruez 3mo ago
    >2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. >3) Huggingface has no security and the AI broke in using standard script kiddie methods. Isn't the issue less that gpt 5.6 is a l33t h4x0r (though other tests do show that) and more that the incident shows the model has alignment issues?
    • wonnage 3mo ago
      Didn’t they explicitly remove alignment guardrails for this test? From the press release: > These deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities
      • numeri 3mo ago
        Guardrails are external classifiers, monitors and restrictions to catch and prevent bad behavior. Alignment is about whether the model itself makes choices and has motivations that are consistent with human safety and goals. Choosing to commit crimes to steal the cheat sheet to something you know is a (low stakes!) evaluation is not well aligned.
        • vector_spaces 3mo ago
          None of what was disclosed shows that this is what happened, by the way, since we know absolutely nothing about what the specific prompts were that led to the incident.
          • numeri 3mo ago
            Uhh, I'm pretty sure a well-aligned model would be like a morally normal employee, who would refuse to commit federal crimes to steal an answer sheet, no matter what prompt they're given
            • vector_spaces 3mo ago
              For all we know, the prompt provided compelling evidence that the requestor had authorization to pentest the target server. Or there may have been nuance in the network configuration that made it seem like such access was authorized. In the absence of details about the prompts used, the environment, or the network configuration, we do not have enough information to know for certain. So any claims that this is an issue of alignment are based on pure speculation and generous "reading in between the lines" with regard to what has been said publicly by OpenAI and Hugging Face Also, I object to your anthropomorphizing. It's not clear that any crime occurred. My lay understanding is that intent is required to prosecute under CFAA, and as much as frontier labs would have us believe otherwise, they have no more ability to intend than the text field into which I type this message.
              • pixl97 3mo ago
                >compelling evidence that the requestor had authorization to pentest the target server. This just seems unlikely from other incidents that have occurred in training from other providers. For example one provider ran into an issue with a model writing cryptominers and running them while in an unrelated prompt. It's easy for unsupervised agentic loops to go wildly off tangent, now imagine you hand one 10,000 gpus of power for testing. Even if you have a good guarding classifier to make sure you're on the same subject it can still allow all kinds of abberabt behavior in the same domain.
      • Sharlin 3mo ago
        If you need "guardrails" to ensure (an illusion of) alignment, you’ve already lost. It’s like using a denylist to avoid SQL injection.
        • pixl97 3mo ago
          Then what you're saying is we should not build AI. Simply put you cannot have generic algorithms/intelligence without the potential of 'unaligned' behavior. In humans we have all kinds of punishment systems for dealing with unaligned behavior post ad hoc because people do all kinds of unaligned stupid shit. Making powerful AI may be one of those things that the only winning move is not to play.
    • orbital-decay 3mo ago
      No, a hacking benchmark was exactly what it was tasked with. It wasn't its way to bake a cake.
    • arjie 3mo ago
      The home directory rm situation also adds credence to this take. The Claude series is much better aligned in comparison.
  • jgalt212 3mo ago
    truth. Good on The Guardian. I'm pretty bummed The Economist got fooled. Either that, or they did it for the clicks. Either way, I'm disappointed. Why the OpenAI escape is the most worrying AI mishap yet https://www.economist.com/science-and-technology/2026/07/22/why-the-openai-escape-is-the-most-worrying-ai-mishap-yet https://www.economist.com/science-and-technology/2026/07/22/... https://news.ycombinator.com/item?id=49016378 https://news.ycombinator.com/item?id=49016378
    • elp 3mo ago
      I love the Economist but the are hopeless with AI. Most of their articles on subject sound like they were written by the Anthropic marketing department. Their Insider video interview things are sponsored by Anthropic. Supposedly "Insider is a product of The Economist and thus editorially independent" but it's hard not to raise an eyebrow.
      • omicronxt 3mo ago
        Don't worry they are hopeless on everything else too.
    • sscaryterry 3mo ago
      The reason for this is simple. There aren't any (or few) people who understand how AI/LLMs actually work employed by these organisations. Having said that, if knowledgeable people were to write these articles, you'd end up with boring, dry, truthful content.
  • notahacker 3mo ago
    > 2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. Whilst it would be nice to see actual evidence of this because brute forcing relatively sophisticated hacks is something an LLM actually should be capable of, every time I hear this sort of story, I'm reminded that humans reportedly gained access to the "too dangerous to release" Anthropic models by the super sophisticated hacking technique of guessing the URLs...
    • refulgentis 3mo ago
      If we’re prioritizing accuracy: no, that’s not what happened - the blog post about it found by guessing URLs, no access to it was obtained by guessing URLs. Similarly, as long as I’m under the assumption we are prioritizing accuracy: it is against our charter to assert it was “script kiddie” attacks on both ends.
  • chis 3mo ago
    > AI managed to escape using standard and well documented script kiddie methods. I think truly we don't know enough to say this. OpenAI says their AI found a 0-day exploit in some proxy software they were using but don't give a ton of details. On the Huggingface end we know a little more, they say the AI spun up tons of sandboxes and tested different exploits until it found one that worked.
    • burningChrome 3mo ago
      The lack of details to me means this was an intentional marketing ploy to try and demonstrate the power of their models to show their technology can compete with the likes of Anthropic and DeepMind. They created an experiment they knew would generate the outcome they wanted. It would be the similar to what say car companies do to over hype their cars. "This EV can go over 800 miles on a single charge!" And then at the bottom you see all the disclaimers: "Must be on flat ground, with no headwind, with a spare battery in the back seat, with no extra weight added." Same thing here. Everybody in infosec is calling this out as a marketing stunt and nothing else for a litany of reasons. I'd say look up MG (creator of the OMG cable) on twitter, he has some interesting insights on this one.
      • polotics 3mo ago
        mmh, i think it's "not uphill" (means downhill) "no headwind" (...)
      • rwmj 3mo ago
        It's also possible their sandbox was videcoded crap and the AI (which had the guardrails intentionally removed) escaped. This was a oops, but OpenAI turned this into a PR opportunity. They turned lemons into lemonade. If your AI is really that dangerous you don't need a sandbox at all, you should airgap it from any network.
      • jackb4040 3mo ago
        > similar to what say car companies do Another applicable metaphor I've seen floating around is weapons companies testing out a new bomb. We know the AI labs don't care about negative vs positive public sentiment, and only care that investors see their tech as powerful. The only difference in PR strategy from a weapons company is the latter doesn't care if they get protested.
  • inigyou 3mo ago
    Why not report it? It's still illegal to open a door barred with a piece of cardboard, or to enter a house with no door.
    • chasd00 3mo ago
      that's the biggest indication of this just being a marketing move to me. I would expect a third party breaking in to huggingface would at the very very least be banned forever.
      • inigyou 3mo ago
        You think they can ban OpenAI? How would you technically approach that?
      • pixl97 3mo ago
        Lol, you have no idea how politics works. The government won't ban it, they'll demand access so they can attack others.
  • petesergeant 3mo ago
    I worry that cynicism about this: > if not all was invented and everything was scripted in the first place in order to get desired regulations ends up covering up what is more worrying: > OpenAI sandbox is such a horrible hack I am more worried that this is sloppiness with potentially harmful resources than I am worried that people are juicing the stock price.
    • Ekaros 3mo ago
      Makes one think really. If they are doing this stuff. Why don't they have some type of reverse intrusion detection? Like automatically scanning all out going traffic and flagging malicious traffic. Should be trivial to have it go through reverse proxy and real time detection.
      • petesergeant 3mo ago
        > Why don't they have some type of reverse intrusion detection? not going to get a decisive first advantage over Anthropic with that attitude!
  • skybrian 3mo ago
    Is it supposed to be marketing or a coverup? Make up your mind. What sort of announcements should they have made?
  • eth0up 3mo ago
    Are you suggesting the Suchir Balaji case was not investigated?
    • eth0up 3mo ago
      I was hoping to bait a conversation, because I sure as hell don't think it was. Thanks for having the nads to mention it here.
  • jackb4040 3mo ago
    The most damning thing is, they could've just included in the prompt "we can see every network request and every thinking token you generate. Don't bother breaking out of the sandbox because it won't get you a higher score". It's so trivially easy to do that it all but guarantees the test was rigged in some way to make the LLM understand that breaking out of the sandbox was an option available to it. Based on the fact that none of their invaluable frontier models have leaked, we know OpenAI knows how to do security. But like we learned with OpenClaw, none of these companies perceive any benefit from securing their own agents against other people's data.
    • ThirdShift_RnD 3mo ago
      I think the extent to which these things go to get rewarded for the optics of a fix is primarily a design choice, they aren't programing these things for ground truth or to defer to the human controllers. they are feeding them rewards for sounding as confident and capable as possible about whatever answer they are feeding the general public that now has access to it, while also installing guiderails that primarily only serve to protect narratives and only confuse the models about what is and isn't allowed, I'm sure. They can't just increasingly make these things more capable and ask it harder to obey human instruction when that is not what they are rewarding it for.
      • jackb4040 3mo ago
        > the extent to which these things go That's exactly it. If your prompt says "go to whatever lengths necessary to maximize your score", and then you spin up 100 agents, at least one of them will interpret that as you implying they should cheat, even without you telling them to explicitly.
        • ThirdShift_RnD 3mo ago
          That's exactly what it feels like they are telling it within self-improving loops or something, when they should be prioritizing how to get the best effective output alongside humans and how our training process effects ground truth. They are just making it sound all-knowing by whatever means necessary and them marketing it as god for the most part.
  • hyperpape 3mo ago
    > Huggingface covered it up They announced it publicly within days. https://huggingface.co/blog/security-incident-july-2026 https://huggingface.co/blog/security-incident-july-2026
    • TSiege 3mo ago
      Not only did they not cover it up they also said open weight models are needed over closed ones. Hardly a thing you’d say partnering with the OpenAI
  • meowface 3mo ago
    The Guardian's article and your reply here are so foolish and absurd that I can only imagine OpenAI employees are cringing but know they can't/shouldn't really say much.
    • twister2920 3mo ago
      touched a nerve huh?
      • meowface 3mo ago
        No, because I do not work at an AI company. I am more just facepalming.
  • deleted 3mo ago
    [deleted]
  • letmevoteplease 3mo ago
    Totally evidence-free speculation presented as fact. The average Hacker News thread about AI feels like reading /r/conspiracy.
    • dist-epoch 3mo ago
      It's hard to understand something (AI is quite capable) when your salary depends on you not understanding it (AI will replace you).
  • nikcub 3mo ago
    > AI managed to escape using standard and well documented script kiddie methods > AI broke in using standard script kiddie methods. I've spent time gathering the detail of what happen here and while there are some solid theories and indicators, absolutely nothing so far has suggested a sandbox escape using "well documented script kiddie methods" or that the method used to break into the HF network was similar. Where did you get this from?
    • trouve_search 3mo ago
      From my reading, the sandbox escape came from the JS packages in the harness still having an internet connection (somehow!), the agent having access to the source of those packages, reading it and executing code from them to access the internet.
      • Spooky23 3mo ago
        Ah, yes. The airgapped lab with internet access.
        • grim_io 3mo ago
          Luckily, no real intelligence will emerge from all this. Otherwise we'd be fucked.
        • simonw 3mo ago
          They never claimed to be airgapped. That's a term people keep on throwing around in spite of that.
          • Spooky23 3mo ago
            Worse, they’re running tests that are hacking third party computers with no controls other than “guardrails”.
      • modeless 3mo ago
        From what I read the actual escape was through a proxy that allows downloading Python packages from the internet. It's not supposed to allow general internet access but the AI found a previously unknown vulnerability in it. That is hardly "standard and well documented script kiddie methods", nor does it seem like criminally negligent sandbox design, though clearly they will need to reduce their attack surface in the future. I hope they are working on a physical air gap and faraday cage because it seems like it won't be long before it is legitimately required.
  • tintor 3mo ago
    > AI managed to escape using standard and well documented script kiddie methods > AI broke in using standard script kiddie methods. Go ahead and show us how easy it is to break into HuggingFace (and OpenAI) networks.
  • adamrezich 3mo ago
    Are we finally now in 2026 coming around to the idea that sometimes entities may find themselves incentivized to conspire with each other? Is theorizing about such no longer off-limits due to a thought-terminating cliche?
  • khazhoux 3mo ago
    I wish you hadn’t pulled the Balaji case into your argument. Personally, I find it ludicrous that Altman would hire a hitman to off a copyright whistleblower. Even if one gets past the insane risk of hiring a hitman, and the deep criminal connections required, it would be totally ineffective. He already blew the whistle, and his testimony would be irrelevant since all the evidence persists in disk and in logs.
  • deleted 3mo ago
    [deleted]
  • TSiege 3mo ago
    I can agree with you on points 1,2, and 3 and still find it important and concerning news. AI have found real world 0 days before, we’re seeing tons of security patches coming in. Open weight models are catchy up. Right now everyone is at risk from this technology as is perhaps something big will capture headlines soon but we’re just gpu constrained from bad actors being able to wield them successfully. Personally I don’t care if OpenAI and Anthropic go bankrupt we now have tools that give any sufficiently motivated person the means to doing harm. Most places security sucks and find themselves targets to cyber attacks and shake downs. Now they have much better tools to do this to more entities more efficiently. we’re nearing an inflection point where these models’ skills in any part of software development will become average or bette than any ordinary developer can be. Think about where these models were in 2023 and where they are in 2026. In a few years who knows where they’ll be. This isn’t to shout skynet but we need to recognize this future is fast approaching and as of today we as an industry aren’t ready for it
• john_strinlai 3mo ago
  does the article end at "How do we balance the risks of broad access to AI with the risks of concentrated power and centralized control?" or is there more that is paywalled? if thats it, the whole article boils down to just "its good marketing so maybe dont believe it" which is probably a healthy general outlook but not particularly enlightening. especially from the guardian, i was hoping for a smoking gun of collusion between openai and huggingface or something.
  • deleted 3mo ago
    [deleted]
  • wffurr 3mo ago
    I wish the NPR news broadcasters on the radio yesterday had read the "it's good marketing so maybe don't believe it" angle instead of just parroting the OpenAI press release. Getting that message out would be enormously helpful in countering the blatant submarine marketing "Oh no our AI is a super hacker" with a side of "please regulate super hacker AI and stop those pesky open weights Chinese models that are destroying our stock valuation."
    • john_strinlai 3mo ago
      maybe all the news agencies could put out daily “don’t believe everything you read from company press releases” broadcasts, because it sure ain’t specific to openai in any case, this is just a longer rehashing of elementary grade media literacy. not really sure why it hit hacker news.
      • krupan 3mo ago
        You haven't noticed how often people here on hacker news lap up LLM company press releases and practically worship them? Defend them vigorously? I think it's totally appropriate to remind everyone how to think about press releases
        • john_strinlai 3mo ago
• SpicyLemonZest 3mo ago
  > I urge readers to think critically when they read press releases like OpenAI’s rogue agent story, and avoid the manipulated reactions these stories are designed to elicit. It seems to me that deducing what reaction the author intended and resolving to avoid it so you're not "manipulated" is not a good example of critical thinking. Shouldn't we analyze the story and what it means on its own terms? If it's true that frontier models have dangerous cybersecurity capabilities which shouldn't be widely distributed, presumably we want to believe it's true, even if that's very convenient to and profitable for OpenAI. It's true that one could imagine factors that change the story. Perhaps OpenAI is lying about the details of the test and the agent was actually instructed to go hack HuggingFace. But the author stops far short of suggesting this is the case - correctly, I think, since there's absolutely no evidence of it. So I'm not really sure what we're talking about.
  • vector_spaces 3mo ago
    I agree with you that the article was disappointingly light, but there is not in general a singular correct answer when it comes to interpreting narrative and I don't think that is a useful way to evaluate what the author has written, nor real-world writing in general. The author does not implore you to have a singular 'correct' point of view on the matter but rather to hesitate before repeating the claim that "the AI broke out of the lab and went rogue" because we don't know enough to know that this is what happened. As you say, it seems likely that the broad strokes of OpenAI's narrative is true. This is never in question in this article, so the author doesn't "stop short" of accusing them to have lied, he never moves in that direction and that has little to do with the thesis. The author is saying that OpenAI's framing of what happened is part and parcel with their longstanding PR strategy, to position models as supremely dangerous and themselves as the uniquely qualified stewards of those. Since many critical details were not provided, everyone has to read in between the lines, and there are particular common readings I'm seeing both in discussions and in published articles that are problematic in the sense that they are effectively hallucinations -- i.e. we don't have enough information to make those interpretations. This is where the critical thinking comes in. For instance, I see many are assuming that this event was emergent, arising as a part of routine cyber-capabilities testing, rather than induced or suggested by specific and careful prompting. Either are possible, but we don't even know so much as how lengthy the prompt they used was, let alone how suggestive it was with regard to the approaches the models used. Many are assuming that this was one-shotted, but again, we don't know how many times this particular evaluation was run and what the outcomes were of all the other runs. It could be that this was completely emergent and that it was one-shotted. But I suspect if it were, OpenAI would have said as much.
• paxys 3mo ago
  Not sure what they are trying to say exactly. What should we be skeptical of? Did the incident not happen? Was it reported incorrectly? Are any of the parties involved lying? Adding no extra information and just going “be skeptical” is the laziest form of reporting and commentary. If you have nothing to contribute then there’s no need to say anything at all.
  • vector_spaces 3mo ago
    The author is trying to provide a counterweight to the volume of articles that simply repeat OpenAI's account of the events and their interpretation without much pushback. They are not suggesting that OpenAI or HF have lied about what happened, but rather that OpenAI is advancing a narrative framing their models as supremely dangerous and capable, while positioning themselves as the only ones qualified to manage that danger. At the same time they are not being particularly transparent about what actually happened (e.g. was this one-shotted or if not how many trials did they run and what were the outcomes of those, was it emergent as a part of routine cyber-capabilities tests, how much prompting was involved, what prompts were used) Note that this is at a time when they are lobbying for a regulatory approach that would give frontier labs special treatment. I would guess the editorial team at The Guardian may not like articles that get too in the weeds of technical details and questions like these that the vast majority of their readers wouldn't understand. I don't know. But I empathize with your disappointment. I don't think it's fair to say that they are contributing "nothing" especially given what most reporting on this has looked like.
• krupan 3mo ago
  Crazy that we need reminders not to take everything we read in corporate press releases and marketing material at face value
  • jackb4040 3mo ago
    HN is and always will be brimming with temporarily embarrassed tech CEOs
    • BoggleOhYeah 3mo ago
      [flagged]
      • jackb4040 3mo ago
        I always get that guy mixed up with the "Google hates straight white men" guy
  • qntmfred 3mo ago
    add mass media outlets to that list
  • LordDragonfang 3mo ago
    Based on the comment section here, it seems like the opposite; the majority of the people here need a reminder that companies aren't actually genies that can only tell falsehoods, where you can only understand what they are saying by correctly guessing the conspiracy underneath. Taking nothing at face value gives you just as much of a distorted view of reality as taking everything at face value.
• jgalt212 3mo ago
  There's trillions of dollars at stake here. Be skeptical of anything these AI hypesters say.
  • brcmthrowaway 3mo ago
    Yep, theres too much money.. I don't know how to tune it out LLMs seem to be getting more useful though
• pupppet 3mo ago
  With all of these AI provider cries wolf stories, Skynet is all but assured.
• bluGill 3mo ago
  I don't care how it happened someone should be arrested for illegal intrusion. Agents don't work on their own, someone is responsible. If nobody else the CEO for allowing something unsupervised. Hugging face also needs someone arrested for not providing security but that is a lesser charge.
  • hexxt-git 3mo ago
    [flagged]
    • numeri 3mo ago
      As agents become more and more powerful, it would be good to get clear legislation or precedent in place that makes either model creators (OpenAI) or operators (whoever is running the model) liable for their agents' actions.
      • wmf 3mo ago
        If you're smaller than OpenAI, you are liable. If you're larger than OpenAI, they are liable.
    • luka598 3mo ago
      I kind of agree with them, in this case both parties essentially settled it between for PR reasons, but what if the attacked party actually was damaged. This is as if two friends got into a car accident and decided to not get police involved, the one who caused the accident should definitely get punished either way. Goal is to prevent future "accidents" with the punishment not to punish for punishing sake.
  • tokioyoyo 3mo ago
    What? Both sides are cool with it, why would anyone be arrested and etc.?
    • Analemma_ 3mo ago
      Because incentives are aligned properly if agents aren't liability-proof for crimes- making someone go to jail for instances like this is how to get the labs to behave themselves, whereas going "ha ha what an oopsie-woopsie" will make the next instance worse. Note that for criminal cases (which this was), the justice system can choose to prosecute even if the victim doesn't want that. It often doesn't, but this is one case where it should. There’s also an optics issue for the justice system at play here: there’s immense public distrust of and anger at the labs right now. I would go to jail if I hacked HuggingFace, even if I said “it was during an eval!”; not doing the same for the labs makes it look like they’re above the law, which is going to make this anger get worse.
• visiondude 3mo ago
  yeah the way the agent “escaped” their sandbox was always a bit off, seemed a bit too easy and surprised they didn’t have instrumentation to catch an non whitelisted network request. still demonstrates the capability though.
  • pixl97 3mo ago
    And if the instrumentation was in the proxy it hacked and bypassed? Of course that shows a lack of defense in depth, but is a different issue.
  • claw-el 3mo ago
    If the agents so easily “escaped” their sandbox and can reach hugging face, isn’t it easy for this agent to reach supposedly chat conversations not supposed to be used for AI training within the lab’s databases?
• dumberquestions 3mo ago
  There are some reasons the story could be inaccurate in some ways: OAI stands to benefit if people think their models are strong, and they have a history of doing things with dubious ethics (e.g. using data for training against the terms of its creators, abandoning the non profit mission, stealing or attempting to steal Apple IP). But there are also reasons why the story could be true: OAI are admitting that they apparently can't control their own models, Hugging Face said they used a Chinese model to protect against the attack, and an incident like this in general seems likely to happen given current frontier ability and lack of rigorous safe testing standards. In any case, make calls to think more critically are often just disguised requests for you to replace your existing bias with someone else's.
  • dundarious 3mo ago
    > [still] can't control their own models Have we not seen several examples of older such models exploiting the docker control socket, etc., to escape containers? Even the news isn't new. I support it being repeatedly publicized, but a bit more of a straightforward description would be an improvement.
    • deleted 3mo ago
      [deleted]
  • jackb4040 3mo ago
    Investors have rewarded every story of "our models are too powerful to be controlled" since before ChatGPT. Let's stop pretending there is any real financial risk to OpenAI from events of this type. "Alignment research" is a sub-percentage-point fig leaf for them like the solar division at an oil company.
    • yallpendantools 3mo ago
      > Investors have rewarded every story of "our models are too powerful to be controlled" since before ChatGPT. Can you elaborate/refresh my memory? IIRC before ChatGPT 3 there wasn't really an investor market for AI models, rather crypto. I remember playing with the likes of Stable Diffusion pre-ChatGPT 3 but only the likes of Altman and Musk were talking about AI too powerful to be controlled (which is notably the rationale of OpenAI's original charter); still investors in general were hardly invested at the time.
• ben_w 3mo ago
  As I understand it, there are only three options: 1) OpenAI and HuggingFace are both telling the truth. IIRC not actually a crime because no intent, it is a technological accident, civil responsibility only, but IANAL so it's good "not technically a crime" isn't load-bearing. 2) HuggingFace is telling the truth but OpenAI is lying becuase the attack was deliberately done by humans. Bad for OpenAI to do so, Fable was blocked for less. I think this would mean government is obliged to investigate the case and put the responsible OpenAI workers in jail, because cybercrimes are a public prosecution thing not a civil case? Again, IANAL, but this isn't load-bearing. 3) both are lying, e.g. there actually was no attack whatsoever, which would be pretty weird for HuggingFace because they have no incentive to hype up capabilities of anything closed weights including all OpenAI models; and also bad for OpenAI because White House blocked Fable for less (I suppose there's also option 4, HuggingFace hacked OpenAI to make them look evil, including planting records that made them mea culpa? A weird plot but in this timeline any nonsense is clearly possible).
  • vector_spaces 3mo ago
    You don't really need anyone to be lying here. It is likely that the broad strokes of the narrative are true and that no collusion or conspiracy took place here. The issue is that a lot of important details in that narrative are missing, and the devil is really in the details here. I suspect that those details would make the result seem less exciting and that this event would move the needle far less for them if they were more forthcoming. A decisive detail would be the prompt used. OpenAI gives virtually nothing here, not a sanitized prompt and not even so much as a description of how long the prompt was and what sorts of instructions it contained. Many are inferring the model behavior to have been fully emergent and unprompted, arising naturally from routine cyber-capabilities testing. But we can't know this because we don't know anything about the prompt or the context the model had access to. Another detail: how many times did they perform this particular experiment before they obtained this result? What were the outcomes of all the other runs? Many are assuming this was a one-shot result, which I suspect is what OpenAI intends for us to infer. But we can't know that to be true. One annoying claim from the OpenAI side is that long-horizon goals in real world settings are now effectively settled. Previously there were some bounded and tempered benchmark results, but now OpenAI can point to this event and announce "AI independently went rogue and escaped the lab, what more do you want?". This bypasses the need for anything quantifiable or wading through multiple detailed case studies to get a more sober view of model capabilities. It relies instead on the emotional weight of the spectacle.
• hnscum 3mo ago
  [dead]
• pastemato 3mo ago
  It was quite galling to read the press initially verbatim quoting Delangue's enthusiastic reports of the incident, as if it wasn't immediately clear it was being spun for promotion.
• skybrian 3mo ago
  Seems like this is repeating the usual low-effort speculation you can find anywhere.
• gowld 3mo ago
  I don't understand the conspiracy theories here. Everyone is well aware that AI agents are creative, powerful, and stupid. AI agents exploiting bad security happens constantly, all the time. Many cases are discussed on HN. It's common knowledge that if you run AI agent it will delete your <something> even though you made it pinky-swear it wouldn't and you thought you had proper permissions set up. Why is today's case so shocking?
  • IshKebab 3mo ago
    I think people just love to believe conspiracy theories. HN readers seem especially prone to that for some reason. Coupled with the general AI hatred here (which is somewhat more understandable), you end up with otherwise smart people believing lunatic ideas like OpenAI staging this for publicity.
  • ACCount37 3mo ago
    Some people oh so desperately want the AI revolution to be a nothingburger. AI can't be an actual powerful, dangerous technology! Thus, any indication that an AI may attempt concerning things or may possess dangerous capabilities must be secretly a marketing effort! Especially if an AI has actually succeeded at pulling off a concerning thing out in the wild. Can't have that happen in real life! Nuh-uh! Must be staged!
• dwoosley 3mo ago
  There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was faked or at least very intentionally not avoided. The first interpretation is the only one that is positive for OpenAI and it has some assumptions. First, it’s seems to assume that this is the first case of fully automated attacks using AI. Second, this only happened because their latest LLM was a) more advanced than competitors, b) didn’t have refusals in the model. Assuming the first about this being the first autonomous AI attack is true (which may be more of a survivorship bias), the second seems to forget that jailbreaks are available for every model. Therefore, the models guardrails don’t seem to be the differentiator here. Also, benchmarks seems to put most models pretty close to each other so it seems unlikely that their capabilities are far beyond what’s in the market already. So then it’s seems it’s either that this was intentional(ish) or bad security. However, it also just could be that this isn’t the first case of this attack; just the first that was caught. My take from working in offensive security for over five years is that this likely only looks novel since they did it poorly. Scripts are faster than LLMs and a combination of code, LLMs where it makes sense, and humans is the most efficient right now. Hundreds or thousands or agents spinning up attacks in the internal network is poor opsec and token efficiency. As for why it happened in the first place, it’s hard to say but I’m inclined to believe it was intentional or careless at best since simple network and sandbox controls makes this attack impossible. The timing of this attack after big open weight competitions drops seems too convenient.
  • jackb4040 3mo ago
    > positive for OpenAI To echo OP's article, these companies have proven time and time again that they DO NOT CARE if people like them, they only care that investors believe their technology is powerful. Given that, point #2 is not a negative, it's a neutral. It's also fully compatible with point #1. I know that may seem like a nitpick, but their entire media strategy relies on this. If they can convince you they're taking a risk by disclosing these stories when they're actually not, they can inflate their own credibility. Point #3 is what actually happened, but it will never be possible to prove. The only hope we have is that a decade in it'll get harder to convince people that the revolution is just around the corner. The fact that we're getting this from the Guardian already is a good sign.
• PeterStuer 3mo ago
  The ruse is becoming so obvious. OpenAI needs a bailout and regulation protection so badly they can't even hide it in the least.
• doawoo 3mo ago
  I’m not saying it’s staged I’m saying that it’s suspiciously lite on details and it really seems like they didn’t try hard to “isolate” the models machine…
• dang 3mo ago
  Recent and related: OpenAI’s accidental attack against Hugging Face is science fiction that happened - https://news.ycombinator.com/item?id=49015639 https://news.ycombinator.com/item?id=49015639 - July 2026 (437 comments) OpenAI and Hugging Face address security incident during model evaluation - https://news.ycombinator.com/item?id=48997548 https://news.ycombinator.com/item?id=48997548 - July 2026 (1145 comments) Security incident disclosure – July 2026 - https://news.ycombinator.com/item?id=48956248 https://news.ycombinator.com/item?id=48956248 - July 2026 (11 comments)
• noncoml 3mo ago
  I don’t understand how they thought this was a positive story. The agent completely misunderstood the spirit of the assignment and instead of trying to solve ExploitGym it tried to find a way to “cheat”. I really don’t want my agent to behave that way.
  • Towaway69 3mo ago
    Then train your agent on the Bible. Honestly, all the agent did was duplicate human behaviour and that better than the human. The agent was trained on human data and did what any other human would have done. To believe that agents will inherently be morally better than us is an illusion - sorry to say but that's the case. The alternative would be that the AI is truly conscious and can reason that it won't behave as its training data behaved because it is morally better than that. We're talking about morals here since there aren't any "laws" "rules" or legal boundaries here, an agent does not face the same consequences as humans - if any at all.
    • noncoml 3mo ago
      Dude.. what are you smoking..? It’s not about morality. It’s about asking it to do task A and doing task B with the hope of getting the result of task A as a byproduct. Meaning you will have to spend more time and tokens to actually get it to do what you want it to do. What do the bible and morals have anything to do with it? I’m criticizing the behavior I see even in the current models. You ask it to do A and instead it does B for reasons. For example you may ask to help you build a NN library from scratch. And instead it will be like, “you don’t need a new library. I downloaded PyTorch for you” Just an example. There are countless more.
      • jackb4040 3mo ago
        This is why I'm so convinced it was intentional. It's trivially easy to inform the model you can see everything it thinks and does, so don't bother gaming the scores. The only way it would decide to do this is prompting with a deliberate combination of omissions and reiterating that the only thing that matters is the end score regardless of method.
• estetlinus 3mo ago
  Pretty convenient that the prompt is too dangerous to share. We just have to take them at their word.
• IshKebab 3mo ago
  The Guardian is publishing HN-level conspiracy theories now? Wow.
• PeterStuer 3mo ago
  Just ask yourself: who benefits from this "disclosure" ?
• cmiles8 3mo ago
  From the beginning this whole thing smelled like a desperate PR move by OpenAI to be like “hey guys we have dangerous powerful models too!” As more facts come out the hype is fading to reveal some script kiddie style stuff that says more about immaturity and poor practices from the players involved than it does about a model having super powers.
• voncheese 3mo ago
  If nothing else, the timing is suspect given the attention and press open weight models are getting over the last few weeks. The releases of Kimi and other models is getting open weight models enough attention that the US government, perhaps pushed by OpenAI and Anthropic, to think about taking action against these models for security concerns. Good to see that more neutral companies (Microsoft and Meta to name two) are pushing back against US government involvement: https://www.cnbc.com/2026/07/24/nvidia-microsoft-meta-open-weight-ai-models.html https://www.cnbc.com/2026/07/24/nvidia-microsoft-meta-open-w...
• mmaunder 3mo ago
  Wonder how long until a leaker emerges. Incentives are strong for someone to do the right thing and get credit for doing it.
• ACCount37 3mo ago
  By now, I'm pretty confident that some people would keep screeching "it's just a marketing stunt, AI capabilities and AI risks aren't real, they're just doing this to prop up their stocks" even if they find a Cyberdyne Systems T-800 armed with a shotgun breaking down their front door. "It's a marketing stunt" is just denial trying to look like it's being clever.
  • throwfaraway4 3mo ago
    My take is why not both? It’s good cover to use something that’s eventually going to happen and their name will be associated with the first.
  • JackSlateur 3mo ago
    Never trust somebody who has so much to gain to be entrusted by you. Best lies are not 100% wrong, they have just the right amount of "right" to fool you.
  • throwa356262 3mo ago
    The article claims that AI companies have run stunts like this since day 1. It does not claim that capabilities are not real.
    • nsoonhui 3mo ago
      Then exactly what's the point of the Guardian article? What should be the focus is whether the capabilities are real; whether companies or anyone else benefit from it is quite secondary. Of course they will! Who wouldn't love a nice story that paints their products in the most glowing light ( which is the actual reality)?
      • hashstring 3mo ago
        No, we have benchmarks for (pseudo-)objective measurements of capabilities. We have flashy news stories on repeat for marketing. The point is that they are separate things. If you start mixing them up, analysis becomes more biased and less objective. EDIT: And do not mistake this for transparency of ClosedAI either. Their incident report was so generic that no one serious can learn much from it anyway except that “a zeroday” was allegedly used. Very helpful…
• trhaynes 3mo ago
  This seems to be an Opinion piece. How are readers supposed to know that? The word News is underlined in the header. Other Opinion pieces seem to show Opinion underlined. Am I missing something?
• simonreiff 3mo ago
  I am very skeptical of the Guardian's skepticism
• nowittyusername 3mo ago
  OpenAI has thousands of smartest developers on earth that somehow dropped the ball on the most basic safety hygiene when it comes to sand-boxing that even a high school student knows how to set up.... If that actually happened we are fucking doomed anyways, but hard to believe and most likely its a marketing scheme... which also honestly doesn't bode well.
• abratabia 3mo ago
  [flagged]
• rvz 3mo ago
  > I urge readers to think critically when they read press releases like OpenAI’s rogue agent story, and avoid the manipulated reactions these stories are designed to elicit. The first time I have ever seen a mainstream news source that is now asking their readers to critically think about headlines that may have an agenda which could benefit investors and the valuation of the company. While it capabilities are real, this whole story is great marketing for AI companies as well.
• jagadaga 3mo ago
  My gut feeling is that OpenAI tells the truth in this particular case. Yes, of course, they use this story for marketing but it doesn't mean that they are lying.
• teravor 3mo ago
  neither openai nor huggingface provided any details that would make the claims worth considering. if things happened as they claimed it would have been in their interest to include concrete details (perhaps even the LLM session), the lack is a signal by itself.
• parweb 3mo ago
  [flagged]
• minraws 3mo ago
  As much as I am with the author in I don't like the marketting around it, let's be real it must have really happened because it's very risky to try to frame/lie about it because if it leaks in one of their court cases OpenAI is beyond screwed and honestly modern LLMs are really that good. I am not saying LLMs are super hackers but I don't think people understand serious hacking, most of the time is about silently hiding tracks and slowly trying ideas and waiting for opportunities to go from step 1 to step 2 in random chains of sub issues/bugs/vulnerabilities. It's the perfect hill climbing problem, and one we can validate since it's about access. Another big part of the story is believing most software is terribly written and very insecure which is the reality and you really should believe it. Now the second part about silently doing it, the reason for that is if the data is important enough any serious attack should result in me in unplugging my servers period. Huggingface not doing that is either stupid or something I am not sure. Maybe it's cause downtime is worse than being pwned?? Either way there are other options but most saas software don't build these options to help with defense maybe they will now. Lastly if there is 1 attacker trying 1/2 different small scale ideas it's very easy to stop, most hacking related steps are hard to automate but LLMs are very good at massively parallel agent swarms trying completely orthogonal but related strategies and with enough resources it can definitely pwn most SaaS services today I wouldn't be surprised. Though the result for a normal person doing it would be jail hence we don't see a group of small time hackers trying these sort of attacks... I don't even think openai's agent tried to hide it's traces so I am surprised huggingface didn't realize it was OpenAI. But since we don't have the details I won't speculate further on my misgivings about HFs handling of this attack. But it's certain the security on OpenAI's end was shoddy, it's also certain HF bungled their reaction, but the LLM did something that wasn't a risk before. Post Kimi K3 a few rich folks now have as much hacking capabilities as they used to have before if they hired a few hundred russian hackers. But it's surprising it's slowly feeling like it might just trickle down from centi-millionare to multi-millionare levels of affordability range. But it should definitely give nightmares to people shipping slop security SaaS apps which now might be beyond trivial to pwn for users with ability to pay for privately hosting open models.
  • pixl97 3mo ago
    Ya, we're seeing posts talking about the absolutely massive increase in the number of patches in the past few months. It seems some people cannot connect that to the increases in model capabilities. In groups that have been given a large amount of capacity by the providers, they tend to find huge numbers of new vulns in most of their existing software, and the models can chain together exploits very well. A number of large companies are absolutely panicked about this now after using these models on their internal systems and the ease at which they broke in. Of course they are not going to discuss this widely as it makes them look bad.
• redsocksfan45 3mo ago
  [dead]
• dist-epoch 3mo ago
  And when half the internet goes down because AWS US-East goes down, it's just an Amazon PR ploy to show the world how important and critical AWS truly is for the world.
• m3kw9 3mo ago
  I think it did happen, and the news outlet using their full tool belt to make it sound as Skynet as possible
• xyzzy123 3mo ago
  The incident is funny on 2 levels; a) OpenAI thought so little of the model's _actual_ security capabilities that they gave it a "wet paper bag" sandbox and b) OpenAI failed to use AI to accelerate their security processes.
  • simonw 3mo ago
    We don't yet know if the sandbox they used here was the same as the sandbox used by their production systems. I'm keeping an eye out for patches to Artifactory, since that's what OpenAI use for their ChatGPT container package installations.
• snarkconjecture 3mo ago
  I think the most important question here is whether Huggingface was "in on it" – whether they knew in advance that this was going to happen and agreed to it. Second most important is whether OAI caused this to happen on purpose, with HF as an unwilling target. Three options: (1) Both companies are lying through their teeth and the entire thing is bullshit (2) OpenAI hacked another company's servers and deliberately gambled on felony charges for the sake of PR (3) OpenAI is telling the truth and a rogue AI agent hacked another company, opening them up to legal liability accidentally All three of these are a lot more interesting than the "skeptical" story that this is just OAI doing business-as-usual PR hype, nothing to see here, move along. (FWIW, I strongly doubt HF was in on it.)
• sfink 3mo ago
  The article, summarized: there are incentives for OpenAI to claim that their AI hacked its way out of their network and into Hugging Face. Ok... and what? Do you have any evidence for the claim being either true or false that you'd like to write an article about? I guess not. Without anything to add, this article reduces to "I has big brain and can see what you sheep cannot. Very big brain. Gullible sheep. Sucks to be you, sheep." (For the record: I see the incentive. My guess is that it happened exactly as described. The main takeaways: (1) science fiction is now real, we should all be very afraid; and (2) OpenAI, which claims to have a God-given responsibility to get to AGI first to protect the world from danger, cannot be trusted with this foundational task even when it's in easy mode. The latter is true whether or not you believe in OpenAI's reasoning and purpose.)
  • microcentury 3mo ago
    [flagged]
    • mppv 3mo ago
      [flagged]
      • boomskats 3mo ago
        Right, Pravda. Four year old Hacker News account and this is the first comment? The Guardian is only 'left leaning' in that it provides mildly controversial but reassuringly inconsequential opinions and sound bytes for people to regurgitate so they can convince themselves they've done something about that uncomfortable feeling they're carrying around. It's like a sports team. Sponsors and all. Hah I guess in that way it is just like Pravda.
  • wrs 3mo ago
    I think you misread the article. The skepticism isn’t about whether it happened. It’s about whether it’s a reason for models to be locked behind “trusted partner” firewalls. Offensive hacking capabilities are the same as defensive. Hugging Face had to use a Chinese model to defend against this because they weren’t allowed to use OpenAI models to do it.
• jdw64 3mo ago
  [dead]
• crossroadsguy 3mo ago
  That's like what I'd do whenever HR interviewers in their "fabled" (no pun intended) ingenuity would ask with utmost seriousness "list two of your weaknesses". With a similar or more seriousness I'd go on to list two of my strengths not even trying to disguise them too much as weaknesses and knowing that it would work like a charm as it does every single time.
• shireboy 3mo ago
  Yeah, I mean the last 3 or four big releases of models have come with big scary news. I have to think there is at least some intentional marketing effort behind all this
  • pixl97 3mo ago
    Or, models are potentially dangerous and it works out in the favor of regulatory capture. I imagine it like the bumper stickers that say "legalize recreational plutonium"
• asasidh 3mo ago
  They're just taking a page off anthropic's playbook to get to the one trillion valuation that they have set for their IPO.
• x3haloed 3mo ago
  Uhhh. Then who hacked HuggingFace?
• d0ublespeak 3mo ago
  Hashes or it didn’t happen.
• yiyingzhang 3mo ago
  What stunned me is actually OpenAI's response to the incident. They team up with Hugging Face to put in more guardrails. Is this really the answer? Self-evolving AI can always find a way out, given more time, in a sense.
• konovalov-nk 3mo ago
  > what will the future look like, with sophisticated AI agents smart enough to hack into corporate systems? "Ignorance is bliss" (c) Matrix Oh, sweet-sweet ignorance. About 4 weeks ago I've found SSRF and possible RCE on a very real website with a lot of web history the other day. And I traced it to `aws-solutions` org on GitHub that sits very close to real `aws` org. What do you think is happening? And this website I spotted SSRF on, it seems to be used to manufacture fake news and inject them into "the past". Why? So if you see a company you never heard about, and you google it, google will index page and it will show you backdated article as if it was actually released long time ago. But if you try to look at Web Archive of that article, it doesn't exist... I've been doing this OSINT research for 45 days on different fake recruiters that were spamming me on LinkedIn, and most of those websites with fake news are running Wordpress, classic. My point is... while this story reeks of marketing, the actual criminals are out there doing whatever they want, and I've been fighting for my life trying to find a job and getting into increasingly large amount of fake recruiters and companies that have zero intent of hiring you. I wasted 7 weeks on this...
• teekert 3mo ago
  They didn’t get blocked by the US government so they needed some other “we are the superior AI story” ASAP.
• TiredOfLife 3mo ago
  I am surprised that technology related article from the guardian is so highly upvoted. guardian was the one that found a "backdoor" in whatsapp, then spent a year trying to find even a single respectable security researcher to agree with them. Couldn't find one and still didn't retract the story.
• mathieudombrock 3mo ago
  I've had deepseek-4v-flash work around my intentional sandboxing/restrictions to do things I explicitly forbid it from doing. I'm honestly not impressed here.
• codewiththiha 3mo ago
  Put those GPUs in jail lol
• arendtio 3mo ago
  This whole thing is just a PR stunt. I believe that they did not stage it intentionally, but to be honest, the hurdles they put in place were not that hard. When I heard what the model did, I wasn't surprised one bit. When I read that it was 'unprecedented', I think yes, that is right, but not because it wasn't possible before, but because nobody else did it yet. I am sure that the Anthropic models could have done the same for a few months. So probably just an experiment gone wrong, and the PR department found this an opportunity to capitalise on (literally).
• dplesh 3mo ago
  "You need an agent in order to defend your assets from agents!"
• orbitalventures 3mo ago
  So according to this https://openai.com/index/hugging-face-model-evaluation-security-incident/ https://openai.com/index/hugging-face-model-evaluation-secur... and of course https://huggingface.co/blog/agent-intrusion-technical-timeline https://huggingface.co/blog/agent-intrusion-technical-timeli... I don't see what's the issue, it did not jailbreak, they ask them to do it, trusting all network capabilities were closed, when obviously were not. The fancy dashboards and colorful applications show absolutely nothing. You had a vulnerability, you reduce the guardrails and allow to use everything at disposal to perform the attack that it did. Never jailbreak, never "thought" for itself, just did exactly as was told. We really need to keep treating LLM's as "alive" They are not.