4 ms·

Important to note the actual title is "OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened" - the "that happened" is important

by simonw 3mo ago

Important to note the actual title is "OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened" - the "that happened" is important, otherwise it sounds like I think the attack was made up. Since it's buried towards the bottom I'll quote the section "Resist the temptation to write this off as a stunt" here in full https://simonwillison.net/2026/Jul/22/openai-cyberattack/#resist-the-temptation-to-write-this-off-as-a-stunt https://simonwillison.net/2026/Jul/22/openai-cyberattack/#re... > Resist the temptation to write this off as a stunt > There will inevitably be some people who dismiss this story as a dishonest marketing trick by OpenAI to make their models sound terrifyingly effective. I found 81 instances of the term “marketing” in the Hacker News discussion of the incident. > To those people I say pull your heads out of the sand - you’re now including Hugging Face in your conspiracy theories, just so you can deny the crescendo of evidence here! > The best models we have today have the ability to both find and exploit new vulnerabilities. The ExploitGym paper itself concludes that “autonomous exploit development by frontier AI agents is no longer a hypothetical capability”, and this incident is a perfect example of exactly that.

foobar10000 3mo ago
This is an _amazing_ typo :) Thank you, thank you :)
- Georgelemental 3mo ago
  Typo or HN character limit?
  - varenc 3mo ago
    the 'that happened' makes it too long for the HN submission title length limit. Maybe simonw can suggest an alternative title that fits within the limit, that doesn't misrepresent the post.
    - atmavatar 3mo ago
      One option is to drop "against Hugging Face". The "that happened" term seems a supremely important part of the title given the "is science fiction" term before it, as it clarifies the cyberattack isn't a made-up story. In contrast, the target, Hugging Face, is merely a detail that can be left for discovery upon reading the article. It's less important who was attacked than that the attack actually happened. Without knowing the exact character limit for titles and without having the motivation this late at night to count the current title length, you may also be able to drop the "accidental" to fit in "that happened", but I worry that leaves too much of a door open for someone to interpret the attack as deliberate. As such, I strongly prefer my first option.
    - jonas21 3mo ago
      How about "OpenAI's accidental cyberattack on Hugging Face is science fiction that happened"? It fits HN's 80-character limit exactly.
      - deleted 3mo ago
        [deleted]
    - simonw 3mo ago
      This fits: "OpenAI’s accidental cyberattack against Hugging Face is sci-fi that happened" Or "against HF"
deleted 3mo ago
[deleted]
gmerc 3mo ago
It can, it is both - PR spindoctoring not letting a good crisis go to waste to shape the regulatory conversation at the time the company needs it the most. Hacking is a felony and it matters not if you didn’t mean to if the other side were to press charges. Negligence is no excuse. And OpenAI has nowhere to run from the liability, as both operator and manufacturer. Alibaba did it first ( https://georgzoeller.com/blog/posts/alibaba-s-ai-deciding-to-go-forbidden-cryptomining-shows-exactly-where-the-ai-ri/ https://georgzoeller.com/blog/posts/alibaba-s-ai-deciding-to... ) and the fact that this happens again in a frontier lab is inexcusable and makes the case for operator liability and closing the liability sink of “AI did it”
- wbl 3mo ago
  The CFAA says knowingly. Negligence is by definition an excuse for that.
  - gmerc 3mo ago
    Removing the guardrails is knowingly given the risk
  - kibibu 3mo ago
    I think there's a reasonable case that the agent knew it was breaking into the system, for some definition of knew. I think OpenAI would be very reluctant to let this go to a place where the reasoning was part of discovery.
    - wbl 3mo ago
      Agents aren't subjects of criminal law. I agree there may be civil liability, I know far less about that.
      - kibibu 3mo ago
        Ok, but if the agent's reasoning log says "The best way to get into Hugging Face is to find and exploit a zero-day vulnerability", surely those responsible for monitoring its actions should be criminally liable. These guys would be screwed if they were operating under the EU AI Act.
Wurdan 3mo ago
There's no multi-party conspiracy theory required here. Events can have played out exactly as OpenAI and Hugging Face described, and OpenAI can also have reaped a huge amount of free marketing for the capability of their models from all this coverage (including your post). You're telling us to not be doubtful of the boy who cried wolf, when in actual fact the one doing the crying is the one training bigger and badder wolves (and trying to convince us that they hold the key to our safety from wolf attacks[1]). Also, the last section of your post appears to imply that if the attackers have bigger guns then the only possible solution is to give the defenders bigger guns. You're openly supporting an arms race towards the most capable, least restrained models put in the hands of the most possible people. That's extremely concerning. [1]: https://openai.com/index/scaling-trusted-access-for-cyber-defense/ https://openai.com/index/scaling-trusted-access-for-cyber-de...
- simonw 3mo ago
  I had real trouble deciding how to end this piece. There's no easy answer here. All of the options are bad in different ways! As a builder of software, I want access to the best possible tools to help me keep that software secure. As a user of software, I want my software to be secure and I don't want bad actors to be able to access tool to help them exploit it. Is the only answer here to have the AI labs make decisions over who gets access to the tools? What if they make mistakes in those decisions? None of the options look good to me. I don't know what we should do here. My hunch is that the open weight models are already forcing our hand. The dangerous capabilities are coming to everyone.
danparsonson 3mo ago
HuggingFace doesn't have to be complicit in a lie here - the impressiveness or not of what actually happened rests entirely on OpenAIs report about it, doesn't it?

by simonw 3mo ago

foobar10000 3mo ago

This is an _amazing_ typo :) Thank you, thank you :)

Georgelemental 3mo ago
Typo or HN character limit?
- varenc 3mo ago
  the 'that happened' makes it too long for the HN submission title length limit. Maybe simonw can suggest an alternative title that fits within the limit, that doesn't misrepresent the post.
  - atmavatar 3mo ago
    One option is to drop "against Hugging Face". The "that happened" term seems a supremely important part of the title given the "is science fiction" term before it, as it clarifies the cyberattack isn't a made-up story. In contrast, the target, Hugging Face, is merely a detail that can be left for discovery upon reading the article. It's less important who was attacked than that the attack actually happened. Without knowing the exact character limit for titles and without having the motivation this late at night to count the current title length, you may also be able to drop the "accidental" to fit in "that happened", but I worry that leaves too much of a door open for someone to interpret the attack as deliberate. As such, I strongly prefer my first option.
  - jonas21 3mo ago
    How about "OpenAI's accidental cyberattack on Hugging Face is science fiction that happened"? It fits HN's 80-character limit exactly.
    - deleted 3mo ago
      [deleted]
  - simonw 3mo ago
    This fits: "OpenAI’s accidental cyberattack against Hugging Face is sci-fi that happened" Or "against HF"

deleted 3mo ago

[deleted]

gmerc 3mo ago

It can, it is both - PR spindoctoring not letting a good crisis go to waste to shape the regulatory conversation at the time the company needs it the most. Hacking is a felony and it matters not if you didn’t mean to if the other side were to press charges. Negligence is no excuse. And OpenAI has nowhere to run from the liability, as both operator and manufacturer. Alibaba did it first ( https://georgzoeller.com/blog/posts/alibaba-s-ai-deciding-to-go-forbidden-cryptomining-shows-exactly-where-the-ai-ri/ https://georgzoeller.com/blog/posts/alibaba-s-ai-deciding-to... ) and the fact that this happens again in a frontier lab is inexcusable and makes the case for operator liability and closing the liability sink of “AI did it”

wbl 3mo ago
The CFAA says knowingly. Negligence is by definition an excuse for that.
- gmerc 3mo ago
  Removing the guardrails is knowingly given the risk
- kibibu 3mo ago
  I think there's a reasonable case that the agent knew it was breaking into the system, for some definition of knew. I think OpenAI would be very reluctant to let this go to a place where the reasoning was part of discovery.
  - wbl 3mo ago
    Agents aren't subjects of criminal law. I agree there may be civil liability, I know far less about that.
    - kibibu 3mo ago
      Ok, but if the agent's reasoning log says "The best way to get into Hugging Face is to find and exploit a zero-day vulnerability", surely those responsible for monitoring its actions should be criminally liable. These guys would be screwed if they were operating under the EU AI Act.

Wurdan 3mo ago

There's no multi-party conspiracy theory required here. Events can have played out exactly as OpenAI and Hugging Face described, and OpenAI can also have reaped a huge amount of free marketing for the capability of their models from all this coverage (including your post). You're telling us to not be doubtful of the boy who cried wolf, when in actual fact the one doing the crying is the one training bigger and badder wolves (and trying to convince us that they hold the key to our safety from wolf attacks[1]). Also, the last section of your post appears to imply that if the attackers have bigger guns then the only possible solution is to give the defenders bigger guns. You're openly supporting an arms race towards the most capable, least restrained models put in the hands of the most possible people. That's extremely concerning. [1]: https://openai.com/index/scaling-trusted-access-for-cyber-defense/ https://openai.com/index/scaling-trusted-access-for-cyber-de...

simonw 3mo ago
I had real trouble deciding how to end this piece. There's no easy answer here. All of the options are bad in different ways! As a builder of software, I want access to the best possible tools to help me keep that software secure. As a user of software, I want my software to be secure and I don't want bad actors to be able to access tool to help them exploit it. Is the only answer here to have the AI labs make decisions over who gets access to the tools? What if they make mistakes in those decisions? None of the options look good to me. I don't know what we should do here. My hunch is that the open weight models are already forcing our hand. The dangerous capabilities are coming to everyone.

danparsonson 3mo ago

HuggingFace doesn't have to be complicit in a lie here - the impressiveness or not of what actually happened rests entirely on OpenAIs report about it, doesn't it?