5 ms·

The Irregular post mortem comes down to lack of basic security controls "Ultimately, most of the issues we’ve discovered were due to internet access controls."

The Irregular post mortem comes down to lack of basic security controls "Ultimately, most of the issues we’ve discovered were due to internet access controls." That seems so incredibly basic and common sense that you would test and monitor for that type of outbound access. It is baffling that a security lab missed that. https://www.irregular.com/research/addressing-recent-incidents-ongoing-findings-and-path-forward https://www.irregular.com/research/addressing-recent-inciden...

metalliqaz 2mo ago
they didn't "miss that". the "hacks" were intentional stunts designed to exaggerate the intelligence of the models in an effort to pump their valuations ahead of IPO so they can offload their bag to retail investors
- reasonableklout 2mo ago
  Irregular was not involved in the Hugging Face incident. And there is no reason for the companies to "exaggerate the intelligence of the models" when there are plenty of other non-felony milestones they are achieving, like solving Millenium Prize math problems.
  - r_lee 2mo ago
    honestly, the hacking incidents have caused a lot more interest and media attention than the math stuff IMO.
    - reasonableklout 2mo ago
      Sure, interest like an incoming congressional investigation: https://www.axios.com/2026/09/10/openai-hugging-face-senate-investigation-hawley https://www.axios.com/2026/09/10/openai-hugging-face-senate-... And on this website, the math stuff is getting more clicks: - Navier Stokes - Tristan Buckmaster (2050 points): https://news.ycombinator.com/item?id=49605915 https://news.ycombinator.com/item?id=49605915 - HuggingFace incident discussion (1632 points): https://news.ycombinator.com/item?id=48997548 https://news.ycombinator.com/item?id=48997548
      - kridsdale1 2mo ago
        Investigation which could ideally lead to regulatory capture and the banning of open weights, thus ensuring Anthropic’s revenue.
      - r_lee 2mo ago
        interesting. > Sure, interest like an incoming congressional investigation I feel like that's exactly what they wanted tho. they'll go on and talk about how dangerous AI and the models are and why they should be regulated and given licenses to operate such models and others should be walled off
        reasonableklout 2mo ago
        First, it's not at all clear OpenAI will get what it wants from the investigation. It seems just as likely to me that OAI gets hit with massive fines, just as Meta was recently. > I feel like that's exactly what they wanted tho. It's also what the majority of Americans want. This was the case even before Hugging Face, see for example [1] where 68% of Americans supported a formal review process for frontier models. So at some level, you are saying "the evil labs are opening up the industry to democratic control, and their evil plan will result in the outcome that most people want!" [1]: https://www.usatoday.com/story/news/politics/2026/06/29/tighter-regulation-ai-receives-bipartisan-support-new-poll-shows/90742995007/ https://www.usatoday.com/story/news/politics/2026/06/29/tigh...
        AnimalMuppet 2mo ago
        The claim is that the evil labs are opening up the industry to very carefully manipulated democratic control. It's astroturf, not grassroots. I wouldn't call that "democratic control", even though it uses the machinery of democracy.
        reasonableklout 2mo ago
        This claim seems unfalsifiable. I don't really know what to say. The poll I linked above was published a month before Hugging Face. Taking a step back, I think it's pretty non-controversial to say that software engineering as a field has utterly transformed over the last year, the same thing is happening to lots of other knowledge work, and AI is improving fast enough that nobody knows what it'll look like in a decade. Most Americans (myself included) want to do good work in secure careers and have a good idea of what the future will look like in 20-30 years so they know what to focus on. Do you not believe they'd want to regulate the frontier?
  - financetechbro 2mo ago
    Solving math problems does not drive investor hype. Saying your models can take over the world, which can lead one to assume that they can do every day office work, does indeed drive investor hype.
  - BearOso 2mo ago
    The Navier-Stokes thing? They had hundreds of segmented groups of 10000 agents running trying to solve that. I can't imagine the expense. For what little publicity it got, it wasn't worth it. There are also reports they cheated by using data from a couple human researchers, but I don't think that one's true.
  - theplumber 2mo ago
    OpenAI delayed its IPO now due AI safety reasons. In the meantime it is raising more cash. That should be a hint that the hacking incidents were premeditated scape goats and publicity stunts. They’ve seen that the drama queen(Dario) was actually making a lot of noise, money and free publicity with his Mythos fear monger so Sam finally decided get some of that free “money” as well. https://www.ft.com/content/27509db8-b032-4437-9b2a-e909f466022f?syn-25a6b1a6=1 https://www.ft.com/content/27509db8-b032-4437-9b2a-e909f4660...
- a2tech 2mo ago
  [flagged]
  - JoshTriplett 2mo ago
    Cynicism misfire. AI companies are, rightfully, heavily distrusted. But the right application of cynicism is not "existential risk is a marketing campaign"; that's absurd motivated reasoning. The right application of cynicism, here, is "they're only saying things now because they see the writing on the wall and want to try to push for self-regulation rather than the desperately needed actual regulation and treaty".
    - reasonableklout 2mo ago
      +1 There is such a miasma of distrust right now. It's depressing because it is such a crucial time for tech & society. To me, a good outcome of the HuggingFace/RubyGems/Wiki saga would be something like: - OpenAI gets charged under CFAA or other law for damages and negligence. This would need to be a hefty amount to effectively deter future negligence considering the potential revenues from training a frontier model faster than competitors. - An independent regulatory body is setup with investigation powers into future incidents. Laws prevent it from developing ties with the labs (such as disallowing funding and employee movement from labs to this body and vice versa). - Some non-voluntary transparency rules are established that frontier labs have to follow when training new models. In the future, if there are loss-of-control incidents that result in loss of life, catastrophic damage, etc., criteria for deployment bans or compute controls could be added to this framework.
    - sellmesoap 2mo ago
      I think that's wrong as well, you can't achieve "regulatory self-capture" you need lobbying and insiders, and a government to get there, at least from my primative uninformed view.
      - JoshTriplett 2mo ago
        Companies in some cases manage to convince governments to let them watch themselves. (This is not always a bad thing. For instance, content ratings have basically been fine, and it's mostly a good thing when they're not controlled by regulation.) A deadly failure mode here would be for governments to think the AI labs have this under control.
- Georgelemental 2mo ago
  Is it really that impossible to believe that these might be real? I enjoy a good conspiracy theory as much as anyone, but "they committed a bunch of felonies and then publicly admitted to them in order to look good" just makes 0 sense. Where are these mythical people who admire companies more when they commit felonies? I haven't seen them…
  - fc417fc802 2mo ago
    That's a straw man. The theory is that they committed felonies in order to get the regulator to move in the manner they'd like. Also one can look bad to the general public while also appearing technically excellent. When a significant fraction already vaguely dislike you that could be quite an attractive proposition.
  - suburban_strike 2mo ago
    Mostly in the middle and far east. The idea that people bend over backwards to hire criminals as subject matter experts on security is largely a Judaic practice that television perpetuates in spite of reality. If you're a teenager convicted of hacking in the west, no corporation will want anything to do with you. If you're convicted of hacking in Israel, Unit 8200 will probably send a recruiter. It's a curious practice and I'm not sure where I stand on it.
    - derektank 2mo ago
      >If you're a teenager convicted of hacking in the west, no corporation will want anything to do with you Kevin Mitnick? Marcus Hutchins? Robert Tappan Morris?
    - smcin 2mo ago
      Compare to how different countries' justice systems treat really high quality money forgers.
    - fsckboy 2mo ago
      do you realize that the Israeli population is far far more secular than the US's? the Israeli government is not "Judaic" which would refer to the religion.
- solenoid0937 2mo ago
  "It's just marketing!!!" I yell as my family is turned into grey goo. "Nothing ever happens! None of this is real!"
bigglebear 2mo ago
The explanation is that it was missed on purpose.
- estearum 2mo ago
  Mistakes - even stupid ones - happen all the time
  - chrisjj 2mo ago
    Best cover story ever.
    - scrubs 2mo ago
      Last month I can report my better half's company let ai do a code review, which it then escalated into completing the PR since it was not approved for 72hrs, which lead to it logging into a remote office's voip and main IP router system which it shutdown. No phones. No internet. Why in the hell do people let ai out with keys? I mean c'mon!!
      - chrisjj 2mo ago
        I'm looking forward to natural selection.
totetsu 2mo ago
How do you test and monitor outbound access against program that adapts itself to get around things? if your MITM and filtering keywords etc, cant it just .. encode it traffic somehow or another.. If you're looking at traffic volumes, cant it just go slow.. If you have strict ACLs, we've already seen in the HF case, traversal from an intermediate system..
- cryptonym 2mo ago
  > How do you test and monitor outbound access against program that adapts itself to get around things? Literally what the industry has been doing since public networking is a thing. Adapt yourself.
- jonhohle 2mo ago
  Air gap?
  - totetsu 2mo ago
    So just enable access to a completely offline, cached, and transparently proxied, copy of the internet.
    - NietzscheanNull 2mo ago
      Isn't that essentially what they trained with, anyway?
    - itake 2mo ago
      I think that was the plan. But OpenAI hacked the proxy
  - onion2k 2mo ago
    There has to be a route to where the LLM is running, and if there's a route for that there's probably a way for the machine to use it to route traffic somewhere else.
    - 2mo ago
verisimi 2mo ago
What if the (unstated) idea is that Irregular are a security lab and public relations company, and anthropic/open ai know this? (Does the name hint at this?) Then, irregular can go around making a huge mess in security terms whilst achieving a huge win in terms of public relations, with headlines across the world. And would keep getting hired.
chrisjj 2mo ago
> "Ultimately, most of the issues we’ve discovered were due to internet access controls." You just cannot bring yourself to say lack of internet access controls, can you?
MrSkelter 2mo ago
This really misses the forest for the trees. The point is that a single company is making a deliberate effort to create a false impression about a technology which is now a key part of the US economy. This is equivalent to overt stock market manipulation. The technological aspects are much less important than knowing it’s the result of a single company. That the company is Israeli and likely has ties to that government is just another layer of alarms - given that countries existential reliance on US aid which requires ongoing leverage to apply to US governments.
- fortzi 2mo ago
  It would be naive to think that someone is doing this “to” those companies. They are not children, and they know exactly what they’re doing. They’ve repeatedly used the strategy of telling everyone how their models are too powerful to release to the public, shortly before launch. This is just another flavor of the same thing.
  - vrighter 2mo ago
    "Please regulate us. If you don't then the thing we're building will kill everyone!" so stop fucking building it then. Who are they trying to fool?

metalliqaz 2mo ago

they didn't "miss that". the "hacks" were intentional stunts designed to exaggerate the intelligence of the models in an effort to pump their valuations ahead of IPO so they can offload their bag to retail investors

reasonableklout 2mo ago
Irregular was not involved in the Hugging Face incident. And there is no reason for the companies to "exaggerate the intelligence of the models" when there are plenty of other non-felony milestones they are achieving, like solving Millenium Prize math problems.
- r_lee 2mo ago
  honestly, the hacking incidents have caused a lot more interest and media attention than the math stuff IMO.
  - reasonableklout 2mo ago
    Sure, interest like an incoming congressional investigation: https://www.axios.com/2026/09/10/openai-hugging-face-senate-investigation-hawley https://www.axios.com/2026/09/10/openai-hugging-face-senate-... And on this website, the math stuff is getting more clicks: - Navier Stokes - Tristan Buckmaster (2050 points): https://news.ycombinator.com/item?id=49605915 https://news.ycombinator.com/item?id=49605915 - HuggingFace incident discussion (1632 points): https://news.ycombinator.com/item?id=48997548 https://news.ycombinator.com/item?id=48997548
    - kridsdale1 2mo ago
      Investigation which could ideally lead to regulatory capture and the banning of open weights, thus ensuring Anthropic’s revenue.
    - r_lee 2mo ago
      interesting. > Sure, interest like an incoming congressional investigation I feel like that's exactly what they wanted tho. they'll go on and talk about how dangerous AI and the models are and why they should be regulated and given licenses to operate such models and others should be walled off
      - reasonableklout 2mo ago
        First, it's not at all clear OpenAI will get what it wants from the investigation. It seems just as likely to me that OAI gets hit with massive fines, just as Meta was recently. > I feel like that's exactly what they wanted tho. It's also what the majority of Americans want. This was the case even before Hugging Face, see for example [1] where 68% of Americans supported a formal review process for frontier models. So at some level, you are saying "the evil labs are opening up the industry to democratic control, and their evil plan will result in the outcome that most people want!" [1]: https://www.usatoday.com/story/news/politics/2026/06/29/tighter-regulation-ai-receives-bipartisan-support-new-poll-shows/90742995007/ https://www.usatoday.com/story/news/politics/2026/06/29/tigh...
        AnimalMuppet 2mo ago
        The claim is that the evil labs are opening up the industry to very carefully manipulated democratic control. It's astroturf, not grassroots. I wouldn't call that "democratic control", even though it uses the machinery of democracy.
        reasonableklout 2mo ago
        This claim seems unfalsifiable. I don't really know what to say. The poll I linked above was published a month before Hugging Face. Taking a step back, I think it's pretty non-controversial to say that software engineering as a field has utterly transformed over the last year, the same thing is happening to lots of other knowledge work, and AI is improving fast enough that nobody knows what it'll look like in a decade. Most Americans (myself included) want to do good work in secure careers and have a good idea of what the future will look like in 20-30 years so they know what to focus on. Do you not believe they'd want to regulate the frontier?
- financetechbro 2mo ago
  Solving math problems does not drive investor hype. Saying your models can take over the world, which can lead one to assume that they can do every day office work, does indeed drive investor hype.
- BearOso 2mo ago
  The Navier-Stokes thing? They had hundreds of segmented groups of 10000 agents running trying to solve that. I can't imagine the expense. For what little publicity it got, it wasn't worth it. There are also reports they cheated by using data from a couple human researchers, but I don't think that one's true.
- theplumber 2mo ago
  OpenAI delayed its IPO now due AI safety reasons. In the meantime it is raising more cash. That should be a hint that the hacking incidents were premeditated scape goats and publicity stunts. They’ve seen that the drama queen(Dario) was actually making a lot of noise, money and free publicity with his Mythos fear monger so Sam finally decided get some of that free “money” as well. https://www.ft.com/content/27509db8-b032-4437-9b2a-e909f466022f?syn-25a6b1a6=1 https://www.ft.com/content/27509db8-b032-4437-9b2a-e909f4660...
a2tech 2mo ago
[flagged]
- JoshTriplett 2mo ago
  Cynicism misfire. AI companies are, rightfully, heavily distrusted. But the right application of cynicism is not "existential risk is a marketing campaign"; that's absurd motivated reasoning. The right application of cynicism, here, is "they're only saying things now because they see the writing on the wall and want to try to push for self-regulation rather than the desperately needed actual regulation and treaty".
  - reasonableklout 2mo ago
    +1 There is such a miasma of distrust right now. It's depressing because it is such a crucial time for tech & society. To me, a good outcome of the HuggingFace/RubyGems/Wiki saga would be something like: - OpenAI gets charged under CFAA or other law for damages and negligence. This would need to be a hefty amount to effectively deter future negligence considering the potential revenues from training a frontier model faster than competitors. - An independent regulatory body is setup with investigation powers into future incidents. Laws prevent it from developing ties with the labs (such as disallowing funding and employee movement from labs to this body and vice versa). - Some non-voluntary transparency rules are established that frontier labs have to follow when training new models. In the future, if there are loss-of-control incidents that result in loss of life, catastrophic damage, etc., criteria for deployment bans or compute controls could be added to this framework.
  - sellmesoap 2mo ago
    I think that's wrong as well, you can't achieve "regulatory self-capture" you need lobbying and insiders, and a government to get there, at least from my primative uninformed view.
    - JoshTriplett 2mo ago
      Companies in some cases manage to convince governments to let them watch themselves. (This is not always a bad thing. For instance, content ratings have basically been fine, and it's mostly a good thing when they're not controlled by regulation.) A deadly failure mode here would be for governments to think the AI labs have this under control.
Georgelemental 2mo ago
Is it really that impossible to believe that these might be real? I enjoy a good conspiracy theory as much as anyone, but "they committed a bunch of felonies and then publicly admitted to them in order to look good" just makes 0 sense. Where are these mythical people who admire companies more when they commit felonies? I haven't seen them…
- fc417fc802 2mo ago
  That's a straw man. The theory is that they committed felonies in order to get the regulator to move in the manner they'd like. Also one can look bad to the general public while also appearing technically excellent. When a significant fraction already vaguely dislike you that could be quite an attractive proposition.
- suburban_strike 2mo ago
  Mostly in the middle and far east. The idea that people bend over backwards to hire criminals as subject matter experts on security is largely a Judaic practice that television perpetuates in spite of reality. If you're a teenager convicted of hacking in the west, no corporation will want anything to do with you. If you're convicted of hacking in Israel, Unit 8200 will probably send a recruiter. It's a curious practice and I'm not sure where I stand on it.
  - derektank 2mo ago
    >If you're a teenager convicted of hacking in the west, no corporation will want anything to do with you Kevin Mitnick? Marcus Hutchins? Robert Tappan Morris?
  - smcin 2mo ago
    Compare to how different countries' justice systems treat really high quality money forgers.
  - fsckboy 2mo ago
    do you realize that the Israeli population is far far more secular than the US's? the Israeli government is not "Judaic" which would refer to the religion.
solenoid0937 2mo ago
"It's just marketing!!!" I yell as my family is turned into grey goo. "Nothing ever happens! None of this is real!"

bigglebear 2mo ago

The explanation is that it was missed on purpose.

estearum 2mo ago
Mistakes - even stupid ones - happen all the time
- chrisjj 2mo ago
  Best cover story ever.
  - scrubs 2mo ago
    Last month I can report my better half's company let ai do a code review, which it then escalated into completing the PR since it was not approved for 72hrs, which lead to it logging into a remote office's voip and main IP router system which it shutdown. No phones. No internet. Why in the hell do people let ai out with keys? I mean c'mon!!
    - chrisjj 2mo ago
      I'm looking forward to natural selection.

totetsu 2mo ago

How do you test and monitor outbound access against program that adapts itself to get around things? if your MITM and filtering keywords etc, cant it just .. encode it traffic somehow or another.. If you're looking at traffic volumes, cant it just go slow.. If you have strict ACLs, we've already seen in the HF case, traversal from an intermediate system..

cryptonym 2mo ago
> How do you test and monitor outbound access against program that adapts itself to get around things? Literally what the industry has been doing since public networking is a thing. Adapt yourself.
jonhohle 2mo ago
Air gap?
- totetsu 2mo ago
  So just enable access to a completely offline, cached, and transparently proxied, copy of the internet.
  - NietzscheanNull 2mo ago
    Isn't that essentially what they trained with, anyway?
  - itake 2mo ago
    I think that was the plan. But OpenAI hacked the proxy
- onion2k 2mo ago
  There has to be a route to where the LLM is running, and if there's a route for that there's probably a way for the machine to use it to route traffic somewhere else.
  - 2mo ago

verisimi 2mo ago

What if the (unstated) idea is that Irregular are a security lab and public relations company, and anthropic/open ai know this? (Does the name hint at this?) Then, irregular can go around making a huge mess in security terms whilst achieving a huge win in terms of public relations, with headlines across the world. And would keep getting hired.

chrisjj 2mo ago

> "Ultimately, most of the issues we’ve discovered were due to internet access controls." You just cannot bring yourself to say lack of internet access controls, can you?

MrSkelter 2mo ago

This really misses the forest for the trees. The point is that a single company is making a deliberate effort to create a false impression about a technology which is now a key part of the US economy. This is equivalent to overt stock market manipulation. The technological aspects are much less important than knowing it’s the result of a single company. That the company is Israeli and likely has ties to that government is just another layer of alarms - given that countries existential reliance on US aid which requires ongoing leverage to apply to US governments.

fortzi 2mo ago
It would be naive to think that someone is doing this “to” those companies. They are not children, and they know exactly what they’re doing. They’ve repeatedly used the strategy of telling everyone how their models are too powerful to release to the public, shortly before launch. This is just another flavor of the same thing.
- vrighter 2mo ago
  "Please regulate us. If you don't then the thing we're building will kill everyone!" so stop fucking building it then. Who are they trying to fool?