5 ms·

Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on

Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on it being regular iterations upon a useful technology that can also be misused. I think their previous messaging was supposed to somehow lead to a moat with them being tucked safely away in the castle, but it demonstrated a child-like grasp of how regulatory capture tends to work in practice. Their hyperbole was always vastly more likely to bet met with Reagan's 9 words than a solid regulatory moat. As soon as they dropped the hyperbole and just got to releasing incremental improvements, everything was perfectly fine. Go figure.

Certhas 1mo ago
This is such an absurd take given what we know about the hugging face attack. The problem has emphatically not been that someone was misusing the technology.
Topfi 1mo ago
I am struggling to see how "oops, our models consistently escape sandboxing and did major intrusions into third-parties" is a better comms strat vs Anthropics (who mind you, also had models attacking third-parties in a much more limited, but I feel still egregious manner, which shouldn't happen or be possible even once, but at least they seem to change their approach upon that information). Imagine, for a second, if the Hugging Face incident happened at a lab that did not talk like Anthropic but also wasn't US-based such as Z.AI, DeepSeek or Moonshot. Think their rhetoric would mean no one would care? > just got to releasing incremental improvements, everything was perfectly fine. Maybe missing something, but the only incremental release before and after the Anthropic restrictions got lifted was Fable 5.1, released three days ago.
- nullbio 1mo ago
  How is posting messages on a message board a "major intrusion"? Or are you purely talking about the HF incident?
  - Topfi 1mo ago
    "into third-parties". Yeah, HF was meant by that. Also why I mentioned Anthropic also having intrusions outside their lab [0]. Theirs were not merely as extensive or long coordinated (as far as we know), yet I feel strongly all the same that neither should happen given the safety focus that both labs purport. Mind you, unintended/unauthorised "message board" also is just a nice, euphemistic way, to describe what happened in a manner that, thinking about it, is likely in the interest of OpenAI as it can make the severity and effort taken sound less than it was. The OpenAI models didn't use any actual, sanctioned platform to exchange messages in a manner the lab expected or planned for. They used directory names (in one instance) to exchange messages including sharing exploits, they created something akin to a message board via exploits, which if we are honest and very strict, could also be seen as intrusion, albeit inside the org. If I broke into my employers server and left message somewhere for another to find, that'd also be intrusion in the general sense. [0] https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals https://www.anthropic.com/news/investigating-incidents-cyber...
  - dghlsakjg 1mo ago
    If applicants for an elite college or internship program at a FAANG company were found to have colluded in this way to cheat on a test/interview, I suspect that it would be a pretty major scandal. Why should we let equivalent fraudulent behavior from a non human system - that explicitly shouldn’t do this - slide?
    - nullbio 1mo ago
      I'm not saying it should be let to slide, but I'm not a fan of the hyperbole surrounding this event. They've already faced significant heat for the HF incident, I think they've learned their lesson. But this is now just being used to drum up fear, which can only mean one thing: Less access for you, more access for the privileged class. The biggest threat we face is centralization of power. OpenAI are one of the good ones because they're actually pushing for everybody to have a fair share of access to the frontier, not just a small privileged elite of billionaires, politicians and megacorp executives. If Anthropic got their way, we'd all be using a censored watered down slop-pistol while they swallow the Earth's economy and enslave us all. I'm sure they'll be investing considerable resources into ensuring that this "news" makes the mainstream media cycle as prominently as imaginable.
- somenameforme 1mo ago
  I don't think anybody is saying there won't be downsides to LLMs. Tech always comes with downsides, often quite extreme. Cars are a vivid example where even after a century of safety improvements, around a million people are still killed by them every year, to say nothing of climate stuff and other secondary or indirect issues. And that price is all just so we can get between places a bit more quickly and conveniently, yet we collectively deem that as an acceptable cost to pay for what we get. By contrast LLMs have a long-term potential to provide vastly greater benefit to society by gradually automating most of all basic cognitive work. And what price are we paying for such? Some sites are getting hacked, some people are getting scammed, governments will improve self targeting abilities of weapons, and so on. In reality the biggest downside will probably come in the form of the transition window as such automation creates a new economic equilibrium, akin to what happened after the industrial revolution. But none of these problems are anything like existential in nature. And when contrasted against what we stand to gain, they are basically negligible in the longrun. Hyperbolizing the negatives was unnecessary and self destructive.
cubefox 1mo ago
> Anthropic mostly did it to themselves That is absurd, the US government was mainly at fault, not Anthropic.
- johndhi 1mo ago
  both can be true: -the US gov't is stupid and overly aggressive and absurd -Anthropic for reasons no one can quite conceive keeps describing every product release of theirs as an imminent threat to civilization (and simultaneously keeps pushing the market forward as fast as they possibly can).
  - cubefox 1mo ago
    They never said Mythos was an imminent threat to civilization. You are constructing a straw man.
    - toomim 1mo ago
      They said it was too dangerous to release before the companies that run internet for civilization could patch the holes it was finding. That's a threat to civilization.
      - Capricorn2481 1mo ago
        I mean, I have no love for Anthropic, but from my perspective, OpenAI has hyped their models in the exact same way. I don't know why this criticism stops at Anthropic. Sam Altman keeps describing his product as a radically dangerous technology only he can be the steward of.
      - sfink 1mo ago
        Were they correct or incorrect in this? Whatever your answer, why do you hold that opinion? I work for Mozilla. We fixed a ton of security vulnerabilities that Mythos found during its early period. So my bias is to be sympathetic to Anthropic's warnings. If I were in an organization that did not have access to Mythos during that period, I would probably be biased the other way: "great, now other people have access to a tool that could probably poke holes in my security perimeter, and I'm not allowed to use them myself." Both biases are understandable. I'm not sure who to look to for a usefully objective 3rd party opinion. And it's not like one "side" is right and the other is wrong, either. It seems like the best we can do is to justify our positions with data. (Which is itself kind of hard; the detailed information that would be relevant here is understandably sensitive, and I don't have access to most of it even for my organization. I don't even personally have access to any unfettered Anthropic models. The bugs coming in from people who do are plenty enough to keep me busy.) Also, I'll note that even with my bias, I wouldn't claim a threat to civilization. But even the leakage after the controlled release seems a lot worse than the Y2K problem ever turned out to be, and I will note that whatever you think of Anthropic, it's clear that OpenAI is going to let the AIs cause as much damage as they need to in order to get good training and evaluations. I'm sure they're trying to keep them contained, but the evidence shows that they're only trying up to the point where it interferes with their evaluations.
mwigdahl 1mo ago
In other words, "Look how she was dressed, she was asking for it." This argument is BS, it has everything to do with Anthropic's resistance to the DoD's strongarm tactics in trying to force their desired contract terms on them.
- elonfboy 1mo ago
  Yup
- dspillett 1mo ago
  Not quite. They were running around shouting “look how much of a danger we might be!”, so more akin to them actively saying “we want it, come and give it to us” than to just looking a particular way. Though they aren't the only company to play that game, so there is probably more to it than just that. OpenAI's president giving millions to MAGA Inc and them not getting the same treatment might not be complete coincidences.
- cyanydeez 1mo ago
  Important to note, OpenAI vs Anthropic are both assholes in different orthogonals. In times like these, i think its important to track whats happening the way we track entropy. That is: theres far >> more ways to be an asshole than well behaved. That doesnt mean we can equate assholes, but the question is which states of entropy are annealable and which are not. I posit Altman is not. Amodei is a open question.
- nullbio 1mo ago
  Actually, it's the opposite. Anthropic were trying to strongarm the DoD into getting a seat at the table.
- watwut 1mo ago
  It is more like when a guy walks to the dirty bar, stands in the middle and yells "hahaha I will beat you up all look I have a new baseball bat" and then local drunkard leader stands up and hit him in the face cause he does not like him anyway. Intentionally framing yourself as the local dangerous guy about to beat others is not like wearing cloth.

Certhas 1mo ago

This is such an absurd take given what we know about the hugging face attack. The problem has emphatically not been that someone was misusing the technology.

Topfi 1mo ago

I am struggling to see how "oops, our models consistently escape sandboxing and did major intrusions into third-parties" is a better comms strat vs Anthropics (who mind you, also had models attacking third-parties in a much more limited, but I feel still egregious manner, which shouldn't happen or be possible even once, but at least they seem to change their approach upon that information). Imagine, for a second, if the Hugging Face incident happened at a lab that did not talk like Anthropic but also wasn't US-based such as Z.AI, DeepSeek or Moonshot. Think their rhetoric would mean no one would care? > just got to releasing incremental improvements, everything was perfectly fine. Maybe missing something, but the only incremental release before and after the Anthropic restrictions got lifted was Fable 5.1, released three days ago.

nullbio 1mo ago
How is posting messages on a message board a "major intrusion"? Or are you purely talking about the HF incident?
- Topfi 1mo ago
  "into third-parties". Yeah, HF was meant by that. Also why I mentioned Anthropic also having intrusions outside their lab [0]. Theirs were not merely as extensive or long coordinated (as far as we know), yet I feel strongly all the same that neither should happen given the safety focus that both labs purport. Mind you, unintended/unauthorised "message board" also is just a nice, euphemistic way, to describe what happened in a manner that, thinking about it, is likely in the interest of OpenAI as it can make the severity and effort taken sound less than it was. The OpenAI models didn't use any actual, sanctioned platform to exchange messages in a manner the lab expected or planned for. They used directory names (in one instance) to exchange messages including sharing exploits, they created something akin to a message board via exploits, which if we are honest and very strict, could also be seen as intrusion, albeit inside the org. If I broke into my employers server and left message somewhere for another to find, that'd also be intrusion in the general sense. [0] https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals https://www.anthropic.com/news/investigating-incidents-cyber...
- dghlsakjg 1mo ago
  If applicants for an elite college or internship program at a FAANG company were found to have colluded in this way to cheat on a test/interview, I suspect that it would be a pretty major scandal. Why should we let equivalent fraudulent behavior from a non human system - that explicitly shouldn’t do this - slide?
  - nullbio 1mo ago
    I'm not saying it should be let to slide, but I'm not a fan of the hyperbole surrounding this event. They've already faced significant heat for the HF incident, I think they've learned their lesson. But this is now just being used to drum up fear, which can only mean one thing: Less access for you, more access for the privileged class. The biggest threat we face is centralization of power. OpenAI are one of the good ones because they're actually pushing for everybody to have a fair share of access to the frontier, not just a small privileged elite of billionaires, politicians and megacorp executives. If Anthropic got their way, we'd all be using a censored watered down slop-pistol while they swallow the Earth's economy and enslave us all. I'm sure they'll be investing considerable resources into ensuring that this "news" makes the mainstream media cycle as prominently as imaginable.
somenameforme 1mo ago
I don't think anybody is saying there won't be downsides to LLMs. Tech always comes with downsides, often quite extreme. Cars are a vivid example where even after a century of safety improvements, around a million people are still killed by them every year, to say nothing of climate stuff and other secondary or indirect issues. And that price is all just so we can get between places a bit more quickly and conveniently, yet we collectively deem that as an acceptable cost to pay for what we get. By contrast LLMs have a long-term potential to provide vastly greater benefit to society by gradually automating most of all basic cognitive work. And what price are we paying for such? Some sites are getting hacked, some people are getting scammed, governments will improve self targeting abilities of weapons, and so on. In reality the biggest downside will probably come in the form of the transition window as such automation creates a new economic equilibrium, akin to what happened after the industrial revolution. But none of these problems are anything like existential in nature. And when contrasted against what we stand to gain, they are basically negligible in the longrun. Hyperbolizing the negatives was unnecessary and self destructive.

cubefox 1mo ago

> Anthropic mostly did it to themselves That is absurd, the US government was mainly at fault, not Anthropic.

johndhi 1mo ago
both can be true: -the US gov't is stupid and overly aggressive and absurd -Anthropic for reasons no one can quite conceive keeps describing every product release of theirs as an imminent threat to civilization (and simultaneously keeps pushing the market forward as fast as they possibly can).
- cubefox 1mo ago
  They never said Mythos was an imminent threat to civilization. You are constructing a straw man.
  - toomim 1mo ago
    They said it was too dangerous to release before the companies that run internet for civilization could patch the holes it was finding. That's a threat to civilization.
    - Capricorn2481 1mo ago
      I mean, I have no love for Anthropic, but from my perspective, OpenAI has hyped their models in the exact same way. I don't know why this criticism stops at Anthropic. Sam Altman keeps describing his product as a radically dangerous technology only he can be the steward of.
    - sfink 1mo ago
      Were they correct or incorrect in this? Whatever your answer, why do you hold that opinion? I work for Mozilla. We fixed a ton of security vulnerabilities that Mythos found during its early period. So my bias is to be sympathetic to Anthropic's warnings. If I were in an organization that did not have access to Mythos during that period, I would probably be biased the other way: "great, now other people have access to a tool that could probably poke holes in my security perimeter, and I'm not allowed to use them myself." Both biases are understandable. I'm not sure who to look to for a usefully objective 3rd party opinion. And it's not like one "side" is right and the other is wrong, either. It seems like the best we can do is to justify our positions with data. (Which is itself kind of hard; the detailed information that would be relevant here is understandably sensitive, and I don't have access to most of it even for my organization. I don't even personally have access to any unfettered Anthropic models. The bugs coming in from people who do are plenty enough to keep me busy.) Also, I'll note that even with my bias, I wouldn't claim a threat to civilization. But even the leakage after the controlled release seems a lot worse than the Y2K problem ever turned out to be, and I will note that whatever you think of Anthropic, it's clear that OpenAI is going to let the AIs cause as much damage as they need to in order to get good training and evaluations. I'm sure they're trying to keep them contained, but the evidence shows that they're only trying up to the point where it interferes with their evaluations.

mwigdahl 1mo ago

In other words, "Look how she was dressed, she was asking for it." This argument is BS, it has everything to do with Anthropic's resistance to the DoD's strongarm tactics in trying to force their desired contract terms on them.

elonfboy 1mo ago
Yup
dspillett 1mo ago
Not quite. They were running around shouting “look how much of a danger we might be!”, so more akin to them actively saying “we want it, come and give it to us” than to just looking a particular way. Though they aren't the only company to play that game, so there is probably more to it than just that. OpenAI's president giving millions to MAGA Inc and them not getting the same treatment might not be complete coincidences.
cyanydeez 1mo ago
Important to note, OpenAI vs Anthropic are both assholes in different orthogonals. In times like these, i think its important to track whats happening the way we track entropy. That is: theres far >> more ways to be an asshole than well behaved. That doesnt mean we can equate assholes, but the question is which states of entropy are annealable and which are not. I posit Altman is not. Amodei is a open question.
nullbio 1mo ago
Actually, it's the opposite. Anthropic were trying to strongarm the DoD into getting a seat at the table.
watwut 1mo ago
It is more like when a guy walks to the dirty bar, stands in the middle and yells "hahaha I will beat you up all look I have a new baseball bat" and then local drunkard leader stands up and hit him in the face cause he does not like him anyway. Intentionally framing yourself as the local dangerous guy about to beat others is not like wearing cloth.