5 ms·

Maybe this will be simpler for Anthropic to understand if they take their own high-minded philosophical nonsense and ego out of it and consider it the way a neu

Maybe this will be simpler for Anthropic to understand if they take their own high-minded philosophical nonsense and ego out of it and consider it the way a neutral party would. Suppose a company calls themselves The Doomsday Device Company. They make and sell excellent-quality doomsday devices. They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated. The Doomsday Device Company then says they have the world's best doomsday device. (They don't, but they claim they do.) The US Government hates the Doosmday Device Company for various political reasons, but also has a vested interest in there not being a massive proliferation of doomsday devices. The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!" (though it probably isn't) and "Everyone can use it!" (for a lot of money) It is completely logical, then, for the US Government to say: No, everyone cannot use your doomsday device, because doomsday is bad. (While also meaning: Only we should be able to use it, and you shouldn't be able to tell us how.) If you do not want to be in the business of having your doomsday devices shut down by the government, well, it would help if you didn't so loudly and aggressively proclaim how doomy they are. It doesn't matter how trustworthy you claim to be, given that your business is making evil doosmday devices. You still won't be trusted!

threatofrain 4mo ago
It's quite a bit more complicated than that. The popular narrative is not exactly on Anthropic, as the general public is far more aware of OpenAI than Anthropic. The narrative is on AI and whether everything we know about society is going to change. Also, as far as priorities and worries go, for most people cybersecurity is way down the list.
- gerdesj 4mo ago
  "It's quite a bit more complicated than that." Ohhh no it isn't! (Ohh yes it is) etc "as the general public is far more aware of OpenAI than Anthropic" I run LLMs on my own gear with llama.cpp (compiled from source) and I could not tell you anything about either company except they fiddle with AI stuff and that (I don't actually care). I glaze over on news about both organisations in equal measure on mention. I think you'll find that the general public would not be able to name either company without being asked to pronounce their name from it being written down.
  - sillysaurusx 4mo ago
    My eighty year old father brought up OpenAI unprompted a few months ago. At this point it’s hard to find anyone who hasn’t heard of OpenAI.
    - jazzyjackson 4mo ago
      Depends entirely on whether they leave CNBC playing as background noise
      - DANmode 4mo ago
        Yes. Can’t wag the dog without centralized networks; even if they’re now Meta algorithms, they’re still there.
    - gerdesj 4mo ago
      I think my point still stands 8) I know plenty of people who will put the letters A and I together and get rather confused about what on earth is going on but very few of those would mention any co apart from Microsoft, Google, Facebook or errm Twitter. No one has a bloody clue about all this stuff apart from us lot and we have no real idea about it either! Oooh tulips!
      - threatofrain 4mo ago
        Is your point that the name people actually remember is ChatGPT? Okay, let's say the actually popular name that threatens to become the Kleenex of LLMs. ChatGPT.
  - satvikpendem 4mo ago
    I'm not sure what your personal anecdote has to do with general trends, there have always been hermits in history. I also run local models with llama.cpp but I actually stay abreast of the issues in the field as we may similarly be impacted by recent actions.
    - 4mo ago
xscott 4mo ago
Another neutral party might not believe it's really a doomsday device and that what currently looks like exponential growth in capability could be an s-curve that plateaus in a year or two. After that, it will be diminishing returns to invest heavily into a tech that won't get much better. So what are the current leaders in the field supposed to do to stave off competition? They should convince the public that they do have a doomsday device, claim it must be regulated, and then they can profit from their duopoly because it's exceedingly expensive to break into the high end of the market. The government has its own nefarious incentives, not limited to collecting fees and using the unrestricted versions for surveillance or black hat stuff.
- ivraatiems 4mo ago
  I completely agree with you. I think the problem is that Anthropic believes their own BS, and thinks it IS a Doomsday device which only THEY can control. I think that's what's produced this outcome.
  - xscott 4mo ago
    It's tough to know who believes what at that level, because if they are aiming for regulatory capture they need to maintain the illusion.
  - bbippin 4mo ago
    I’m surprised people are reacting so strongly over this. Suppose there is a huge concern—would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks? I used Fable 5 for a fair amount of tasks before they pulled it and I can only imagine what an untapped version of that could do. It’s very capable and equally aggressive in accomplishing its goals.
    - satvikpendem 4mo ago
      Is this not the same as security by obscurity? Open source is more secure because it's more open and thus is able to have flaws found in it more easily. So I'd probably prefer more people to have Fable level models than not.
Barrin92 4mo ago
>It is completely logical, then, for the US Government to say: No Not sure about that one given that the US government just reversed a ban on the exports on the very chips to China that enable said technologies, you don't hear so much about the chip wars any more. I think entrepreneurs largely approached this administration with the attitude that if you're running Doomsday Incorporated they aren't going to say, "no, don't export that", but "hell yeah baby, how do we get a 25% cut on every sale?" because that's quite literally what they did on the hardware front. I mean I have no strong opinion on whether Antrophics statement are true or smart, but the idea that it was regulated because someone in this administration thought it posed incalculable risks is a bit funny, i"m pretty sure they wear that as a logo printed on their t-shirts, it seems to be the sole guiding principle of their foreign policies. Palantir has successfully used doom-mongering as an advertisement strategy for a decade or so
- ivraatiems 4mo ago
  In fairness, I think this might be a case where the pure id represented by this administration happens to align with the correct and logical choice. You could be right, but for this instance, the outcome is the same.
s3p 4mo ago
Shame on a company for sticking to their values, I guess. The dichotomy between Anthropic and OpenAI's treatment honestly couldn't be more obvious. OpenAI has also asked for increased AI regulation, and they've also released GPT 5.5 Cyber which is claimed to have the same vulnerability-finding abilities as Mythos. OpenAI received no such notices like Anthropic. OpenAI also received a government contract, while Anthropic was banned from DoD use. Regardless of your thoughts about Dario or his company, this treatment is obviously not based in any rational principle, and pretending it is would be stupid. It's only a matter of months before the open source models achieve this same capability. What is the US government going to do then? Ban all people in the world from accessing the Chinese models? If you think about these arguments for more than five minutes they really do fall flat.
- matheusmoreira 4mo ago
  Their values are garbage. Paternalism, censorship, suppression. These are people who think they are the enlightened ones while we're all dangerous terrorists. They are in fact pulling up the ladder behind them, just like all the other big techs. It's indeed quite satisfying to watch them be the first ones burned by the heavy hand of government they worshipped so much.
knollimar 4mo ago
>(They don't, but they claim they do.) I hate to shill them, but wasn't mythos/Fable SOTA? Your main point still stands without this aside
- cassianoleal 4mo ago
  What does this question mean? Of course they are state-of-the-art. After all, they are the most recent and most advanced models out of Anthropic. If/when they release a new version of their models, Mythos/Fable will cease to be state-of-the-art, as the new ones become it.
  - nearbuy 4mo ago
    It means Mythos/Fable was the strongest model globally, not just the newest model from a company.
    - knollimar 4mo ago
      Yeah if the analogy is LLMs are doomsday devices it's hard to say mythos wasn't the best. That was my point
andai 4mo ago
Okay, but hasn't OpenAI been doing the same thing for years? They seem to be on slightly better terms though...
- amazingamazing 4mo ago
  Yes, albeit not to the dame extent
- lazide 4mo ago
  OpenAI is happy to sell their device to the gov’t to blow things up with, Anthropic tried to tell the gov’t to pound sand, no blowing things up for you.
  - slopinthebag 4mo ago
    It was more like "you can use our device to blow things up you just need a human to type --dangerously-skip-permissions when they run it"
  - SilverElfin 4mo ago
    To be clear, Anthropic was completely okay with their tech being used in war, including current conflicts. They just wanted a human making some decisions. The military didn’t want a vendor giving them restrictions on how to operate a tool, and since Anthropic’s restrictions could be a problem anywhere they appear among other military vendors as well, the government used the supply chain risk designation to say “this can’t be anywhere in our tools”.
    - ajmurmann 4mo ago
      After a contract had been negotiated with the exact terms that Anthropic pushed to keep in place.
      - SilverElfin 4mo ago
        We don't have the actual contracts publicly. So I bet it has terms that let the government do what it wants ultimately by naming exceptions. That way OpenAI can claim it has some controls in place while the military has the freedom it needs practically.
SecretDreams 4mo ago
How does this look if it's a perpetual motion machine instead of a doomsday device?
satvikpendem 4mo ago
David Sacks corroborates this: https://twitter.com/DavidSacks/status/2065853007619588171 https://twitter.com/DavidSacks/status/2065853007619588171
- munksbeer 4mo ago
  He is a vested interest, so we can pretty much dismiss what he says. Even in that post he makes some logical fallacies.
ajmurmann 4mo ago
Well, regulating doomsday devices is a reasonable thing to want. A reasonable regulation of such devices would call for proper safeguards and safety testing. I think Anthropic would have been fine with that. Instead what happened is a one-off nationalist decree that solves none of the two concerns.
estearum 4mo ago
Yes this is also why if you're building a startup in the nuclear power space (empirically demonstrated Doomsday Device), then you can expect USG to come in and apply arbitrary, opaque, and unexplained rules on you, steal your assets, and destroy your business. And also probably not do that to any of your competitors who are doing the exact same thing as you.
adahn 4mo ago
When I read this comment I don't get "Neutral Party" from it. I'm finding it hard to decide whether to focus on the loaded characterisations or (my best guess at) the underlying substance. To address the point I think you (and the article) are making; there's a difference between advocating that a scoped, due-process-protected power exist and endorsing any given exercise of it. This point is made in Anthropic's original statement, but it's seemingly been missed by everyone taking a position against them. >As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles. [1] But with respect to your specific post: >high-minded philosophical nonsense Taking a sophisticated approach to a problem will generally improve the outcome. I suppose your point is that it can be difficult to deliniate sophisticated & good-faith reasoning from self-serving rationalisation. Much of what I've seen from Anthropic supports the former. Perhaps others could do more to build up the case for the latter. You can look at things like the circumstances that the company was founded in, the lack of political cozying with the current admin, the lack of controversies or disgruntled ex-employees, them taking a stand against the DoD (against their business interests), positions they've taken on various issues including data centres in gulf states. >Suppose a company calls themselves The Doomsday Device Company Anthropic hasn't. What's the explanatory purpose of adding this to the analogy? >They make and sell excellent-quality doomsday devices | They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated. Anthropic invests a reasonable amount into assuring the safety of the products they sell. No-one is claiming they sell doomsday devices. Concerns of doom relate to future iterations of the underlying technology. >The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!" They have what's essentially an automated vulnerability-discovering machine. They've done their best to be open about the very real implications of general availability of a system like this. Using the word "doomy" positions them as childish and unsophisticated for doing this, when it's actually the reasonable and responsible thing to do. 1: https://www.anthropic.com/news/fable-mythos-access https://www.anthropic.com/news/fable-mythos-access

threatofrain 4mo ago

It's quite a bit more complicated than that. The popular narrative is not exactly on Anthropic, as the general public is far more aware of OpenAI than Anthropic. The narrative is on AI and whether everything we know about society is going to change. Also, as far as priorities and worries go, for most people cybersecurity is way down the list.

gerdesj 4mo ago
"It's quite a bit more complicated than that." Ohhh no it isn't! (Ohh yes it is) etc "as the general public is far more aware of OpenAI than Anthropic" I run LLMs on my own gear with llama.cpp (compiled from source) and I could not tell you anything about either company except they fiddle with AI stuff and that (I don't actually care). I glaze over on news about both organisations in equal measure on mention. I think you'll find that the general public would not be able to name either company without being asked to pronounce their name from it being written down.
- sillysaurusx 4mo ago
  My eighty year old father brought up OpenAI unprompted a few months ago. At this point it’s hard to find anyone who hasn’t heard of OpenAI.
  - jazzyjackson 4mo ago
    Depends entirely on whether they leave CNBC playing as background noise
    - DANmode 4mo ago
      Yes. Can’t wag the dog without centralized networks; even if they’re now Meta algorithms, they’re still there.
  - gerdesj 4mo ago
    I think my point still stands 8) I know plenty of people who will put the letters A and I together and get rather confused about what on earth is going on but very few of those would mention any co apart from Microsoft, Google, Facebook or errm Twitter. No one has a bloody clue about all this stuff apart from us lot and we have no real idea about it either! Oooh tulips!
    - threatofrain 4mo ago
      Is your point that the name people actually remember is ChatGPT? Okay, let's say the actually popular name that threatens to become the Kleenex of LLMs. ChatGPT.
- satvikpendem 4mo ago
  I'm not sure what your personal anecdote has to do with general trends, there have always been hermits in history. I also run local models with llama.cpp but I actually stay abreast of the issues in the field as we may similarly be impacted by recent actions.
  - 4mo ago

xscott 4mo ago

Another neutral party might not believe it's really a doomsday device and that what currently looks like exponential growth in capability could be an s-curve that plateaus in a year or two. After that, it will be diminishing returns to invest heavily into a tech that won't get much better. So what are the current leaders in the field supposed to do to stave off competition? They should convince the public that they do have a doomsday device, claim it must be regulated, and then they can profit from their duopoly because it's exceedingly expensive to break into the high end of the market. The government has its own nefarious incentives, not limited to collecting fees and using the unrestricted versions for surveillance or black hat stuff.

ivraatiems 4mo ago
I completely agree with you. I think the problem is that Anthropic believes their own BS, and thinks it IS a Doomsday device which only THEY can control. I think that's what's produced this outcome.
- xscott 4mo ago
  It's tough to know who believes what at that level, because if they are aiming for regulatory capture they need to maintain the illusion.
- bbippin 4mo ago
  I’m surprised people are reacting so strongly over this. Suppose there is a huge concern—would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks? I used Fable 5 for a fair amount of tasks before they pulled it and I can only imagine what an untapped version of that could do. It’s very capable and equally aggressive in accomplishing its goals.
  - satvikpendem 4mo ago
    Is this not the same as security by obscurity? Open source is more secure because it's more open and thus is able to have flaws found in it more easily. So I'd probably prefer more people to have Fable level models than not.

Barrin92 4mo ago

>It is completely logical, then, for the US Government to say: No Not sure about that one given that the US government just reversed a ban on the exports on the very chips to China that enable said technologies, you don't hear so much about the chip wars any more. I think entrepreneurs largely approached this administration with the attitude that if you're running Doomsday Incorporated they aren't going to say, "no, don't export that", but "hell yeah baby, how do we get a 25% cut on every sale?" because that's quite literally what they did on the hardware front. I mean I have no strong opinion on whether Antrophics statement are true or smart, but the idea that it was regulated because someone in this administration thought it posed incalculable risks is a bit funny, i"m pretty sure they wear that as a logo printed on their t-shirts, it seems to be the sole guiding principle of their foreign policies. Palantir has successfully used doom-mongering as an advertisement strategy for a decade or so

ivraatiems 4mo ago
In fairness, I think this might be a case where the pure id represented by this administration happens to align with the correct and logical choice. You could be right, but for this instance, the outcome is the same.

s3p 4mo ago

Shame on a company for sticking to their values, I guess. The dichotomy between Anthropic and OpenAI's treatment honestly couldn't be more obvious. OpenAI has also asked for increased AI regulation, and they've also released GPT 5.5 Cyber which is claimed to have the same vulnerability-finding abilities as Mythos. OpenAI received no such notices like Anthropic. OpenAI also received a government contract, while Anthropic was banned from DoD use. Regardless of your thoughts about Dario or his company, this treatment is obviously not based in any rational principle, and pretending it is would be stupid. It's only a matter of months before the open source models achieve this same capability. What is the US government going to do then? Ban all people in the world from accessing the Chinese models? If you think about these arguments for more than five minutes they really do fall flat.

matheusmoreira 4mo ago
Their values are garbage. Paternalism, censorship, suppression. These are people who think they are the enlightened ones while we're all dangerous terrorists. They are in fact pulling up the ladder behind them, just like all the other big techs. It's indeed quite satisfying to watch them be the first ones burned by the heavy hand of government they worshipped so much.

knollimar 4mo ago

>(They don't, but they claim they do.) I hate to shill them, but wasn't mythos/Fable SOTA? Your main point still stands without this aside

cassianoleal 4mo ago
What does this question mean? Of course they are state-of-the-art. After all, they are the most recent and most advanced models out of Anthropic. If/when they release a new version of their models, Mythos/Fable will cease to be state-of-the-art, as the new ones become it.
- nearbuy 4mo ago
  It means Mythos/Fable was the strongest model globally, not just the newest model from a company.
  - knollimar 4mo ago
    Yeah if the analogy is LLMs are doomsday devices it's hard to say mythos wasn't the best. That was my point

andai 4mo ago

Okay, but hasn't OpenAI been doing the same thing for years? They seem to be on slightly better terms though...

amazingamazing 4mo ago
Yes, albeit not to the dame extent
lazide 4mo ago
OpenAI is happy to sell their device to the gov’t to blow things up with, Anthropic tried to tell the gov’t to pound sand, no blowing things up for you.
- slopinthebag 4mo ago
  It was more like "you can use our device to blow things up you just need a human to type --dangerously-skip-permissions when they run it"
- SilverElfin 4mo ago
  To be clear, Anthropic was completely okay with their tech being used in war, including current conflicts. They just wanted a human making some decisions. The military didn’t want a vendor giving them restrictions on how to operate a tool, and since Anthropic’s restrictions could be a problem anywhere they appear among other military vendors as well, the government used the supply chain risk designation to say “this can’t be anywhere in our tools”.
  - ajmurmann 4mo ago
    After a contract had been negotiated with the exact terms that Anthropic pushed to keep in place.
    - SilverElfin 4mo ago
      We don't have the actual contracts publicly. So I bet it has terms that let the government do what it wants ultimately by naming exceptions. That way OpenAI can claim it has some controls in place while the military has the freedom it needs practically.

SecretDreams 4mo ago

How does this look if it's a perpetual motion machine instead of a doomsday device?

satvikpendem 4mo ago

David Sacks corroborates this: https://twitter.com/DavidSacks/status/2065853007619588171 https://twitter.com/DavidSacks/status/2065853007619588171

munksbeer 4mo ago
He is a vested interest, so we can pretty much dismiss what he says. Even in that post he makes some logical fallacies.

ajmurmann 4mo ago

Well, regulating doomsday devices is a reasonable thing to want. A reasonable regulation of such devices would call for proper safeguards and safety testing. I think Anthropic would have been fine with that. Instead what happened is a one-off nationalist decree that solves none of the two concerns.

estearum 4mo ago

Yes this is also why if you're building a startup in the nuclear power space (empirically demonstrated Doomsday Device), then you can expect USG to come in and apply arbitrary, opaque, and unexplained rules on you, steal your assets, and destroy your business. And also probably not do that to any of your competitors who are doing the exact same thing as you.

adahn 4mo ago

When I read this comment I don't get "Neutral Party" from it. I'm finding it hard to decide whether to focus on the loaded characterisations or (my best guess at) the underlying substance. To address the point I think you (and the article) are making; there's a difference between advocating that a scoped, due-process-protected power exist and endorsing any given exercise of it. This point is made in Anthropic's original statement, but it's seemingly been missed by everyone taking a position against them. >As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles. [1] But with respect to your specific post: >high-minded philosophical nonsense Taking a sophisticated approach to a problem will generally improve the outcome. I suppose your point is that it can be difficult to deliniate sophisticated & good-faith reasoning from self-serving rationalisation. Much of what I've seen from Anthropic supports the former. Perhaps others could do more to build up the case for the latter. You can look at things like the circumstances that the company was founded in, the lack of political cozying with the current admin, the lack of controversies or disgruntled ex-employees, them taking a stand against the DoD (against their business interests), positions they've taken on various issues including data centres in gulf states. >Suppose a company calls themselves The Doomsday Device Company Anthropic hasn't. What's the explanatory purpose of adding this to the analogy? >They make and sell excellent-quality doomsday devices | They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated. Anthropic invests a reasonable amount into assuring the safety of the products they sell. No-one is claiming they sell doomsday devices. Concerns of doom relate to future iterations of the underlying technology. >The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!" They have what's essentially an automated vulnerability-discovering machine. They've done their best to be open about the very real implications of general availability of a system like this. Using the word "doomy" positions them as childish and unsophisticated for doing this, when it's actually the reasonable and responsible thing to do. 1: https://www.anthropic.com/news/fable-mythos-access https://www.anthropic.com/news/fable-mythos-access