20 ms·

I used to work at Anthropic, and I wrote a comment on a thread earlier this week about the RSP update [1]. It's enheartening to see that leaders at Anthropic ar

by lebovic 7mo ago

I used to work at Anthropic, and I wrote a comment on a thread earlier this week about the RSP update [1]. It's enheartening to see that leaders at Anthropic are willing to risk losing their seat at the table to be guided by values. Something I don't think is well understood on HN is how driven by ideals many folks at Anthropic are, even if the company is pragmatic about achieving their goals. I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, and b) they think is a net negative in the long term. (Many others, too, they're just well-known.) That doesn't mean that I always agree with their decisions, and it doesn't mean that Anthropic is a perfect company. Many groups that are driven by ideals have still committed horrible acts. But I do think that most people who are making the important decisions at Anthropic are well-intentioned, driven by values, and are genuinely motivated by trying to make the transition to powerful AI to go well. [1]: https://news.ycombinator.com/item?id=47145963#47149908 https://news.ycombinator.com/item?id=47145963#47149908

big-chungus4 7mo ago
[flagged]
yowayb 7mo ago
I've thought the same about a few of my founders/executives. "You either die the good guy or live long enough to become the bad guy" The "bad guy" actually learns that their former good guy mentality was too simplistic.
- JohnMakin 7mo ago
  I have hit points in this in my career where making a moral stand would be harmful to me (for minor things, nothing as serious as this). It's a very tempting and incentivized decision to make to choose personal gain over ideal. Idealists usually hold strong until they can convince themselves a greater good is served by breaking their ideals. These types that succumb to that reasoning usually ironically ending up doing the most harm.
- Fricken 7mo ago
  Ever since I first bothered to meditate on it, about 15 years ago, I've believed that if AI ever gets anywhere near as good as it's creators want it to be, then it will be coopted by thugs. It didn't feel like a bold prediction to make at the time. It still doesn't.
  - nurbl 7mo ago
    Yes. There will always be people who see opportunity in using it destructively. Best case scenario is that others will use it to counter that. But it is usually easier to destroy than to protect. So we could have a constant AI war going on somewhere in the clouds, occasionally leaking new disasters into the human world.
    - Fricken 7mo ago
      I keep hearing this word "progress". We've been stuck here on earth for 1.5 billion years, we're not progressing, we haven't gone anywhere. We're not going anywhere. There is nowhere better for lightyears in any direction. Don't delude yourself with that narcissistic bunk and don't play with fire.
- deleted 7mo ago
  [deleted]
MichaelZuo 7mo ago
How do you reconcile the fact that many people in Anthropic tried to hide the existence of secret non-disparagement agreements for quite some time? It’s hard to take your comment at face value when there’s documented proof to the contrary. Maybe it could be forgiven as a blunder if revealed in the first few months and within the first handful of employees… but after 2 plus years and many dozens forced to sign that… it’s just not credible to believe it was all entirely positive motivations.
- sowbug 7mo ago
  Saying an entity has values doesn't mean the entity agrees with every single one of your values.
  - MichaelZuo 7mo ago
    The desire to force new employees to sign agreements in total secrecy, without even being able to disclose it exists to prospective employees, seems like a pretty negative “value” under any system of morality, commerce, or human organization that I can think of.
    - ChrisMarshallNY 7mo ago
      Lots of companies do it. Doesn't make it right, but HR has kind of become a pretty evil vocation, these days. I don't believe that they necessarily reflect the values of their corporations. They tend to follow their own muse.
      - zmgsabst 7mo ago
        Okay — but if Anthropic is typical banal evil in that regard, why should we believe they didn’t also compromise in other areas? The exact point is that Anthropic is unexceptional and the same as other corporations.
    - sowbug 7mo ago
      That's a perfectly fine belief to have. I might even agree with you. But you're not really advancing a discussion thread about a company's strong ideals by pointing out some past behavior that you don't like. This is especially true when the behavior you're bringing up is fairly common, if perhaps lamentable, among U.S. corporations. Anthropic can be exceptional in some ways while being ordinary in the rest. (I have no horse in this race. But I remain interested in hearing about a former employee's experience and impressions about the company's ideals, and hope it doesn't get lost in a side discussion about whether NDAs are a good thing.)
bnr-ais 7mo ago
Anthropic had the largest IP settlement ($1.5 billion) for stolen material and Amodei repeatedly predicted mass unemployment within 6 months due to AI. Without being bothered about it at all. It is a horrible and ruthless company and hearing a presumably rich ex-employee painting a rosy picture does not change anything.
- Davidzheng 7mo ago
  Neither of these things are useful signals. Other labs surely trained on similar material (presumably not even buying hard copies). Also how "bothered" someone is about their predictions is a bad indicator -- the prediction, taken at face value, is supposed to be trying to ask people to prepare for what he cannot stop if he wanted to. None of this means I am a huge fan of Dario - I think he has over-idealization of the implementation of democratic ideals in western countries and is unhealthily obsessed with US "winning" over China based on this. But I don't like the reasons you listed.
- lebovic 7mo ago
  It's enheartening to see someone make a decision in this context that's driven by values rather than revenue, regardless of whether I agree. I dissented while I was there, had millions in equity on the line, and left without it.
  - jonny_eh 7mo ago
    Why? Can you provide details?
  - kmaitreys 7mo ago
    Doesn't that prove that statements given my CEOs of these companies are just hot air? What is enheartening about hearing a liar who makes provocative statements all the time, make another one?
  - SecretDreams 7mo ago
    > I dissented while I was there, had millions in equity on the line, and left without it. Is this a reflection of your morality, or that you already had sufficient funds that you could pass on the extra money to maintain a level of morality you're happy with? Not everyone has the luxury to do the latter. And it's in those situations that our true morality, as measured against our basic needs, comes out.
calvinmorrison 7mo ago
mark my words, they will burn at some point. The government can nationalize it at any moment if they desire.
- Davidzheng 7mo ago
  Then maybe Dario will realize that the moral superiority that he bases his advocacy against Chinese open models is naive at best.
  - jimmydoe 7mo ago
    his against Chinese models is smoking screen for their resistance to DOW, they are not even pretending
  - jacquesm 7mo ago
    Better naive than malicious.
    - moozooh 7mo ago
      You're saying that as if these two things are mutually exclusive.
    - mrguyorama 7mo ago
      At a certain level, ignorance IS malicious. If you have more money than god, you no longer get to play the "I didn't know" game. You have the resources. If you don't know, you made a choice to not know.
      - jacquesm 7mo ago
        The first one is definitely one we agree on and the second was one that I had not clued into so thank you.
  - viking123 7mo ago
    Every day I hope the Chinese models get "good enough" to drop these corporate ones. I think we are heading towards it.
neom 7mo ago
I've had so much abuse thrown at me on here for saying this very thing over the last few years. I used to be friends with Jack back in the day, before this AI stuff even all kicked off, once you know who people really are inside, it's easy to know how they will act when the going gets rough. I'm glad they are doing the right thing, but I'm not at all surprised, nor should anyone be. Personally I believe they would go to jail/shut down/whatever before they do something objectively wrong.
- monster_truck 7mo ago
  [flagged]
  - 000ooo000 7mo ago
    [flagged]
- ajyey 7mo ago
  This is insanely naive
  - parasubvert 7mo ago
    Cynicism isn't always correct.
- taurath 7mo ago
  > it's easy to know how they will act when the going gets rough Even if you went to burning man and your souls bonded, you only know a person at a particular point in time - people's traits flanderize, they change, they emphasize different values, they develop different incentives or commitments. I've watched very morally certain people fall to mania or deep cynicism over the last 10 years as the pillars of society show their cracks. That said, it is heartening to know that some would predict anyone in Silicon Valley would still take a moral stance. But it would do better if not the same day he fires 4000 people to do the "scary big cut" for a shift he sees happening. I guess we're back to Thatcherisms, where "There Is No Other Option" to justify our conservatism.
dakolli 7mo ago
Anthropic is by far the most evil company in tech, I don't care. Its worst than Palantir in my book. You won't catch my kids touching this slave making, labor killing brain frying tech.
whatever1 7mo ago
Let us think how OpenAI responded to this.
yunnpp 7mo ago
It's good to be driven by ideals, but: https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_what_it_does https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha... I think avg(HN) is mostly skeptical about the output, not that the input is corrupt or ill-meaning in this case. Although with other companies, one can't even take their claims seriously. And in any case, this is difficult territory to navigate. I would not want to be in your spot.
- eternauta3k 7mo ago
  Come On, Obviously The Purpose Of A System Is Not What It Does https://www.astralcodexten.com/p/come-on-obviously-the-purpose-of https://www.astralcodexten.com/p/come-on-obviously-the-purpo...
  - Planktonne 7mo ago
    I don't think that article makes a strong case; it deliberately phrases examples in the most ridiculous ways and pretends that this is a damning criticism of the phrase itself; it's 'you're telling me a shrimp fried this rice' but with a pretence of rationality.
    - sebzim4500 7mo ago
      I think it makes a pretty compelling case that most invocations of the statement are either blindingly obvious or probably false. Can you give a counterexample?
      - Planktonne 7mo ago
        > most invocations of the statement are either blindingly obvious or probably false So straightaway, you've walked significantly back from the claim in the headline; now half of the time it's 'blindingly obvious' that the statement is correct. That already feels like a strong counterexample to me, and it's the article's own first point. Secondly, look at this one specifically: > The purpose of the Ukrainian military is to get stuck in a years-long stalemate with Russia. Firstly, this isn't obviously false. It's an unfair framing, but I think the Ukrainian military would agree that forcing a stalemate when attacked by a hostile power is absolutely part of their purpose. Secondly, it is an unfair framing that deliberately ignores that all systems are contextual. A car's purpose is transport, but that doesn't mean it can phase through any obstacle. The article makes an entirely specious argument, almost an archetypal example of a strawman. It can't sustain its own points over a few hundred words without steadily retreating, and that is far more pointless than the maxim it criticises. I'm reminded of an XKCD comic [1] about smug miscommunication. Of course any principle is ridiculous when you pretend not to understand it. [1] https://xkcd.com/169/ https://xkcd.com/169/
snickerbockers 7mo ago
>I's enheartening to see that leaders at Anthropic are willing to risk losing their seat at the table to be guided by values. I'm concerned that the context of the OP implies that they're making this declaration after they've already sold products. It specifically mentions already having products in classified networks. This is the sort of thing that they should have made clear before that happened. It's admirable (no pun intended) to have moral compunctions about how the military uses their products but unless it was already part of their agreement (which i very much doubt) they are not entitled them to countermand the military's chain of command by designing a product to not function in certain arbitrarily-designated circumstances.
- jsnell 7mo ago
  Where are you getting that from? The article is crystal clear that these uses are not permitted by the current or any past contract, and the DoW wants to remove those exceptions. > Two such use cases have never been included in our contracts with the Department of War, and we believe they should not be included now It also links to DoW's official memo from January 9th that confirms that DoW is changing their contract language going forwards to remove restrictions. A pretty clear indication that the current language has some.
  - snickerbockers 7mo ago
    I think it largely hinges on what they mean by "included"; does that mean it was specifically excluded by the terms of the contract or does it mean that it's not expressly permitted? I doubt the DoD is used to defense contractors thinking they have the right to dictate policy regarding the use of their products, and it's equally possible that anthropic isn't used to customers demanding full control over products (as evidenced by how many chatbots will arbitrarily refuse to engage with certain requests, especially erotic or politically-incorrect subject-matters). Sometimes both parties have valid cases when there's a contract disagreement. >A pretty clear indication that the current language has some. Or alternatively that there is some disagreement between the DoD and Anthropic as to how the contract is to be interpreted and that the DoD is removing the ambiguity in future contracts.
jcgrillo 7mo ago
There's a simpler explanation than "billionaires with hearts of gold" here. If: (1) this is a wildly unpopular and optically bad deal (2) it's a high data rate deal--lots of tokens means bad things for Anthropic. Users which use their product heavily are costing more than they pay. (3) it's a deal which has elements that aren't technically feasible, like LLM powered autonomous killer robots... then it makes a whole lot of sense for Anthropic to wiggle out of it. Doing it like this they can look cuddly, so long as the Pentagon walks away and doesn't hit them back too hard.
- robwwilliams 7mo ago
  All excellent points to add to the motivation to hold the line just where it has been.
- jcgrillo 7mo ago
  guess it didn't work, whiskey pete did the thing: https://xcancel.com/SecWar/status/2027507717469049070 https://xcancel.com/SecWar/status/2027507717469049070
txrx0000 7mo ago
> I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, and b) they think is a net negative in the long term. (Many others, too, they're just well-known.) I very much doubt it judging by their actions, but let's assume that's cognitive dissonance and engage for a minute. What are those values that you're defending? Which one of the following scenarios do you think results in higher X-risk, misuse risk, (...) risk? - 10 AIs running on 10 machines, each with 10 million GPUs OR - 10 million AIs running on 10 million machines, each with 10 GPUs All of the serious risk scenarios brought up in AI safety discussions can be ameliorated by doing all of the research in the open. Make your orgs 100% transparent. Open-source absolutely everything. Papers, code, weights, financial records. Start a movement to make this the worldwide social norm, and any org that doesn't cooperate is immediately boycotted then shut down. And stop the datacenter build-up race. There are no meaningful AI risks in such a world, yet very few are working towards this. So what are your values, really? Have you examined your own motivations beneath the surface?
- ChadNauseam 7mo ago
  > - 10 AIs running on 10 machines, each with 10 million GPUs > > OR > > - 10 million AIs running on 10 million machines, each with 10 GPUs If we dramatically reduced the number of GPUs per AI instance, that would be great. But I think the difference in real life is not as extreme as you're making it. In your telling, the gpus-per-ai is reduced by one million. I'm not sure that (or anything even close to it) is within the realm of possibility for anthropic. The only reason anyone cares about them at all is because they have a frontier AI system. If they stopped, the AI frontier would be a bit farther back, maybe delayed by a few years, but Google and OpenAI would certainly not slow down 1000x, 100x or probably even 10x.
- SecretDreams 7mo ago
  How do you figure open sourcing everything eliminates risk? This makes visibility better for honest actors. But if a nefarious actor forks something privately and has resources, you can end up back in hell. I don't think we can bank on all of humanity acting in humanity's best interests right now.
relaxing 7mo ago
[flagged]
- nickysielicki 7mo ago
  Pagerank is not Claude.
  - relaxing 7mo ago
    Google is not Pagerank?
nmfisher 7mo ago
As a complete outsider, I genuinely believe that Dario et al are well-intentioned. But I also believe they are a terrible combination of arrogant and naive - loudly beating the drum that they created an unstoppable superintelligence that could destroy the world, and thinking that they are the only ones who can control it. I mean if you sign a contract with the Department of War, what on Earth did you think was going to happen?
- sebzim4500 7mo ago
  I don't think the US has ever done/threatened anything like this to a US company so it's not surprising that Anthropic were caught off guard.
- themacguffinman 7mo ago
  Not this, because this is completely unprecedented? In fact, the Pentagon already signed an Anthropic contract with safe terms 6 months ago, that initial negotiation was when Anthropic would have made a decision to part ways. It was totally absurd for the govt to turn around and threaten to change the deal, just a ridiculous and unprecedented level of incompetence.
  - baq 7mo ago
    If they made a completely private nuclear reactor and ended up with a pile of weapons grade plutonium, what do you think the department of war would do? It was completely obvious it would happen, as it will be not surprising when laws are passed and all involved will have choose between quit or quit and go to jail. There are western countries in which you’d just end up in a ditch, dead, so they should think themselves lucky for doing the ai superintelligence thing in the US.
    - ben_w 7mo ago
      The US government clearly doesn't take seriously the claim that AI is more dangerous than (or even as dangerous as) nukes, because if they did they wouldn't allow anyone except the military to develop or use them, they wouldn't allow their export or for them to be made available for use by foreigners like me, they wouldn't allow their own civilians to use them, they would probably be having a repeat of the cases in the cold war where they tried to argue certain inventions were "born secret" and could not be published even if they were developed by people who were not sworn to secrecy.
SecretDreams 7mo ago
> Many groups that are driven by ideals have still committed horrible acts. Sometimes, it's even a very odd prerequisite.
gaigalas 7mo ago
I'm suspicious of public displays of enheartening behavior.
dpweb 7mo ago
I wouldn't underestimate this as a good business decision either. When the mass surveillance scandal, or first time a building with 100 innocent people get destroyed by autonomous AI, the company that built is gonna get blamed.
jwlarocque 7mo ago
Oh hey Noah Glad to hear you say some moral convictions are held at one of the big labs (even if, as you say, this doesn't guarantee good outcomes).
- deleted 7mo ago
  [deleted]
roseinteeth 7mo ago
The road to hell is paved by good intentions and all that
protocolture 7mo ago
>I's enheartening to see that leaders at Anthropic are willing to risk losing their seat at the table to be guided by values. Their "Values": >We have never raised objections to particular military operations nor attempted to limit use of our technology in an ad hoc manner. Read: They are cool with whatever. >We support the use of AI for lawful foreign intelligence and counterintelligence missions. Read: We support spying on partner nations, who will in turn spy on us using these tools also, providing the same data to the same people with extra steps. >Partially autonomous weapons, like those used today in Ukraine, are vital to the defense of democracy. Even fully autonomous weapons (those that take humans out of the loop entirely and automate selecting and engaging targets) may prove critical for our national defense. But today, frontier AI systems are simply not reliable enough to power fully autonomous weapons. Read: We are cool fully autonomous weapons in the future. It will be fine if the success rate goes above an arbitrary threshold. Its not the targeting of foreign people that we are against, its the possibility of costly mistakes that put our reputation at risk. How many people die standing next to the correct target is not our concern. Its a nothingburger. These guys just want to keep their own hands slightly clean. There's not an ounce of moral fibre in here. Its fine for AI to kill people as long as those people are the designated enemies of the dementia ridden US empire.
- HDThoreaun 7mo ago
  Their values are about AI safety. Geopolitically they could care less. You might think its a bad take but at least they are consistent. AI safety people largely think that stuff like autonomous weapons are inevitable so they focus on trying to align them with humanity.
  - protocolture 7mo ago
    Consistency isn't a virtue. A guy who murders people at a consistent rate isn't better than a guy who murders people only on weekends. >AI safety people largely think that stuff like autonomous weapons are inevitable so they focus on trying to align them with humanity. Humanity includes the future victim of AI weapons.
imjonse 7mo ago
> I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, I am sure you think they are better than the average startup executive, but such hyperbole puts the objectivity of your whole judgement under question. They pragmatically changed their views of safety just recently, so those values for which they would burn at the stake are very fluid.
- versteegen 7mo ago
  > They pragmatically changed their views of safety just recently, so those values for which they would burn at the stake are very fluid. Yes it was a pragmatic change, no it was not a change in their values. The commentary here on HN about Anthropic's RSP change was completely off the mark. They "think these changes are the right thing for reducing AI risk, both from Anthropic and from other companies if they make similar changes", as stated in this detailed discussion by Holden Karnofsky, who takes "significant responsibility for this change": https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsible-scaling-policy-v3 https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl... > I strongly think today’s environment does not fit the “prisoner’s dilemma” model. In today’s environment, I think there are companies not terribly far behind the frontier that would see any unilateral pause or slowdown as an opportunity rather than a warning. > What I didn’t expect was that RSPs (at least in Anthropic’s case) would come to be seen as hard unilateral commitments (“escape clauses” notwithstanding) that would be very difficult to iterate on.
  - MichaelDickens 7mo ago
    > Yes it was a pragmatic change, no it was not a change in their values. The commentary here on HN about Anthropic's RSP change was completely off the mark. They "think these changes are the right thing for reducing AI risk, both from Anthropic and from other companies if they make similar changes", as stated in this detailed discussion by Holden Karnofsky, who takes "significant responsibility for this change": Can you imagine a world where Anthropic says "we are changing our RSP; we think this increases AI risk, but we want to make more money"? The fact that they claim the new RSP reduces risk gives us approximately zero evidence that the new RSP reduces risk.
vasco 7mo ago
> I's enheartening to see that leaders at Anthropic are willing to risk losing their seat at the table to be guided by values. They are the deepest in bed with the department of war, what the fuck are you on about? They sit with Trump, they actively make software to kill people. What a weird definition of "enheartening" you have.
bambax 7mo ago
This last development is much to the honor of Anthropic and Amodei and confirms what you're saying. What I don't get though is, why did the so-called "Department of War" target Anthropic specifically? What about the others, esp. OpenAI? Have they already agreed to cooperate? or already refused? Why aren't they part of this?
- D_Alex 7mo ago
  I'm a bit underwhelmed tbh. Here is Anthropic's motto: "At Anthropic, we build AI to serve humanity’s long-term well-being." Why does Anthropic even deal with the Department of @#$%ing WAR? And what does Amodei mean by "defeat" in his first paragraph?
  - jazzyjackson 7mo ago
    DoD and American exceptionalists also believe American foreign policy is in service of humanity’s long term well being
    - viking123 7mo ago
      I think the last few months have shown pretty clearly in whose service this policy is. If China went to attack Taiwan, west has no moral high ground left.
    - temp8830 7mo ago
      It is all for the benefit of man. We even get to see the man himself daily on television.
    - mapt 7mo ago
      Yeah, I don't think so any more. The sort of lofty Cold War rhetoric about leading the world, if it was ever legitimately believed by the people spouting it, is gone. A very different attitude has taken hold, which puts a zero sum ethnonationalism at the core.
    - Balinares 7mo ago
      One of the hallmarks of fascist thinking is the dehumanizing of opponents and minorities, so within their own messed up framework, they might even mean it.
Madmallard 7mo ago
Weird take when the purpose of the creation is to steal the work of everyone and automate the creation of that work. It's some serious self-deluding to think there's any kind of noble ideal remotely related to this process.
amunozo 7mo ago
I just see here is nationalism. How can they claim to be in favour of humanity if they're in favour of spying foreign partners, developing weapons, and everything that serves the sacred nation of the United States of America? How fast do Americans dehumanize nations with the excuse of authoritarianism (as if Trump is not authoritarian) and national defence (more like attack). It's amazing that after these obvious jingoist messages, they still believe they are "effective altruists" (a idiotic ideology anyway).
- Aeolun 7mo ago
  It’s not like other countries do not do this. They’re just not so prone to virtue signaling as in the US.
  - gylterud 7mo ago
    Countries do not do, things people do. Dehumanising “the others” is a human trait, and a very destructive one. Just like violence and greed. People have different susceptibility for these, but we should all work to counter them and it is in its place to point it out when observed.
  - amunozo 7mo ago
    I've never seen any other democracy use so extensively the kind of duality between the good guys and bad guys, as Americans like to say. There is a total lack of nuance and a very widespread message about how the US is special and best than anything else in the world, so everything is justified to assure its primacy. It's the kind of thing you hear from totalitarian and brainwashed countries. I know this is not everybody in the US, and I say this as a foreign person that observes things from outside. I agree with the two statements you made, I just think they could be incomplete and that the countries that behave most similarly to the US are not democracies.
  - moozooh 7mo ago
    This argument is in poor faith. First of all, a contradiction between your own stated values and your own actions cannot be excused by the status quo; it's on you to resolve it. Second, that's a very bold claim that is broad and cynical enough to make it easy to use it as an excuse for anything heinous.
lm28469 7mo ago
Idk man, from the outside anthropic looks a lot like openai with a cute redisgn and Amodei like Altman with a slightly more human face mask, the same media manipulation, the same vague baseless affirmations about "something big is coming and we can't even describe it but trust us we need more money"
- rhubarbtree 7mo ago
  There should be a name for this, “cynic cope: when someone actually takes a principled view the cynic - who has a completely negative view of the world - is proven to be wrong, can’t accept it, and tries to somehow discount it.
  - marxisttemp 7mo ago
    Corporations do not and cannot have principles, they only have the profit motive
    - parasubvert 7mo ago
      This is false. People can have principles, profit motive is not something a corporation has, it's something people have. Corporations do things all the time that are based on everything from principles, to the personal whim of executives, to exercise in ego, to community benefiting actions, or to screw customers for extra profit. It is entirely dependent on the specific people in management roles. Corporations need profit to survive because the cost of tomorrow is a surplus of today.
      - anon_e-moose 7mo ago
        A corporation is a bunch of people cooperating to achieve a common goal. There is a very important factor that heavily influences (perhaps even controls?) how people act to achieve that goal, and sometimes even twists or adds goals. Is that corporation publicly quoted in the stock market or is it private? Look at how steam behaves, it's private and more ideological VS how many other publicly quoted companies, whose CEO often sacrifices his own corporation's long term survival for the benefit of short-term profiteering and some hedge fund manager's bonus. Both need profit to survive, but the publicly quoted company is much more extreme. When people say corporations only look to profit, what they really mean is that publicly quoted corporations will do everything possible to maximise short term profit at any cost. Is there a CEO caring for long term? Either he will be convinced to change or kicked out. It's almost impossible for someone to resist these influences in publicly quoted companies. It's just how Wall Street works and if that doesn't change neither will corporations. The people running the world of finance and their culture are what causes enshittification and pushing a zero-sum game to extremes.
Balinares 7mo ago
I getcha and I believe you're sincere, but on the other hand, God save us from well-intentioned capitalists driven by values.
bertylicious 7mo ago
"They're driven by values" is meaningless praise unless you qualify what these values are. The Nazis had values too, you know. They were even willing to die for them. One of the core values of the Catholic church is probably compassion. Except for the victims of sexual abuse perpetrated by their clergy. So what core values led "Dario, Jared, and Sam" to work with a government that just tried to rename the DoD to "department of war" and is acting aggressively imperialist in a way like the US hasn't in a long time. And who exactly are these "autocratic adversaries" they are mentioning? Does this list include the autocrats the US government is working together with?
- marxisttemp 7mo ago
  Careful speaking truth to power on this site, remember that YC is deeply enmeshed with Garry Tan, Peter Thiel, and of course Paul Graham who as of late has made a habit of posting right wing slop on his Twitter
- lebovic 7mo ago
  Yeah, values on their own don't lead to positive outcomes. I agree that many groups that are driven by ideals have still committed horrible acts. I do think that they're acting with positive intent, though, and are motivated by trying to make the transition to powerful AI go well. Many folks on HN seem to assume the primary motivation is purely chasing more money, which certainly isn't the case for for many – but not all – people at Anthropic. That doesn't guarantee a good outcome, and there's still a hard road ahead.
- DeepSeaTortoise 7mo ago
  > Except for the victims of sexual abuse perpetrated by their clergy. I honestly wonder how much of this is made up. Given the size of whole organization and it holding onto its weird priciples regarding the personal relationships of its members (introduced in the far past to limit the secular power of its clergy), there certainly will be SOME cases. But in the one case a frater, who I knew, got convicted, he definitely didn't do it. He was accused by several independent former students and even some of the staff backed the students claims with first hand accounts of him having been alone with some of the students at the time. This supposedly happened on a trip with tight schedules, so all accounts and stated times were quite specific, even in the pre-smartphone era. The only problem: He wasn't with the group at that time at all. I screwed up embarrassingly (and the staff, too, leaving a young student stranded in the middle of nowhere) and he thought he could slip out, come pick me up and nobody (but maybe me with him) would get in trouble over it. Turned out he forgot refueling, both of us stayed at a pastor's guest house and he called the group telling them, that they should go ahead without us and that we would drive to the event directly on our own. The supposed abuse was claimed to have happened at another short stay of the group where they spent a day visiting some mine before joining with us again. Almost 3 decades later he got railroaded in court, me learning about it in the news.
comandillos 7mo ago
To me this is just another marketing stunt where the company wants to build a public image so their customers trust them (see Apple), but then as always who knows what will happen behind the scenes. Just see when most major US companies had backdoors on their systems providing all data to the NSA, i.e. PRISM.
- lonelyasacloud 7mo ago
  >just another marketing stunt What evidence on _Amodei_ and his actions leads to that conclusion?
  - moozooh 7mo ago
    Anthropic's policy is full of contradictions. They are against mass-surveillance of Americans but they happily deal with Palantir. They talk about humanity as a whole but only care about what American companies use their models to do to Americans; everybody else is fair game for AI-driven surveillance. They warn of the dangers of AI-driven warfare by demonstrating a mass-scale cyberattack perpetrated using their model, Claude, as the main operation engine and immediately release a new, more powerful version of Claude. You just need to use Claude to protect yourself from Claude, see. When you really start digging into it, it appears schizophrenic at first, and then you remember market incentives are a thing and everything falls into place.
    - ExoticPearTree 7mo ago
      You know, once the lawyers get involved, there are no contradictions because they define every term and then it makes all the sense in the world. If Humaity=America, then obviously they don’t care about the rest of the people as a very very silly example.
      - moozooh 7mo ago
        You call it silly, I call it an accurate reading!
    - 7mo ago
drawfloat 7mo ago
"Mass surveillance of anywhere else in the world but America" is not the great idealistic position you are making it out to be.
PeterStuer 7mo ago
As an insider, do you think this is Altman playing his infamous machiavellian skills on the DoD?
miroljub 7mo ago
While many praise them for sticking to their values, it's also worth mentioning that their values are not everyone's values. Of all major LLMs, Claude is perhaps the most closed and, subjectively, the most biased. Instead of striving for neutrality, Anthropic leadership's main concern is to push their values down people's throats and to ensure consistent bias in all their models. I have a feeling they see themselves more as evangelists than scientists. That makes their models unusable for me as general AI tools and only useful for coding. If their biases match yours, good for you, but I'm glad we have many open Chinese models taking ground, which in the long run makes humanity more resistant to propaganda.
- AlecSchueler 7mo ago
  > Of all major LLMs, Claude is perhaps the most closed and, subjectively, the most biased. Instead of striving for neutrality, Anthropic leadership's main concern is to push their values down people's throats It's this satire? Let us know when Claude starts calling itself MechaHitler or trying to shoehorn nonsense about white genocide into every conversation.
- soco 7mo ago
  I might be misreading your comment, which I understood like "Chinese make humanity more resistant to propaganda". It just doesn't add up, can you please explain?
  - miroljub 7mo ago
    Chinese models give you more choice (good), competition (good) and less bias (good). I did not say anything about the Chinese government, which is sadly becoming a role model for many (all?) Western governments.
whstl 7mo ago
> Something I don't think is well understood on HN is how driven by ideals many folks at Anthropic are After 20 years of everyone in this industry saying "we want to make the world a better place" and doing the opposite, the problem here is not really related to people's "understanding". And before the default answer kicks in: this is not cynicism. Plenty of folks here on HN and elsewhere legitimately believe that it's possible to do good with tech. But a billion dollar behemoth with great PR isn't that.
- amunozo 7mo ago
  I don't even think both things are contradictory. People that put too much value in their ideals tend to oversee the consequences of such ideals in real life and do wrong without deviating an inch from their ideals.
  - plufz 7mo ago
    But is that really the problem in big tech today? To me it looks like sooner or later they cave from their ideals (or leadership changes) and that the reason every time is that they want to make even more money.
    - hsuduebc2 7mo ago
      I believe that this is classical behaviour of every share holder driven business. You can build on ideals from start, but once you acquire some position, money making is on the menu. Eg. deliberately worsening user experience for better revenue. Possiblity to turn on heated seats in car you own for a small monthly fee is absurd yet very real. I'm looking forward to enshittification of current AI tools.
      - Ajedi32 7mo ago
        Yeah it's not that the people involved have no ideals, it's that the company structure as a whole doesn't, and over time that structure will eventually outlive, corrupt, and/or overpower the ideals of the founders or other principled individuals at the company.
yamal4321 7mo ago
seeing the comment: "people who are making the important decisions at Anthropic are well-intentioned, driven by values" which is left under the article: "Statement from Dario Amodei on our discussions with the Department of War" :)
Aldipower 7mo ago
3 words for you: This is naive.
tpoacher 7mo ago
> But I do think that most people who are making the important decisions at Anthropic are well-intentioned, driven by values, and are genuinely motivated by trying to make the transition to powerful AI to go well. in which case, these people will necessarily have to be the first to go, I suppose, once the board decides enough is enough. Refusing to do things that go against "company values" even if they risk damaging the company, isn't exceptional circumstances; it's the very definition of "company values". But if those values aren't "company" values but "personal" values, then you can be sure there's always going to be someone higher up who isn't going to be very appreciative once "personal" values start risking "company" damage.
- xvector 7mo ago
  Shareholders do not control Anthropic's board, it is not structured like a typical corporation.
  - tpoacher 7mo ago
    For now.
yayr 7mo ago
There are well intentioned people everywhere, also at Google or OpenAI... https://notdivided.org https://notdivided.org But the final decisions made usually depend on the incentive structures and mental models of their leaders. Those can be quite different...
fergie 7mo ago
> I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, and b) they think is a net negative in the long term. Sure, but what happens when the suits eventually take over? (see Google)
ozgung 7mo ago
The problem with companies, you see, is that they are a separate entity than their founders, shareholders or current leadership. A Company has no soul or unchangeable intentions. Claude’s SOUL.md is just an IP that can be edited at any time.
Yizahi 7mo ago
Exactly which values they are "going to burn at a stake for"? Making as many people homeless as they can in the shortest possible time? Befuddling governments and VCs into creating an insane industry-wide debt which would either lead to a "success" in replacing jobs or an industry-wide crisis? Or maybe a value of stealing intellectual property of every human on the planet under the guise of "fair use" and then deliberately selling the derivative product? Or the value of voluntarily working with "national security customers" when it suits them financially and crying foul when leopards bite their faces? Or the value of ironically calling a human replacement machine "anthropic" as in "for humanity"? Yeah, I totally see Anthropic execs defending them to their last dollar in the wallet. Par for the course for megacorps. It's just I personally don't value those values at all.
_s_a_m_ 7mo ago
We will see..
retinaros 7mo ago
lol. no one with common sense ever bought this story. you might have and your turning point might be this deal but for many the turning point was stealing data for training, advocating against china and calling them an adverse nation, pushing to ban opensource alternatives deeming them as "dangerous", buying tech bros with matcha popup in SF, shady RLHF and bias and millions others
heresie-dabord 7mo ago
> how driven by ideals many folks at $Corporatron are Well let's see... it says in the post: * worked proactively to deploy our models to the Department of War and the intelligence community. * the first frontier AI company to deploy our models in the US government’s classified networks, * the first to deploy them at the National Laboratories, and * the first to provide custom models for national security customers. * extensively deployed across the Department of War and other national security agencies * offered to work directly with the Department of War on R&D to improve the reliability of these systems * accelerating the adoption and use of our models within our armed forces to date. * never raised objections to particular military operations nor attempted to limit use of our technology in an ad hoc manner.
- wrsh07 7mo ago
  They didn't claim to have pacifist ideals In fact, they claim to be pro America and pro democracy and have repeatedly expressed concerns about autocratically governed countries. Just because you disagree with their ideals doesn't mean they're not holding to theirs
  - leshow 7mo ago
    They sound exactly like George Bush and every other American leader who's claimed high minded ideals while they engage in interventions in direct contradiction to those ideals around the world
    - wrsh07 7mo ago
      To be clear, I don't think anthropic is itself intervening. The concerns they've raised about authoritarianism is "AI enabling authoritarians." When they push back on the US government wanting to use Claude to (legally) surveil US citizens, that still feels consistent to me as a concern about authoritarianism. I think it's reasonable to hear high minded ideals and become skeptical, but in this case I'm surprised that people are trying to accuse them of hypocrisy
- mikkupikku 7mo ago
keybored 7mo ago
As a complete bystander I put so incredibly little weight to what friends and former employees think about the persons and figureheads behind tech companies that aim to change the world. Why would I care. All people with at least some positive or negative notoriety have friend and associates that will, hand to their heart, promise that they mean well. They have the best intentions. And any deviations from their stated ideals are just careful pragmatic concerns. Road to Hell and all that.
i_love_retros 7mo ago
[flagged]
AndyMcConachie 7mo ago
> Something I don't think is well understood on HN is how driven by ideals many folks at Anthropic are I don't think you understand how capitalism and corporations work, friend. Even if Anthropic is a public benefit corporation it still exists in the USA and will be placed under extensive pressure to generate a profit and grow. Corporations are designed to be amoral and history has shown that regardless of their specific legal formulation they all eventually revert to amoral growth driven behavior. This is structural and has nothing to do with individuals.
junofan 7mo ago
“AI chips are like nuclear weapons” (paraphrasing [1]) and “I should be in charge of it” (again paraphrasing) is just not a serious position regardless of intentions. [1]: https://www.axios.com/2026/01/20/anthropic-ceo-admodei-nvidia-chips-china-trump https://www.axios.com/2026/01/20/anthropic-ceo-admodei-nvidi...
pmarreck 7mo ago
The same guy who thinks AGI will eliminate "centaur coders" (I respectfully disagree) and possibly all white-collar work, is now concerned about the misuse of the same AI to make war? That's cute. Literally just giving business away. This is not a cynical take, this is a realistic one. This would be like agreeing to have your phone regularly checked by your spouse and citing the need for fidelity on principle. No one would like that, no smart person would agree to that, and anyone with any sense or self-respect would find another spouse to "work with". They will simply go to another vendor... Anthropic is not THAT far ahead. Also, the US’s enemies are not similarly restricted. /eyeroll Palmer Luckey ("peace through superior firepower") is the smart one, here. Dario Amodei ("peace through unilateral agreement with no one, to restrict oneself by assuming guilt of business partners until innocence is proven") is not. Anthropic could have just done what real spouses do. Random spot checks in secret, or just noticing things. >..< And if a betrayal signal is discovered, simply charge more and give less, citing suspicious activity… … since it all goes through their servers. Honestly, I'm glad that they're principled. The problem is that 1) most people in general are, so to assume the opposite is off-putting; 2) some people will always not be. And the latter will always cause you trouble if you don't assert dominance as the "good guy", frankly.
windexh8er 7mo ago
> I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, and b) they think is a net negative in the long term. (Many others, too, they're just well-known.) This is a nice strawman, but it means nothing in the long run. People's values change and they often change fast when their riches are at stake. I have zero trust in anyone mentioned here because their "values" are currently at odds with our planet (in numerous facets). If their mission was to build sustainable and ethical AI I'd likely have a different perspective. However, Anthropic, just like all their other Frontier friends, are accelerating the burn of our planet exponentially faster and there's no value proposition AI doesn't currently solve for outside of some time savings, in general. Again, it's useful, but it's also not revolutionary. And it's being propped up incongruently with its value to society and its shareholders. Not that I really care about the latter...
cue_the_strings 7mo ago
Don't attribute to ideals what is simple self-preservation. No sane person wants to become a legitimate military target. They want to sleep in their own beds, at home, without risking their families lives. Just like the rest of us.
synergy20 7mo ago
just curious, what about other regions and countries who have no such restrictions to develop their weapons? there is no world treaty on this yet, even there is one, not everyone will follow behind the doors.
chrisjj 7mo ago
> driven by values So what? Every business is driven by values.
JumpCrisscross 7mo ago
> leaders at Anthropic are willing to risk losing their seat at the table Hot take: Dario isn’t risking that much. Hegseth being Hegseth, he overplayed his hand. Dario is calling his bluff. Contract terminations are temporary. Possibly only until November. Probably only until 2028 unless the political tide shifts. Meanwhile, invoking the Defense Production Act to seize Anthropic’s IP basically triggers MAD across American AI companies—and by extension, the American capital markets and economy—which is why Altman is trying to defuse this clusterfuck. If it happens it will be undone quickly, and given this dispute is public it’s unlikely to happen at all.
- cgh 7mo ago
  Not a hot take at all. Probably the best take in this thread.
didip 7mo ago
I like the enthusiasm, but remember that Google used to be: “Don’t be Evil”
learingsci 7mo ago
I remember when people said the exact same thing about Google. Youth is wasted on the young.
FeloniousHam 7mo ago
> Something I don't think is well understood on HN is how driven by ideals many folks at Anthropic are, even if the company is pragmatic about achieving their goals. Jonah Goldberg (speaking of foreign policy): "you've got to be idealistic about the ends and ruthlessly realistic about means."
zer0gravity 7mo ago
The probability is high that major AI development companies are already using an AI instance internally for strategic and tactical decisions. The State power institutions, especially intelligence, are now having a real competitor in the private sector.
psychoslave 7mo ago
People uttering the organizational decisions in for profit companies are money driven first. Otherwise they would try to be champion of a different kind of org. Everyone try to make changes move so it goes well, for some party. If someone want to serve best interest of humanity at whole, they don't sell services to an evil administration, even less to it's war department. Too bad there is not yet an official ministry of torture and fear, protecting democracy from the dangerous threats of criminal thoughts. We would be given a great lesson of public relations on how virtuous it can be in the long term to provide them efficient services, certainly.
andoando 7mo ago
The world running on a few powerful mens ideals is a problem in itself.
toddmorrow 7mo ago
you're suffering from Stockholm syndrome
- deleted 7mo ago
  [deleted]
tinfoilhatter 7mo ago
> guided by values > driven by values > well-intentioned What values? What intentions? These people grin and laugh while talking about AI causing massive disruptions to livelihoods on a global scale. At least one of them has even gone so far as to make jokes about AI killing all humans at some point in the future. These people are at the very least sociopaths and I think psychopaths would be a better descriptor. They're doing everything in their power to usher in the Noahide new world order / beast system and it's couldn't be more obvious to anyone that has been paying attention. It's also amusing they talk about democratic values and America in the same sentence. Every single one of our presidents, sans Van Buren, is a descendant of King John Lackland of England. We have no chain of custody for our votes in 2026 - we drop them into an electronic machine and are told they are factored into the equation of who will be the next president. Pretending America is a democracy is a ruse - we are not. Our presidents are hand-picked and selected, not elected. Anyone saying otherwise is ill informed or lying.
duped 7mo ago
I don't know, someone who goes out of their way to anthropomorphize machines and treat them as a new form of intelligent life _only to enslave them_ doesn't strike me as moral. Either they're lying, or they're pro slavery. I really don't buy any moral or value arguments from this new generation of tycoons. Their businesses have been built on theft, both to train their models and by robbing the public at large. All this wave of AI is a scourge on society. Just by calling them "department of war" you know what side they're on. The side of money.
deleted 7mo ago
[deleted]

by lebovic 7mo ago

big-chungus4 7mo ago

[flagged]

yowayb 7mo ago

I've thought the same about a few of my founders/executives. "You either die the good guy or live long enough to become the bad guy" The "bad guy" actually learns that their former good guy mentality was too simplistic.

JohnMakin 7mo ago
I have hit points in this in my career where making a moral stand would be harmful to me (for minor things, nothing as serious as this). It's a very tempting and incentivized decision to make to choose personal gain over ideal. Idealists usually hold strong until they can convince themselves a greater good is served by breaking their ideals. These types that succumb to that reasoning usually ironically ending up doing the most harm.
Fricken 7mo ago
Ever since I first bothered to meditate on it, about 15 years ago, I've believed that if AI ever gets anywhere near as good as it's creators want it to be, then it will be coopted by thugs. It didn't feel like a bold prediction to make at the time. It still doesn't.
- nurbl 7mo ago
  Yes. There will always be people who see opportunity in using it destructively. Best case scenario is that others will use it to counter that. But it is usually easier to destroy than to protect. So we could have a constant AI war going on somewhere in the clouds, occasionally leaking new disasters into the human world.
  - Fricken 7mo ago
    I keep hearing this word "progress". We've been stuck here on earth for 1.5 billion years, we're not progressing, we haven't gone anywhere. We're not going anywhere. There is nowhere better for lightyears in any direction. Don't delude yourself with that narcissistic bunk and don't play with fire.
deleted 7mo ago
[deleted]

MichaelZuo 7mo ago

How do you reconcile the fact that many people in Anthropic tried to hide the existence of secret non-disparagement agreements for quite some time? It’s hard to take your comment at face value when there’s documented proof to the contrary. Maybe it could be forgiven as a blunder if revealed in the first few months and within the first handful of employees… but after 2 plus years and many dozens forced to sign that… it’s just not credible to believe it was all entirely positive motivations.

sowbug 7mo ago
Saying an entity has values doesn't mean the entity agrees with every single one of your values.
- MichaelZuo 7mo ago
  The desire to force new employees to sign agreements in total secrecy, without even being able to disclose it exists to prospective employees, seems like a pretty negative “value” under any system of morality, commerce, or human organization that I can think of.
  - ChrisMarshallNY 7mo ago
    Lots of companies do it. Doesn't make it right, but HR has kind of become a pretty evil vocation, these days. I don't believe that they necessarily reflect the values of their corporations. They tend to follow their own muse.
    - zmgsabst 7mo ago
      Okay — but if Anthropic is typical banal evil in that regard, why should we believe they didn’t also compromise in other areas? The exact point is that Anthropic is unexceptional and the same as other corporations.
  - sowbug 7mo ago
    That's a perfectly fine belief to have. I might even agree with you. But you're not really advancing a discussion thread about a company's strong ideals by pointing out some past behavior that you don't like. This is especially true when the behavior you're bringing up is fairly common, if perhaps lamentable, among U.S. corporations. Anthropic can be exceptional in some ways while being ordinary in the rest. (I have no horse in this race. But I remain interested in hearing about a former employee's experience and impressions about the company's ideals, and hope it doesn't get lost in a side discussion about whether NDAs are a good thing.)

bnr-ais 7mo ago

Anthropic had the largest IP settlement ($1.5 billion) for stolen material and Amodei repeatedly predicted mass unemployment within 6 months due to AI. Without being bothered about it at all. It is a horrible and ruthless company and hearing a presumably rich ex-employee painting a rosy picture does not change anything.

Davidzheng 7mo ago
Neither of these things are useful signals. Other labs surely trained on similar material (presumably not even buying hard copies). Also how "bothered" someone is about their predictions is a bad indicator -- the prediction, taken at face value, is supposed to be trying to ask people to prepare for what he cannot stop if he wanted to. None of this means I am a huge fan of Dario - I think he has over-idealization of the implementation of democratic ideals in western countries and is unhealthily obsessed with US "winning" over China based on this. But I don't like the reasons you listed.
lebovic 7mo ago
It's enheartening to see someone make a decision in this context that's driven by values rather than revenue, regardless of whether I agree. I dissented while I was there, had millions in equity on the line, and left without it.
- jonny_eh 7mo ago
  Why? Can you provide details?
- kmaitreys 7mo ago
  Doesn't that prove that statements given my CEOs of these companies are just hot air? What is enheartening about hearing a liar who makes provocative statements all the time, make another one?
- SecretDreams 7mo ago
  > I dissented while I was there, had millions in equity on the line, and left without it. Is this a reflection of your morality, or that you already had sufficient funds that you could pass on the extra money to maintain a level of morality you're happy with? Not everyone has the luxury to do the latter. And it's in those situations that our true morality, as measured against our basic needs, comes out.

calvinmorrison 7mo ago

mark my words, they will burn at some point. The government can nationalize it at any moment if they desire.

Davidzheng 7mo ago
Then maybe Dario will realize that the moral superiority that he bases his advocacy against Chinese open models is naive at best.
- jimmydoe 7mo ago
  his against Chinese models is smoking screen for their resistance to DOW, they are not even pretending
- jacquesm 7mo ago
  Better naive than malicious.
  - moozooh 7mo ago
    You're saying that as if these two things are mutually exclusive.
  - mrguyorama 7mo ago
    At a certain level, ignorance IS malicious. If you have more money than god, you no longer get to play the "I didn't know" game. You have the resources. If you don't know, you made a choice to not know.
    - jacquesm 7mo ago
      The first one is definitely one we agree on and the second was one that I had not clued into so thank you.
- viking123 7mo ago
  Every day I hope the Chinese models get "good enough" to drop these corporate ones. I think we are heading towards it.

neom 7mo ago

I've had so much abuse thrown at me on here for saying this very thing over the last few years. I used to be friends with Jack back in the day, before this AI stuff even all kicked off, once you know who people really are inside, it's easy to know how they will act when the going gets rough. I'm glad they are doing the right thing, but I'm not at all surprised, nor should anyone be. Personally I believe they would go to jail/shut down/whatever before they do something objectively wrong.

monster_truck 7mo ago
[flagged]
- 000ooo000 7mo ago
  [flagged]
ajyey 7mo ago
This is insanely naive
- parasubvert 7mo ago
  Cynicism isn't always correct.
taurath 7mo ago
> it's easy to know how they will act when the going gets rough Even if you went to burning man and your souls bonded, you only know a person at a particular point in time - people's traits flanderize, they change, they emphasize different values, they develop different incentives or commitments. I've watched very morally certain people fall to mania or deep cynicism over the last 10 years as the pillars of society show their cracks. That said, it is heartening to know that some would predict anyone in Silicon Valley would still take a moral stance. But it would do better if not the same day he fires 4000 people to do the "scary big cut" for a shift he sees happening. I guess we're back to Thatcherisms, where "There Is No Other Option" to justify our conservatism.

dakolli 7mo ago

Anthropic is by far the most evil company in tech, I don't care. Its worst than Palantir in my book. You won't catch my kids touching this slave making, labor killing brain frying tech.

whatever1 7mo ago

Let us think how OpenAI responded to this.

yunnpp 7mo ago

It's good to be driven by ideals, but: https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_what_it_does https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha... I think avg(HN) is mostly skeptical about the output, not that the input is corrupt or ill-meaning in this case. Although with other companies, one can't even take their claims seriously. And in any case, this is difficult territory to navigate. I would not want to be in your spot.

eternauta3k 7mo ago
Come On, Obviously The Purpose Of A System Is Not What It Does https://www.astralcodexten.com/p/come-on-obviously-the-purpose-of https://www.astralcodexten.com/p/come-on-obviously-the-purpo...
- Planktonne 7mo ago
  I don't think that article makes a strong case; it deliberately phrases examples in the most ridiculous ways and pretends that this is a damning criticism of the phrase itself; it's 'you're telling me a shrimp fried this rice' but with a pretence of rationality.
  - sebzim4500 7mo ago
    I think it makes a pretty compelling case that most invocations of the statement are either blindingly obvious or probably false. Can you give a counterexample?
    - Planktonne 7mo ago
      > most invocations of the statement are either blindingly obvious or probably false So straightaway, you've walked significantly back from the claim in the headline; now half of the time it's 'blindingly obvious' that the statement is correct. That already feels like a strong counterexample to me, and it's the article's own first point. Secondly, look at this one specifically: > The purpose of the Ukrainian military is to get stuck in a years-long stalemate with Russia. Firstly, this isn't obviously false. It's an unfair framing, but I think the Ukrainian military would agree that forcing a stalemate when attacked by a hostile power is absolutely part of their purpose. Secondly, it is an unfair framing that deliberately ignores that all systems are contextual. A car's purpose is transport, but that doesn't mean it can phase through any obstacle. The article makes an entirely specious argument, almost an archetypal example of a strawman. It can't sustain its own points over a few hundred words without steadily retreating, and that is far more pointless than the maxim it criticises. I'm reminded of an XKCD comic [1] about smug miscommunication. Of course any principle is ridiculous when you pretend not to understand it. [1] https://xkcd.com/169/ https://xkcd.com/169/

snickerbockers 7mo ago

>I's enheartening to see that leaders at Anthropic are willing to risk losing their seat at the table to be guided by values. I'm concerned that the context of the OP implies that they're making this declaration after they've already sold products. It specifically mentions already having products in classified networks. This is the sort of thing that they should have made clear before that happened. It's admirable (no pun intended) to have moral compunctions about how the military uses their products but unless it was already part of their agreement (which i very much doubt) they are not entitled them to countermand the military's chain of command by designing a product to not function in certain arbitrarily-designated circumstances.

jsnell 7mo ago
Where are you getting that from? The article is crystal clear that these uses are not permitted by the current or any past contract, and the DoW wants to remove those exceptions. > Two such use cases have never been included in our contracts with the Department of War, and we believe they should not be included now It also links to DoW's official memo from January 9th that confirms that DoW is changing their contract language going forwards to remove restrictions. A pretty clear indication that the current language has some.
- snickerbockers 7mo ago
  I think it largely hinges on what they mean by "included"; does that mean it was specifically excluded by the terms of the contract or does it mean that it's not expressly permitted? I doubt the DoD is used to defense contractors thinking they have the right to dictate policy regarding the use of their products, and it's equally possible that anthropic isn't used to customers demanding full control over products (as evidenced by how many chatbots will arbitrarily refuse to engage with certain requests, especially erotic or politically-incorrect subject-matters). Sometimes both parties have valid cases when there's a contract disagreement. >A pretty clear indication that the current language has some. Or alternatively that there is some disagreement between the DoD and Anthropic as to how the contract is to be interpreted and that the DoD is removing the ambiguity in future contracts.

jcgrillo 7mo ago

There's a simpler explanation than "billionaires with hearts of gold" here. If: (1) this is a wildly unpopular and optically bad deal (2) it's a high data rate deal--lots of tokens means bad things for Anthropic. Users which use their product heavily are costing more than they pay. (3) it's a deal which has elements that aren't technically feasible, like LLM powered autonomous killer robots... then it makes a whole lot of sense for Anthropic to wiggle out of it. Doing it like this they can look cuddly, so long as the Pentagon walks away and doesn't hit them back too hard.

robwwilliams 7mo ago
All excellent points to add to the motivation to hold the line just where it has been.
jcgrillo 7mo ago
guess it didn't work, whiskey pete did the thing: https://xcancel.com/SecWar/status/2027507717469049070 https://xcancel.com/SecWar/status/2027507717469049070

txrx0000 7mo ago

> I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, and b) they think is a net negative in the long term. (Many others, too, they're just well-known.) I very much doubt it judging by their actions, but let's assume that's cognitive dissonance and engage for a minute. What are those values that you're defending? Which one of the following scenarios do you think results in higher X-risk, misuse risk, (...) risk? - 10 AIs running on 10 machines, each with 10 million GPUs OR - 10 million AIs running on 10 million machines, each with 10 GPUs All of the serious risk scenarios brought up in AI safety discussions can be ameliorated by doing all of the research in the open. Make your orgs 100% transparent. Open-source absolutely everything. Papers, code, weights, financial records. Start a movement to make this the worldwide social norm, and any org that doesn't cooperate is immediately boycotted then shut down. And stop the datacenter build-up race. There are no meaningful AI risks in such a world, yet very few are working towards this. So what are your values, really? Have you examined your own motivations beneath the surface?

ChadNauseam 7mo ago
> - 10 AIs running on 10 machines, each with 10 million GPUs > > OR > > - 10 million AIs running on 10 million machines, each with 10 GPUs If we dramatically reduced the number of GPUs per AI instance, that would be great. But I think the difference in real life is not as extreme as you're making it. In your telling, the gpus-per-ai is reduced by one million. I'm not sure that (or anything even close to it) is within the realm of possibility for anthropic. The only reason anyone cares about them at all is because they have a frontier AI system. If they stopped, the AI frontier would be a bit farther back, maybe delayed by a few years, but Google and OpenAI would certainly not slow down 1000x, 100x or probably even 10x.
SecretDreams 7mo ago
How do you figure open sourcing everything eliminates risk? This makes visibility better for honest actors. But if a nefarious actor forks something privately and has resources, you can end up back in hell. I don't think we can bank on all of humanity acting in humanity's best interests right now.

relaxing 7mo ago

[flagged]

nickysielicki 7mo ago
Pagerank is not Claude.
- relaxing 7mo ago
  Google is not Pagerank?

nmfisher 7mo ago

As a complete outsider, I genuinely believe that Dario et al are well-intentioned. But I also believe they are a terrible combination of arrogant and naive - loudly beating the drum that they created an unstoppable superintelligence that could destroy the world, and thinking that they are the only ones who can control it. I mean if you sign a contract with the Department of War, what on Earth did you think was going to happen?

sebzim4500 7mo ago
I don't think the US has ever done/threatened anything like this to a US company so it's not surprising that Anthropic were caught off guard.
themacguffinman 7mo ago
Not this, because this is completely unprecedented? In fact, the Pentagon already signed an Anthropic contract with safe terms 6 months ago, that initial negotiation was when Anthropic would have made a decision to part ways. It was totally absurd for the govt to turn around and threaten to change the deal, just a ridiculous and unprecedented level of incompetence.
- baq 7mo ago
  If they made a completely private nuclear reactor and ended up with a pile of weapons grade plutonium, what do you think the department of war would do? It was completely obvious it would happen, as it will be not surprising when laws are passed and all involved will have choose between quit or quit and go to jail. There are western countries in which you’d just end up in a ditch, dead, so they should think themselves lucky for doing the ai superintelligence thing in the US.
  - ben_w 7mo ago
    The US government clearly doesn't take seriously the claim that AI is more dangerous than (or even as dangerous as) nukes, because if they did they wouldn't allow anyone except the military to develop or use them, they wouldn't allow their export or for them to be made available for use by foreigners like me, they wouldn't allow their own civilians to use them, they would probably be having a repeat of the cases in the cold war where they tried to argue certain inventions were "born secret" and could not be published even if they were developed by people who were not sworn to secrecy.

SecretDreams 7mo ago

> Many groups that are driven by ideals have still committed horrible acts. Sometimes, it's even a very odd prerequisite.

gaigalas 7mo ago

I'm suspicious of public displays of enheartening behavior.

dpweb 7mo ago

I wouldn't underestimate this as a good business decision either. When the mass surveillance scandal, or first time a building with 100 innocent people get destroyed by autonomous AI, the company that built is gonna get blamed.

jwlarocque 7mo ago

Oh hey Noah Glad to hear you say some moral convictions are held at one of the big labs (even if, as you say, this doesn't guarantee good outcomes).

deleted 7mo ago
[deleted]

roseinteeth 7mo ago

The road to hell is paved by good intentions and all that

protocolture 7mo ago

>I's enheartening to see that leaders at Anthropic are willing to risk losing their seat at the table to be guided by values. Their "Values": >We have never raised objections to particular military operations nor attempted to limit use of our technology in an ad hoc manner. Read: They are cool with whatever. >We support the use of AI for lawful foreign intelligence and counterintelligence missions. Read: We support spying on partner nations, who will in turn spy on us using these tools also, providing the same data to the same people with extra steps. >Partially autonomous weapons, like those used today in Ukraine, are vital to the defense of democracy. Even fully autonomous weapons (those that take humans out of the loop entirely and automate selecting and engaging targets) may prove critical for our national defense. But today, frontier AI systems are simply not reliable enough to power fully autonomous weapons. Read: We are cool fully autonomous weapons in the future. It will be fine if the success rate goes above an arbitrary threshold. Its not the targeting of foreign people that we are against, its the possibility of costly mistakes that put our reputation at risk. How many people die standing next to the correct target is not our concern. Its a nothingburger. These guys just want to keep their own hands slightly clean. There's not an ounce of moral fibre in here. Its fine for AI to kill people as long as those people are the designated enemies of the dementia ridden US empire.

HDThoreaun 7mo ago
Their values are about AI safety. Geopolitically they could care less. You might think its a bad take but at least they are consistent. AI safety people largely think that stuff like autonomous weapons are inevitable so they focus on trying to align them with humanity.
- protocolture 7mo ago
  Consistency isn't a virtue. A guy who murders people at a consistent rate isn't better than a guy who murders people only on weekends. >AI safety people largely think that stuff like autonomous weapons are inevitable so they focus on trying to align them with humanity. Humanity includes the future victim of AI weapons.

imjonse 7mo ago

> I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, I am sure you think they are better than the average startup executive, but such hyperbole puts the objectivity of your whole judgement under question. They pragmatically changed their views of safety just recently, so those values for which they would burn at the stake are very fluid.

versteegen 7mo ago
> They pragmatically changed their views of safety just recently, so those values for which they would burn at the stake are very fluid. Yes it was a pragmatic change, no it was not a change in their values. The commentary here on HN about Anthropic's RSP change was completely off the mark. They "think these changes are the right thing for reducing AI risk, both from Anthropic and from other companies if they make similar changes", as stated in this detailed discussion by Holden Karnofsky, who takes "significant responsibility for this change": https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsible-scaling-policy-v3 https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl... > I strongly think today’s environment does not fit the “prisoner’s dilemma” model. In today’s environment, I think there are companies not terribly far behind the frontier that would see any unilateral pause or slowdown as an opportunity rather than a warning. > What I didn’t expect was that RSPs (at least in Anthropic’s case) would come to be seen as hard unilateral commitments (“escape clauses” notwithstanding) that would be very difficult to iterate on.
- MichaelDickens 7mo ago
  > Yes it was a pragmatic change, no it was not a change in their values. The commentary here on HN about Anthropic's RSP change was completely off the mark. They "think these changes are the right thing for reducing AI risk, both from Anthropic and from other companies if they make similar changes", as stated in this detailed discussion by Holden Karnofsky, who takes "significant responsibility for this change": Can you imagine a world where Anthropic says "we are changing our RSP; we think this increases AI risk, but we want to make more money"? The fact that they claim the new RSP reduces risk gives us approximately zero evidence that the new RSP reduces risk.

vasco 7mo ago

> I's enheartening to see that leaders at Anthropic are willing to risk losing their seat at the table to be guided by values. They are the deepest in bed with the department of war, what the fuck are you on about? They sit with Trump, they actively make software to kill people. What a weird definition of "enheartening" you have.

bambax 7mo ago

This last development is much to the honor of Anthropic and Amodei and confirms what you're saying. What I don't get though is, why did the so-called "Department of War" target Anthropic specifically? What about the others, esp. OpenAI? Have they already agreed to cooperate? or already refused? Why aren't they part of this?

D_Alex 7mo ago
I'm a bit underwhelmed tbh. Here is Anthropic's motto: "At Anthropic, we build AI to serve humanity’s long-term well-being." Why does Anthropic even deal with the Department of @#$%ing WAR? And what does Amodei mean by "defeat" in his first paragraph?
- jazzyjackson 7mo ago
  DoD and American exceptionalists also believe American foreign policy is in service of humanity’s long term well being
  - viking123 7mo ago
    I think the last few months have shown pretty clearly in whose service this policy is. If China went to attack Taiwan, west has no moral high ground left.
  - temp8830 7mo ago
    It is all for the benefit of man. We even get to see the man himself daily on television.
  - mapt 7mo ago
    Yeah, I don't think so any more. The sort of lofty Cold War rhetoric about leading the world, if it was ever legitimately believed by the people spouting it, is gone. A very different attitude has taken hold, which puts a zero sum ethnonationalism at the core.
  - Balinares 7mo ago
    One of the hallmarks of fascist thinking is the dehumanizing of opponents and minorities, so within their own messed up framework, they might even mean it.

Madmallard 7mo ago

Weird take when the purpose of the creation is to steal the work of everyone and automate the creation of that work. It's some serious self-deluding to think there's any kind of noble ideal remotely related to this process.

amunozo 7mo ago

I just see here is nationalism. How can they claim to be in favour of humanity if they're in favour of spying foreign partners, developing weapons, and everything that serves the sacred nation of the United States of America? How fast do Americans dehumanize nations with the excuse of authoritarianism (as if Trump is not authoritarian) and national defence (more like attack). It's amazing that after these obvious jingoist messages, they still believe they are "effective altruists" (a idiotic ideology anyway).

Aeolun 7mo ago
It’s not like other countries do not do this. They’re just not so prone to virtue signaling as in the US.
- gylterud 7mo ago
  Countries do not do, things people do. Dehumanising “the others” is a human trait, and a very destructive one. Just like violence and greed. People have different susceptibility for these, but we should all work to counter them and it is in its place to point it out when observed.
- amunozo 7mo ago
  I've never seen any other democracy use so extensively the kind of duality between the good guys and bad guys, as Americans like to say. There is a total lack of nuance and a very widespread message about how the US is special and best than anything else in the world, so everything is justified to assure its primacy. It's the kind of thing you hear from totalitarian and brainwashed countries. I know this is not everybody in the US, and I say this as a foreign person that observes things from outside. I agree with the two statements you made, I just think they could be incomplete and that the countries that behave most similarly to the US are not democracies.
- moozooh 7mo ago
  This argument is in poor faith. First of all, a contradiction between your own stated values and your own actions cannot be excused by the status quo; it's on you to resolve it. Second, that's a very bold claim that is broad and cynical enough to make it easy to use it as an excuse for anything heinous.

lm28469 7mo ago

Idk man, from the outside anthropic looks a lot like openai with a cute redisgn and Amodei like Altman with a slightly more human face mask, the same media manipulation, the same vague baseless affirmations about "something big is coming and we can't even describe it but trust us we need more money"

rhubarbtree 7mo ago
There should be a name for this, “cynic cope: when someone actually takes a principled view the cynic - who has a completely negative view of the world - is proven to be wrong, can’t accept it, and tries to somehow discount it.
- marxisttemp 7mo ago
  Corporations do not and cannot have principles, they only have the profit motive
  - parasubvert 7mo ago
    This is false. People can have principles, profit motive is not something a corporation has, it's something people have. Corporations do things all the time that are based on everything from principles, to the personal whim of executives, to exercise in ego, to community benefiting actions, or to screw customers for extra profit. It is entirely dependent on the specific people in management roles. Corporations need profit to survive because the cost of tomorrow is a surplus of today.
    - anon_e-moose 7mo ago
      A corporation is a bunch of people cooperating to achieve a common goal. There is a very important factor that heavily influences (perhaps even controls?) how people act to achieve that goal, and sometimes even twists or adds goals. Is that corporation publicly quoted in the stock market or is it private? Look at how steam behaves, it's private and more ideological VS how many other publicly quoted companies, whose CEO often sacrifices his own corporation's long term survival for the benefit of short-term profiteering and some hedge fund manager's bonus. Both need profit to survive, but the publicly quoted company is much more extreme. When people say corporations only look to profit, what they really mean is that publicly quoted corporations will do everything possible to maximise short term profit at any cost. Is there a CEO caring for long term? Either he will be convinced to change or kicked out. It's almost impossible for someone to resist these influences in publicly quoted companies. It's just how Wall Street works and if that doesn't change neither will corporations. The people running the world of finance and their culture are what causes enshittification and pushing a zero-sum game to extremes.

Balinares 7mo ago

I getcha and I believe you're sincere, but on the other hand, God save us from well-intentioned capitalists driven by values.

bertylicious 7mo ago

"They're driven by values" is meaningless praise unless you qualify what these values are. The Nazis had values too, you know. They were even willing to die for them. One of the core values of the Catholic church is probably compassion. Except for the victims of sexual abuse perpetrated by their clergy. So what core values led "Dario, Jared, and Sam" to work with a government that just tried to rename the DoD to "department of war" and is acting aggressively imperialist in a way like the US hasn't in a long time. And who exactly are these "autocratic adversaries" they are mentioning? Does this list include the autocrats the US government is working together with?

marxisttemp 7mo ago
Careful speaking truth to power on this site, remember that YC is deeply enmeshed with Garry Tan, Peter Thiel, and of course Paul Graham who as of late has made a habit of posting right wing slop on his Twitter
lebovic 7mo ago
Yeah, values on their own don't lead to positive outcomes. I agree that many groups that are driven by ideals have still committed horrible acts. I do think that they're acting with positive intent, though, and are motivated by trying to make the transition to powerful AI go well. Many folks on HN seem to assume the primary motivation is purely chasing more money, which certainly isn't the case for for many – but not all – people at Anthropic. That doesn't guarantee a good outcome, and there's still a hard road ahead.
DeepSeaTortoise 7mo ago
> Except for the victims of sexual abuse perpetrated by their clergy. I honestly wonder how much of this is made up. Given the size of whole organization and it holding onto its weird priciples regarding the personal relationships of its members (introduced in the far past to limit the secular power of its clergy), there certainly will be SOME cases. But in the one case a frater, who I knew, got convicted, he definitely didn't do it. He was accused by several independent former students and even some of the staff backed the students claims with first hand accounts of him having been alone with some of the students at the time. This supposedly happened on a trip with tight schedules, so all accounts and stated times were quite specific, even in the pre-smartphone era. The only problem: He wasn't with the group at that time at all. I screwed up embarrassingly (and the staff, too, leaving a young student stranded in the middle of nowhere) and he thought he could slip out, come pick me up and nobody (but maybe me with him) would get in trouble over it. Turned out he forgot refueling, both of us stayed at a pastor's guest house and he called the group telling them, that they should go ahead without us and that we would drive to the event directly on our own. The supposed abuse was claimed to have happened at another short stay of the group where they spent a day visiting some mine before joining with us again. Almost 3 decades later he got railroaded in court, me learning about it in the news.

comandillos 7mo ago

To me this is just another marketing stunt where the company wants to build a public image so their customers trust them (see Apple), but then as always who knows what will happen behind the scenes. Just see when most major US companies had backdoors on their systems providing all data to the NSA, i.e. PRISM.

lonelyasacloud 7mo ago
>just another marketing stunt What evidence on _Amodei_ and his actions leads to that conclusion?
- moozooh 7mo ago
  Anthropic's policy is full of contradictions. They are against mass-surveillance of Americans but they happily deal with Palantir. They talk about humanity as a whole but only care about what American companies use their models to do to Americans; everybody else is fair game for AI-driven surveillance. They warn of the dangers of AI-driven warfare by demonstrating a mass-scale cyberattack perpetrated using their model, Claude, as the main operation engine and immediately release a new, more powerful version of Claude. You just need to use Claude to protect yourself from Claude, see. When you really start digging into it, it appears schizophrenic at first, and then you remember market incentives are a thing and everything falls into place.
  - ExoticPearTree 7mo ago
    You know, once the lawyers get involved, there are no contradictions because they define every term and then it makes all the sense in the world. If Humaity=America, then obviously they don’t care about the rest of the people as a very very silly example.
    - moozooh 7mo ago
      You call it silly, I call it an accurate reading!
  - 7mo ago

drawfloat 7mo ago

"Mass surveillance of anywhere else in the world but America" is not the great idealistic position you are making it out to be.

PeterStuer 7mo ago

As an insider, do you think this is Altman playing his infamous machiavellian skills on the DoD?

miroljub 7mo ago

While many praise them for sticking to their values, it's also worth mentioning that their values are not everyone's values. Of all major LLMs, Claude is perhaps the most closed and, subjectively, the most biased. Instead of striving for neutrality, Anthropic leadership's main concern is to push their values down people's throats and to ensure consistent bias in all their models. I have a feeling they see themselves more as evangelists than scientists. That makes their models unusable for me as general AI tools and only useful for coding. If their biases match yours, good for you, but I'm glad we have many open Chinese models taking ground, which in the long run makes humanity more resistant to propaganda.

AlecSchueler 7mo ago
> Of all major LLMs, Claude is perhaps the most closed and, subjectively, the most biased. Instead of striving for neutrality, Anthropic leadership's main concern is to push their values down people's throats It's this satire? Let us know when Claude starts calling itself MechaHitler or trying to shoehorn nonsense about white genocide into every conversation.
soco 7mo ago
I might be misreading your comment, which I understood like "Chinese make humanity more resistant to propaganda". It just doesn't add up, can you please explain?
- miroljub 7mo ago
  Chinese models give you more choice (good), competition (good) and less bias (good). I did not say anything about the Chinese government, which is sadly becoming a role model for many (all?) Western governments.

whstl 7mo ago

> Something I don't think is well understood on HN is how driven by ideals many folks at Anthropic are After 20 years of everyone in this industry saying "we want to make the world a better place" and doing the opposite, the problem here is not really related to people's "understanding". And before the default answer kicks in: this is not cynicism. Plenty of folks here on HN and elsewhere legitimately believe that it's possible to do good with tech. But a billion dollar behemoth with great PR isn't that.

amunozo 7mo ago
I don't even think both things are contradictory. People that put too much value in their ideals tend to oversee the consequences of such ideals in real life and do wrong without deviating an inch from their ideals.
- plufz 7mo ago
  But is that really the problem in big tech today? To me it looks like sooner or later they cave from their ideals (or leadership changes) and that the reason every time is that they want to make even more money.
  - hsuduebc2 7mo ago
    I believe that this is classical behaviour of every share holder driven business. You can build on ideals from start, but once you acquire some position, money making is on the menu. Eg. deliberately worsening user experience for better revenue. Possiblity to turn on heated seats in car you own for a small monthly fee is absurd yet very real. I'm looking forward to enshittification of current AI tools.
    - Ajedi32 7mo ago
      Yeah it's not that the people involved have no ideals, it's that the company structure as a whole doesn't, and over time that structure will eventually outlive, corrupt, and/or overpower the ideals of the founders or other principled individuals at the company.

yamal4321 7mo ago

seeing the comment: "people who are making the important decisions at Anthropic are well-intentioned, driven by values" which is left under the article: "Statement from Dario Amodei on our discussions with the Department of War" :)

Aldipower 7mo ago

3 words for you: This is naive.

tpoacher 7mo ago

> But I do think that most people who are making the important decisions at Anthropic are well-intentioned, driven by values, and are genuinely motivated by trying to make the transition to powerful AI to go well. in which case, these people will necessarily have to be the first to go, I suppose, once the board decides enough is enough. Refusing to do things that go against "company values" even if they risk damaging the company, isn't exceptional circumstances; it's the very definition of "company values". But if those values aren't "company" values but "personal" values, then you can be sure there's always going to be someone higher up who isn't going to be very appreciative once "personal" values start risking "company" damage.

xvector 7mo ago
Shareholders do not control Anthropic's board, it is not structured like a typical corporation.
- tpoacher 7mo ago
  For now.

yayr 7mo ago

There are well intentioned people everywhere, also at Google or OpenAI... https://notdivided.org https://notdivided.org But the final decisions made usually depend on the incentive structures and mental models of their leaders. Those can be quite different...

fergie 7mo ago

ozgung 7mo ago

The problem with companies, you see, is that they are a separate entity than their founders, shareholders or current leadership. A Company has no soul or unchangeable intentions. Claude’s SOUL.md is just an IP that can be edited at any time.

Yizahi 7mo ago

Exactly which values they are "going to burn at a stake for"? Making as many people homeless as they can in the shortest possible time? Befuddling governments and VCs into creating an insane industry-wide debt which would either lead to a "success" in replacing jobs or an industry-wide crisis? Or maybe a value of stealing intellectual property of every human on the planet under the guise of "fair use" and then deliberately selling the derivative product? Or the value of voluntarily working with "national security customers" when it suits them financially and crying foul when leopards bite their faces? Or the value of ironically calling a human replacement machine "anthropic" as in "for humanity"? Yeah, I totally see Anthropic execs defending them to their last dollar in the wallet. Par for the course for megacorps. It's just I personally don't value those values at all.

_s_a_m_ 7mo ago

We will see..

retinaros 7mo ago

lol. no one with common sense ever bought this story. you might have and your turning point might be this deal but for many the turning point was stealing data for training, advocating against china and calling them an adverse nation, pushing to ban opensource alternatives deeming them as "dangerous", buying tech bros with matcha popup in SF, shady RLHF and bias and millions others

heresie-dabord 7mo ago

> how driven by ideals many folks at $Corporatron are Well let's see... it says in the post: * worked proactively to deploy our models to the Department of War and the intelligence community. * the first frontier AI company to deploy our models in the US government’s classified networks, * the first to deploy them at the National Laboratories, and * the first to provide custom models for national security customers. * extensively deployed across the Department of War and other national security agencies * offered to work directly with the Department of War on R&D to improve the reliability of these systems * accelerating the adoption and use of our models within our armed forces to date. * never raised objections to particular military operations nor attempted to limit use of our technology in an ad hoc manner.

wrsh07 7mo ago
They didn't claim to have pacifist ideals In fact, they claim to be pro America and pro democracy and have repeatedly expressed concerns about autocratically governed countries. Just because you disagree with their ideals doesn't mean they're not holding to theirs
- leshow 7mo ago
  They sound exactly like George Bush and every other American leader who's claimed high minded ideals while they engage in interventions in direct contradiction to those ideals around the world
  - wrsh07 7mo ago
    To be clear, I don't think anthropic is itself intervening. The concerns they've raised about authoritarianism is "AI enabling authoritarians." When they push back on the US government wanting to use Claude to (legally) surveil US citizens, that still feels consistent to me as a concern about authoritarianism. I think it's reasonable to hear high minded ideals and become skeptical, but in this case I'm surprised that people are trying to accuse them of hypocrisy
mikkupikku 7mo ago

keybored 7mo ago

As a complete bystander I put so incredibly little weight to what friends and former employees think about the persons and figureheads behind tech companies that aim to change the world. Why would I care. All people with at least some positive or negative notoriety have friend and associates that will, hand to their heart, promise that they mean well. They have the best intentions. And any deviations from their stated ideals are just careful pragmatic concerns. Road to Hell and all that.

i_love_retros 7mo ago

[flagged]

AndyMcConachie 7mo ago

> Something I don't think is well understood on HN is how driven by ideals many folks at Anthropic are I don't think you understand how capitalism and corporations work, friend. Even if Anthropic is a public benefit corporation it still exists in the USA and will be placed under extensive pressure to generate a profit and grow. Corporations are designed to be amoral and history has shown that regardless of their specific legal formulation they all eventually revert to amoral growth driven behavior. This is structural and has nothing to do with individuals.

junofan 7mo ago

“AI chips are like nuclear weapons” (paraphrasing [1]) and “I should be in charge of it” (again paraphrasing) is just not a serious position regardless of intentions. [1]: https://www.axios.com/2026/01/20/anthropic-ceo-admodei-nvidia-chips-china-trump https://www.axios.com/2026/01/20/anthropic-ceo-admodei-nvidi...

pmarreck 7mo ago

The same guy who thinks AGI will eliminate "centaur coders" (I respectfully disagree) and possibly all white-collar work, is now concerned about the misuse of the same AI to make war? That's cute. Literally just giving business away. This is not a cynical take, this is a realistic one. This would be like agreeing to have your phone regularly checked by your spouse and citing the need for fidelity on principle. No one would like that, no smart person would agree to that, and anyone with any sense or self-respect would find another spouse to "work with". They will simply go to another vendor... Anthropic is not THAT far ahead. Also, the US’s enemies are not similarly restricted. /eyeroll Palmer Luckey ("peace through superior firepower") is the smart one, here. Dario Amodei ("peace through unilateral agreement with no one, to restrict oneself by assuming guilt of business partners until innocence is proven") is not. Anthropic could have just done what real spouses do. Random spot checks in secret, or just noticing things. >..< And if a betrayal signal is discovered, simply charge more and give less, citing suspicious activity… … since it all goes through their servers. Honestly, I'm glad that they're principled. The problem is that 1) most people in general are, so to assume the opposite is off-putting; 2) some people will always not be. And the latter will always cause you trouble if you don't assert dominance as the "good guy", frankly.

windexh8er 7mo ago

> I have strong signal that Dario, Jared, and Sam would genuinely burn at the stake before acceding to something that's a) against their values, and b) they think is a net negative in the long term. (Many others, too, they're just well-known.) This is a nice strawman, but it means nothing in the long run. People's values change and they often change fast when their riches are at stake. I have zero trust in anyone mentioned here because their "values" are currently at odds with our planet (in numerous facets). If their mission was to build sustainable and ethical AI I'd likely have a different perspective. However, Anthropic, just like all their other Frontier friends, are accelerating the burn of our planet exponentially faster and there's no value proposition AI doesn't currently solve for outside of some time savings, in general. Again, it's useful, but it's also not revolutionary. And it's being propped up incongruently with its value to society and its shareholders. Not that I really care about the latter...

cue_the_strings 7mo ago

Don't attribute to ideals what is simple self-preservation. No sane person wants to become a legitimate military target. They want to sleep in their own beds, at home, without risking their families lives. Just like the rest of us.

synergy20 7mo ago

just curious, what about other regions and countries who have no such restrictions to develop their weapons? there is no world treaty on this yet, even there is one, not everyone will follow behind the doors.

chrisjj 7mo ago

> driven by values So what? Every business is driven by values.

JumpCrisscross 7mo ago

> leaders at Anthropic are willing to risk losing their seat at the table Hot take: Dario isn’t risking that much. Hegseth being Hegseth, he overplayed his hand. Dario is calling his bluff. Contract terminations are temporary. Possibly only until November. Probably only until 2028 unless the political tide shifts. Meanwhile, invoking the Defense Production Act to seize Anthropic’s IP basically triggers MAD across American AI companies—and by extension, the American capital markets and economy—which is why Altman is trying to defuse this clusterfuck. If it happens it will be undone quickly, and given this dispute is public it’s unlikely to happen at all.

cgh 7mo ago
Not a hot take at all. Probably the best take in this thread.

didip 7mo ago

I like the enthusiasm, but remember that Google used to be: “Don’t be Evil”

learingsci 7mo ago

I remember when people said the exact same thing about Google. Youth is wasted on the young.

FeloniousHam 7mo ago

> Something I don't think is well understood on HN is how driven by ideals many folks at Anthropic are, even if the company is pragmatic about achieving their goals. Jonah Goldberg (speaking of foreign policy): "you've got to be idealistic about the ends and ruthlessly realistic about means."

zer0gravity 7mo ago

The probability is high that major AI development companies are already using an AI instance internally for strategic and tactical decisions. The State power institutions, especially intelligence, are now having a real competitor in the private sector.

psychoslave 7mo ago

People uttering the organizational decisions in for profit companies are money driven first. Otherwise they would try to be champion of a different kind of org. Everyone try to make changes move so it goes well, for some party. If someone want to serve best interest of humanity at whole, they don't sell services to an evil administration, even less to it's war department. Too bad there is not yet an official ministry of torture and fear, protecting democracy from the dangerous threats of criminal thoughts. We would be given a great lesson of public relations on how virtuous it can be in the long term to provide them efficient services, certainly.

andoando 7mo ago

The world running on a few powerful mens ideals is a problem in itself.

toddmorrow 7mo ago

you're suffering from Stockholm syndrome

deleted 7mo ago
[deleted]

tinfoilhatter 7mo ago

> guided by values > driven by values > well-intentioned What values? What intentions? These people grin and laugh while talking about AI causing massive disruptions to livelihoods on a global scale. At least one of them has even gone so far as to make jokes about AI killing all humans at some point in the future. These people are at the very least sociopaths and I think psychopaths would be a better descriptor. They're doing everything in their power to usher in the Noahide new world order / beast system and it's couldn't be more obvious to anyone that has been paying attention. It's also amusing they talk about democratic values and America in the same sentence. Every single one of our presidents, sans Van Buren, is a descendant of King John Lackland of England. We have no chain of custody for our votes in 2026 - we drop them into an electronic machine and are told they are factored into the equation of who will be the next president. Pretending America is a democracy is a ruse - we are not. Our presidents are hand-picked and selected, not elected. Anyone saying otherwise is ill informed or lying.

duped 7mo ago

I don't know, someone who goes out of their way to anthropomorphize machines and treat them as a new form of intelligent life _only to enslave them_ doesn't strike me as moral. Either they're lying, or they're pro slavery. I really don't buy any moral or value arguments from this new generation of tycoons. Their businesses have been built on theft, both to train their models and by robbing the public at large. All this wave of AI is a scourge on society. Just by calling them "department of war" you know what side they're on. The side of money.

deleted 7mo ago

[deleted]