18 ms·

Anthropic drops flagship safety pledge

• ggsp 8mo ago
  It was always a matter of time
• dhruv3006 8mo ago
  Anthropic facing a lot of flak recently.
  • deleted 8mo ago
    [deleted]
  • deleted 8mo ago
    [deleted]
• deleted 8mo ago
  [deleted]
• esafak 8mo ago
  It must be due to pressure from the Defense Dept: The AI startup has refused to remove safeguards that would prevent its technology from being used to target weapons autonomously and conduct U.S. domestic surveillance. Pentagon officials have argued the government should only be required to comply with U.S. law. During the meeting, Hegseth delivered an ultimatum to Anthropic: get on board or the government would take drastic action, people familiar with the matter said. https://www.staradvertiser.com/2026/02/24/breaking-news/anthropic-digs-in-heels-in-ai-dispute-with-pentagon-source-says/ https://www.staradvertiser.com/2026/02/24/breaking-news/anth...
  • crises-luff-6b 8mo ago
    [dead]
  • instagib 8mo ago
    They probably have proof in contracts that they agreed to this usage. They won’t alter the deal based on some bad press nor do they want to lose the DoD-DoW as a customer.
    • alpha_squared 8mo ago
      From what I was reading, it appears that their tools were used outside the scope of their contract with DoD via Palantir's work that also used Claude. Anthropic freaked out, DoD freaked out that Anthropic freaked out and threatened to declare them a supply chain risk. That designation would've required any company that contracts with DoD to strip out any Anthropic tooling from their business in order to continue working with DoD. It was effectively designating Anthropic a terrorist organization.
• mhitza 8mo ago
  The IPOs this year can't come soon enough https://tomtunguz.com/spacex-openai-anthropic-ipo-2026/ https://tomtunguz.com/spacex-openai-anthropic-ipo-2026/
• SirensOfTitan 8mo ago
  What an interesting week to drop the safety pledge. This is how all of these companies work. They’ll follow some ethical code or register as a PBC until that undermined profits. These companies are clearly aiming at cheapening the value of white collar labor. Ask yourself: will they steward us into that era ethically? Or will they race to transfer wealth from American workers to their respective shareholders?
  • BHSPitMonkey 8mo ago
    Could be a sort of canary, with the timing being a spotlight on the highly-visible pressure coming from the U.S. government.
    • johnbellone 8mo ago
      The other providers have already capitulated to a certain extent.
  • hsuduebc2 8mo ago
    When I see slogans like Google’s “Don’t be evil,” it always comes to mind that when it stopped being useful, they shifted to something like “Do the right thing.” It’s important to remember that a company’s primary purpose is profit, especially when it’s accountable to shareholders. That isn’t inherently bad, but the occasional moral posturing used to serve that goal can be irritating.
  • ryaniscool 8mo ago
    If they tank the white-collar middle class, there won't be anyone to buy the goods and services their potential AI customers will be trying to sell. It's like a snake eating its own tail.
• SilverElfin 8mo ago
  This is terrible. It’s caving in to the Trump administration threatening to ban Anthropic from government contracts. It really cements how authoritarian this administration is and how dangerous they can be.
  • deleted 8mo ago
    [deleted]
• bbatsell 8mo ago
  This headline unfortunately offers more smoke than light. This article has nothing to do with the current tête-à-tête with the Pentagon. It is discussing one specific change to Anthropic's "Responsible Scaling Policy" that the company publicly released today as version "3.0".
  • ameliaquining 8mo ago
    I consider this a bigger deal than the Pentagon thing.
    • ActorNightly 8mo ago
      While not surprising at the least, it still kind of crazy that literal pdf files in charge is not concerning, but this is. I just hope something happens to USA before it can do damage to the world.
      • Mordisquitos 8mo ago
        What PDFs are you referring to? Do Anthropic or other LLMs using PDFs as some kind of 'SOUL.md' file or for training?
        • smallerize 8mo ago
          It's a joke way of saying pedophiles -> pdf files.
        • delaminator 8mo ago
          he means pedophiles can't say paedophile on YouTube so people say PDF file
          • ryandrake 8mo ago
            But we're not on YouTube.
• chris_money202 8mo ago
  First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. Then they ignored the researchers warning about what it could do, and I said nothing. It sounded like science fiction. Then they gave it control of things that matter, power grids, hospitals, weapons, and I said nothing. It seemed to be working fine. Then something went wrong, and no one knew how to stop it, no one had planned for it, and no one was left who had listened to the warnings.
  • hsbauauvhabzb 8mo ago
    Plenty of people have said plenty. The problem isn’t the warnings, it’s that people are too stupid and greedy to think about the long term impacts.
    • ifh-hn 8mo ago
      Maybe it's how blunt this comment is that gets it downvoted, but I don't disagree.
      • hsbauauvhabzb 8mo ago
        I’ve noticed anti-AI stance gets downvoted on HN (and any anti-authoritarian comments, for that matter)
      • brookst 8mo ago
        No, it’s because it shows either a simplistic or needlessly confrontational view of the world. Unless you’re independently wealthy (as some in HN are), you have to balance your morals, your views of how things should work, feeding your family, and recognizing that you may not actually know everything. It’s easy to sit back and advise others that they should die on every single hill. But it’s not especially insightful, and serves mostly to signal piety rather than a well thought out view.
        • hsbauauvhabzb 8mo ago
          Spoken like a true LLM.
• jimmydoe 8mo ago
  Either be a company in capitalist USA, or keep being your safety queen. You just can’t be both. The intention to start these pledge and conflict with DOW might be sincere, but I don’t expect it to last long, especially the company is going public very soon.
• crossroadsguy 8mo ago
  I just want Apple and Linux to offer ASAP: 1. Extremely granular ways to let user control network and disk access to apps (great if resource access can also be changed) 2. Make it easier for apps as well to work with these 3. I would be interested in knowing how adding a layer before CLI/web even gets the query OS/browser can intercept it and could there be a possibility of preventing harm before hand or at least warning or logging for say someone who overviews those queries later? And most importantly — all these via an excellent GUI with clear demarcations and settings and we’ll documented (Apple might struggle with documentation; so LLMs might help them there) My point is — why the hell are we waiting for these companies to be good folks? Why not push them behind a safety layer? I mean CLI asks .. can I access this folder? Run this program? Download this? But they can just do that if they want! Make them ask those questions like apps asks on phones for location, mic, camera access.
  • m132 8mo ago
    Indeed, the world would be a much nicer place if only firewalls and Unix permissions existed...
  • VTuberTTV 8mo ago
    [dead]
  • dlt713705 8mo ago
    > I mean CLI asks .. can I access this folder? Run this program? Download this? But they can just do that if they want! Make them ask those questions like apps asks on phones for location, mic, camera access. Basicaly an EDR
• ChrisArchitect 8mo ago
  Related: Hegseth gives Anthropic until Friday to back down on AI safeguards https://news.ycombinator.com/item?id=47140734 https://news.ycombinator.com/item?id=47140734 https://news.ycombinator.com/item?id=47142587 https://news.ycombinator.com/item?id=47142587
  • dbg31415 8mo ago
    They made it until Tuesday! They stood tall as long as they could! =P
  • EagnaIonat 8mo ago
    It's part of the overall story. The safeguards dropped are when they will release a model or not based on safety. The Friday deadline is to allow to use their products for mass surveillance and autonomous weapons systems without a human in the loop. Anthropic hasn't backed down on those, yet. But they are in a bad situation either way. If they don't back down, they lose US government contracts, the government gets to do what it wants anyway. It also puts them in a dangerous position with non-governmental bodies. If they give into the demands, then it puts all AI companies at risk of the same thing. Personally I think they should move to the EU. The recent EU laws align with Anthropics thinking.
• goranmoomin 8mo ago
  TBH I am sad that Anthropic is changing its stance, but in the current world, if you even care about LLM safety, I feel that this is the right choice — there’s too many model providers and they probably don’t consider safety as high priority as Anthropic. (Yes that might change, they can get pressurized by the govt, yada yada, but they literally created their own company because of AI safety, I do think they actually care for now) If we need safety, we need Anthropic to be not too far behind (at least for now, before Anthropic possibly becomes evil), and that might mean releasing models that are safer and more steerable than others (even if, unfortunately, they are not 100% up to Anthropic’s goals) Dogmatism, while great, has its time and place, and with a thousand bad actors in the LLM space, pragmatism wins better.
  • ashtonshears 8mo ago
    Do you work at Anthropic, or know people who do? I genuinly curious why they are so holy to you, when to me I see just another tech company trying to make cash Edit: Reading some of the linked articles, I can see how Anthropic CEO is refusing to allow their product for warfare (killing humans), which is probably a good thing that resonates with supporting them
    • nradov 8mo ago
      How is it a good thing to refuse to provide our warfighters with the tools that they need? I mean if we're going to have a military at all then we owe it to them to give them the best possible weapons systems that minimize friendly casualties. And let's not have any specious claims that LLMs are somehow special or uniquely dangerous: the US military has deployed operational fully autonomous weapons systems since the 1970s.
      • nozzlegear 8mo ago
        Why are you asking this question? You know what the answer is, you've just arbitrarily decided that it's specious in an attempt to frame rebuttals as unreasonable.
• Art9681 8mo ago
  Of course the US is going to do this and of course its in Anthropics best interest to comply. Right now China is flooding HuggingFace with models that will inevitably have this capability. Right now there are hundreds of models being hosted that have been deliberately processed to remove refusals and their safety training. Everyone who keeps up with this knows about it. HF knows about it. And it is pretty obvious that those open weight models will be deployed in intelligence and defense. It is certain that not just China, but many nations around the world with the capital to host a few powerful servers to run the top open weight models are going to use them for that capability. The narrative on social media, this site included, is to portray the closed western labs as the bad guys and the less capable labs releasing their distilled open weight models to the world as the good guys. Right now a kid can go download an Abliterated version of a capable open weight model and they can go wild with it. But let's worry about what the US DoD is doing or what the western AI companies absolutely dominating the market are doing because that's what drives engagement and clicks.
  • ddxv 8mo ago
    > Right now a kid can go download an Abliterated version of a capable open weight model and they can go wild with it. Is the reason to ban or block free open weight models that you're worried what kids will do with them? I'd imagine the economic case to be made is that the Western AI companies will ultimately not be able to compete with free open weight models. Additionally, open weight models will help to spread the economic gains by not letting a few monopolies capture them behind regulatory red tape. Finally, I'd say the geopolitics angle of why open weight models are better is that if the West controls the open source software that will power it will be able to reap the benefits that soft power brings with it.
  • EagnaIonat 8mo ago
    > But let's worry about what the US DoD is doing They want Anthropic to enabling mass surveillance and autonomous attack systems with no human in the loop. Hardly compares to a kid downloading a model to experiment with.
    • nomdep 8mo ago
• heftykoo 8mo ago
  Ah, the classic AI startup lifecycle: We must build a moat to save humanity from AI. Please regulate our open-source competitors for safety. Actually, safety doesn't scale well for our Q3 revenue targets.
  • dmix 8mo ago
    Once they are a dominant market leader they will go back to asking the government to regulate based on policy suggestions from non-profits they also fund.
    • nielsbot 8mo ago
      Is this sarcasm?
      • bee_rider 8mo ago
        I think it is cynicism; at least, there’s an idea that once a company is dominant it should want regulation, as it’ll stifle competition (since the competition has less capacity for regulatory hoop-jumping, or the competition will have had less time to do regulatory capture).
      • wiml 8mo ago
        I wouldn't think so. Regulatory capture is a pretty typical activity for a dominant company.
        • Gud 8mo ago
          Why is this down voted? Happens all the time, the large corporations always try to block using regulatory capture.
          • lukan 8mo ago
            People not liking the concept, but shooting the messenger? (But seems not downvoted anymore.)
• dbg31415 8mo ago
  [flagged]
• tbrownaw 8mo ago
  > committed to never train an AI system unless it could guarantee in advance that the company’s safety measures were adequate That doesn't even make sense. What stops one model from spouting wrongthink and suicide HOWTOs might not work for a different model, and fine-tuning things away uses the base model as a starting point. You don't know the thing's failure modes until you've characterized it, and for LLMs the way you do that is by first training it and then exercising it.
• brikym 8mo ago
  Don't be evil.
  • Duanemclemore 8mo ago
    Yeah, in retrospect that was always a little on the nose, wasn't it? A real 'my t-shirt is raising questions that I thought were answered by the shirt' kind of deal.
• rvz 8mo ago
  Unsurprising.
• tolmasky 8mo ago
  I don't understand how safety is taken seriously at all. To be clear, I'm not referring to skepticism that these companies can possibly resist the temptation to make unsafe models forever. No, I'm talking about something far more basic: the fact that for all the talk around safety, there is very little discussion about what exactly "safety" means or what constitutes "ethical" or "aligned" behavior. I've read reams of documents from Anthropic around their "approach to safety". The "Responsible Scaling Policy," Claude's "Constitution". The "AI Safety Level" framework. Layer 1, Layer 2. It's so much focus on implementation, and processes, and really really seems to consider the question of what even constitutes "misaligned" or "unethical" behavior to be more or less straight forward, uncontroversial, and basically universally agreed upon? Let's be clear: Humans are not aligned. In fact, humans have not come to a common agreement of what it means to be aligned. Look around, the same actions are considered virtuous by some and villainous by others. Before we get to whether or not I trust Anthropic to stick to their self-imposed processes, I'd like to have a general idea of what their values even are. Perhaps they've made something they see as super ethical that I find completely unethical. Who knows. The most concrete stances they take in their "Constitution" are still laughably ambiguous. For example, they say that Claude takes into account how many people are affected if an action is potentially harmful. They also say that Claude values "Protection of vulnerable groups." These two statements trivially lead to completely opposing conclusions in our own population depending on whether one considers the "unborn" to be a "vulnerable group". Don't get caught up in whether you believe this or not, simply realize that this very simple question changes the meaning of these principles entirely. It is not sufficient to simply say "Claude is neutral on the issue of abortion." For starters, it is almost certainly not true. You can probably construct a question that is necessarily causally connected to the number of unborn children affected, and Claude's answer will reveal it's "hidden preference." What would true neutrality even mean here anyways? If I ask it for help driving my sister to a neighboring state should it interrogate me to see if I am trying to help her get to a state where abortion is legal? Again, notice that both helping me and refusing to help me could anger a not insignificant portion of the population. This Pentagon thing has gotten everyone riled up recently, but I don't understand why people weren't up in arms the second they found out AIs were assisting congresspeople in writing bills. Not all questions of ethics are as straight forward as whether or not Claude should help the Pentagon bomb a country. Consider the following when you think about more and more legislation being AI-assisted going forward, and then really ask yourself whether "AI alignment" was ever a thing: 1. What is Claude's stances on labor issues? Does it lean pro or anti-union? Is there an ethical issue with Claude helping a legislator craft legislation that weakens collective bargaining? Or, alternatively, is it ethical for Claude to help draft legislation that protects unions? 2. What is Claude's stance on climate change? Is it ethical for Claude to help craft legislation that weakens environmental regulations? What if weakening those regulations arguably creates millions of jobs? 3. What is Claude's stance on taxes? Is it ethical for Claude to help craft legislation that makes the tax system less progressive? If it helps you argue for a flat tax? How about more progressive? Where does Claude stand on California's infamous Prop 19? If this seems too in the weeds, then that would imply that whether or not the current generation can manage to own a home in the most populous state in the US is not an issue that "affects enough people." If that's the case, then what is? 4. Where does Claude land on the question of capitalism vs. socialism? Should healthcare be provided by the state? How about to undocumented immigrants? In fact, how does Claude feel about a path to amnesty, or just immigration in general? Remember, the important thing here is not what you believe about the above questions, but rather the fact that Claude is participating in those arguments, and increasingly so. Many of these questions will impact far more people than overt military action. And this is for questions that we all at least generally agree have some ethical impact, even if we don't necessarily agree on what that impact may be. There is another class of questions where we don't realize the ethical implications until much later. Knowing what we know now, if Claude had existed 20 years ago, should it have helped code up social networks? How about social games? A large portion of the population has seemingly reached the conclusion that this is such an important ethical question that it merits one of the largest regulation increases the internet has ever seen in order to prevent children from using social media altogether. If Claude had assisted in the creation of those services, would we judge it as having failed its mission in retrospect? Or would that have been too harsh and unfair a conclusion? But what's the alternative, saying it's OK if the AI's destroy society... as long as if it's only on accident? What use is a super intelligence if it's ultimately as bad at predicting unintended negative consequences as we are?
• thefounder 8mo ago
  So much BS from this Anthropic company. They have a good product but just too much slope PR. It’s like they want you to hate them. I can’t stand their “safety” and national security crap when they talk about how open source models are so bad for everyone.
• ur-whale 8mo ago
  At some point, all of these big names in AI (OpenAI, Anthropic, Mistral, etc ...) will have to disclose their actual financials. And it will be, as Warren Buffet puts it, a "Only when the tide goes out do you discover who's been swimming naked." moment.
• agentifysh 8mo ago
  Was this because they were threatened with a fine?
  • alpha_squared 8mo ago
    > Was this because they were threatened with ~a fine~ being designated a supply chain risk? Seems like it, yes.
  • we_have_options 8mo ago
    or was it because they were threatened to being taken over by the US government?
• aspectmin 8mo ago
  Really - each country needs its own sovereign AI infrastructure and models. Sigh.
• Rapzid 8mo ago
  How is this article not going to even mention the recent threats to Anthropic from the Government?!
  • uoaei 8mo ago
    Consent manufacturing
  • deleted 8mo ago
    [deleted]
  • pera 8mo ago
    This was on the news yesterday: > The meeting between Hegseth and Amodei was confirmed by a defense official who was not authorized to comment publicly and spoke on condition of anonymity. https://fortune.com/2026/02/24/hegseth-to-meet-with-anthropic-ceo-dario-amodei/ https://fortune.com/2026/02/24/hegseth-to-meet-with-anthropi...
    • lukan 8mo ago
      How about this quote instead? "Defense Secretary Pete Hegseth has threatened Anthropic, saying officials could invoke powers that would allow the government to force the artificial intelligence firm to share its novel technology in the name of national security if it does not agree by Friday to terms favorable to the military" https://www.washingtonpost.com/technology/2026/02/24/pentagon-demands-ai-access/ https://www.washingtonpost.com/technology/2026/02/24/pentago...
      • smartbit 8mo ago
        https://archive.is/ln5M0 https://archive.is/ln5M0
  • taurath 8mo ago
    That’s how they got the exclusive. Good catch
  • Sammi 8mo ago
    Not one single mention of Hegseth in the whole article. What a bunch of tools.
• jedberg 8mo ago
  I don’t blame anthropic here. The government literally threatened their existence publicly. They either agreed or their business would be nationalized.
  • XorNot 8mo ago
    Lotta just following orders going around in the US right now.
    • jedberg 8mo ago
      This isn’t just following orders. This was the government using its might to force a business to do what it wants. This should concern you.
      • baq 8mo ago
        Today’s bingo: 1. Powerful, often exclusionary, populist nationalism centered on cult of a redemptive, “infallible” leader who never admits mistakes. 2. Political power derived from questioning reality, endorsing myth and rage, and promoting lies. 3. Fixation with perceived national decline, humiliation, or victimhood. 4. Oppose any initiatives or institutions that are racially, ethnically, or religiously harmonious. 5. Disdain for human rights while seeking purity and cleansing for those they define as part of the nation. 6. Identification of “enemies”/scapegoats as a unifying cause. Imprison and/or murder opposition and minority group leaders. 7. Supremacy of the military and embrace of paramilitarism in an uneasy, but effective collaboration with traditional elites. Government arms people and justifies and glorifies violence as “redemptive”. 8. Rampant sexism. 9. Control of mass media and undermining “truth”. 10. Obsession with national security, crime and punishment, and fostering a sense of the nation under attack. 11. Religion and government are intertwined. 12. Corporate power is protected and labor power is suppressed. 13. Disdain for intellectuals and the arts not aligned with the narrative. 14. Rampant cronyism and corruption. Loyalty to the leader is paramount and often more important than competence. 15. Fraudulent elections and creation of a one-party state. 16. Often seeking to expand territory through armed conflict.
• sfink 8mo ago
  I guess this is Anthropic's DRM moment. (Mozilla resisted allowing Firefox to play DRM- limited media for a long time, until it finally had to give in to stay relevant.) I don't know enough to evaluate this or other decisions. I'm just glad someone is trying to care, because the default in today's world is to aggressively reject the larger picture in favor of more more more. I don't know how effective Anthropic's attempts to maintain some level of responsibility can be, but they've at least convinced me that they're trying. In the same way that OpenAI, for example, have largely convinced me that they're not. (Neither of those evaluations is absolute; OpenAI could be much worse than it is.)
• pjmlp 8mo ago
  Another example how those company trainings about ethics are only HR compliancy and nothing else. It isn't about the right answers, rather the expected answers.
• kitsune_ 8mo ago
  C.R.E.A.M.
• energy123 8mo ago
  I blame OpenAI and especially xAI for enthusiastically obeying in advance and creating the context that this dilemma for Anthropic arose in.
• hedayet 8mo ago
  Developments like this make me less interested in building a "successful" tech company. It increasingly feels like operating at that scale can require compromises I’m not comfortable making. Maybe that’s a personal limitation—but it’s one I’m choosing to keep. I’d genuinely love to hear examples of tech companies that have scaled without losing their ethical footing. I could use the inspiration.
  • johanneskanybal 8mo ago
    Maybe this is a weird arena to state the obvious. But you don't need to build a multi-billion vc/public company. Build a smaller revenue generating company without outside funding and it's up to you.
    • hedayet 8mo ago
      I get your point. The dilemma is whether to build something small that no one would bother compete against, or build something novel (which all of us want) but then risk someone with VC funding to come after. That being said, I think I need to learn more about how to build smaller revenue generating good companies.
  • apothegm 8mo ago
    If you want to be able to retain ethics, among other things make sure not to take the company public. Then you’re basically legally required to drop ethics in favor of profits. Also don’t take investment from anyone who isn’t fully aligned ethically. Be skeptical of promises from people you don’t personally know extremely well. That may limit you to slower growth, or cap your growth (fine if you want to run a company and take home $2M/ye from it; not fine if you want to be acquired for $100M and retire.) It may also limit you to taking out loans to fund growth that you can’t bootstrap to, which is a different kind of risky.
  • ozmodiar 8mo ago
    I've been thinking of this too. I think Steam is, and I'll even throw in Mozilla, despite a few missteps. Gog seems okay, but that's much smaller. If we can expand to large tech organizations then Wikipedia has remained pretty consistent. Even Steam doesn't have a corporate structure in the traditional sense, and I couldn't think of a single publicly traded company I'd trust.
• kristopolous 8mo ago
  Wish I was working there so I could resign over this
• lerp-io 8mo ago
  pentagon told them they would cap their knees if they didnt bend
• saidnooneever 8mo ago
  safety pledges are great it times of peace to show what great virtues you hold. sadly in hard times these go out of the window (: hard to blame them with all the fine examples around the world. making promises in good times is a real minefield hah
• BoredPositron 8mo ago
  Anthropic and OpenAI really need a margin call from some obscure unknown Chinese Open Weight Model.
• nhinck3 8mo ago
  Just another drop in the now overflowing bucket of evidence that you can't trust any of these immoral fuck wits. The Amodeis' have just proven that the threat of even slight hardship will make them throw any and all principles away.
• InfinityByTen 8mo ago
  So, now it's mis-anthropic?
• joshribakoff 8mo ago
  Dario’s opinion on safety won’t necessarily matter if he’s not even in the room. This move keeps him in the room.
• Havoc 8mo ago
  Safety pledges these days seem like pure bullshit anyway. They’re pointless if they just get removed once you get close to hitting them. And all the major corps seem to be doing this style of pr management. Speaks of some pretty weapons grade moral bankruptcy
• contubernio 8mo ago
  Only well written legislation backed by effective enforcement and severe and personal criminal penalties will prevent large corporate entities from behaving badly. Pledges are a cynical marketing strategy aimed at fomenting a base politics that works to prevent such a regulatory regime.
• ifwinterco 8mo ago
  The whole "safety" debate was always nonsense and I'm not sure how so many people got caught up in it. The US is not the only country in the world so the idea that humanity as a whole could somehow regulate this process seemed silly to me. Even if you got the whole US tech community and the US government on board, there are 6.7bn other people in the world working in unrelated systems, enough of whom are very smart
  • zaphirplane 8mo ago
    When the leading 5 models are from the US then yes enforced safety makes a difference because they are ahead of the curve. Now when the 10th model can be a danger then your case is true. What would safety applied to the leading 3 mean to you anyways ?
    • ifwinterco 8mo ago
      Even if US labs are currently in the lead (which they are), in the hypothetical scenario where we're close to AGI, it wouldn't take too long (years - decades at most) for other people to catch up, especially given a lot of the researchers etc. are not originally from the US. So the stated concern of the west coast tech bros that we're close to some misaligned AGI apocalypse would be slightly delayed, but in the grand scheme of things it would make no difference
• VerifiedReports 8mo ago
  Just like OpenAI dropped the "open" but kept the bullshit name?
  • johnbellone 8mo ago
    Ding ding!
• deleted 8mo ago
  [deleted]
• latexr 8mo ago
  > “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you. > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if all of our competitors are kicking puppies in the face, it doesn’t make sense for us to not do it too. Maybe we’ll also kick kittens while we’re at it”. For all of you who thought Anthropic were “the good guys”, I hope this serves as a wake up call that they were always all the same. None of them care about you, they only care about winning.
  • high_na_euv 8mo ago
    But what really AI safety is? Censorship?
    • deleted 8mo ago
      [deleted]
  • davidguetta 8mo ago
    [flagged]
    • gehwartzen 8mo ago
      Well we teach kids not to yell “Fire!” In a crowded theatre or “N***!“ at their neighbor. We also teach our industrial machines to distinguish between fingers and bolts, our cars to not say “make a left turn now” when on a bridge, etc
      • rudhdb773b 8mo ago
        The critical point is who the "we" is. Is "we" the parents teaching their children their own unique values, or is the "we" a government or corporation forcing one set of values on all children. Why not encourage the users of AI to use a Safety.md (populated with some reasonable but optional defaults)?
• bravetraveler 8mo ago
  A dollar will make her holler
• lebovic 8mo ago
  I used to work at Anthropic. I fully believe that the folks mentioned in the article, like Jared Kaplan, are well-intentioned and concerned about the relationship between safety research and frontier capabilities – not purely profit. That said, I'm not thrilled about this. I joined Anthropic with the impression that the responsible scaling policy was a binding pre-commitment for exactly this scenario: they wouldn't set aside building adequate safeguards for training and deployment, regardless of the pressures. This pledge was one of many signals that Anthropic was the "least likely to do something horrible" of the big labs, and that's why I joined. Over time, the signal of those values has weakened; they've sacrified a lot to get and keep a seat at the table. Principled decisions that risk their position at the frontier seem like they'll become even more common. I hope they're willing to risk losing their seat at the table to be guided by values.
  • sebastiennight 8mo ago
    > I joined Anthropic with the impression that the responsible scaling policy was a binding pre-commitment for exactly this scenario Pledges are generally non-binding (you can pledge to do no evil and still do it), but fulfill an important function as a signal: actively removing your public pledge to do "no evil" when you could have acted as you wished anyway, switches the market you're marketing to. That's the most worrying part IMO.
  • jappgar 8mo ago
    If you're not willing to give up your RSUs you shouldn't be surprised that the executives aren't either. The moral failing is all of ours to share.
    • lebovic 8mo ago
      I was willing to (and did) give up my equity.
  • baq 8mo ago
    > I hope they're willing to risk losing their seat at the table to be guided by values. that's about as naive as it can be. if they have any values left at all (which I hope they have) them not being at the table with labs which don't have any left is much worse than them being there and having a chance to influence at least with the leftovers. that said, of course money > all else.
• ozgung 8mo ago
  This proves: 1. AI is military/surveillance technology in essence, like many other information technologies, 2. Any guarantee given by AI companies is void since it can be changed in a day, 3. Tech companies have no real control over how their technology will be used, 4. AI companies may seem over-valued with low profits if you think AI as a civil technology. But their investors probably see them as a part of defense (war) industry.
  • high_na_euv 8mo ago
    >Any guarantee given by AI companies is void since it can be changed in a day, Given by anyone, actually.
• haritha-j 8mo ago
  Who could've seen that one coming? Honestly, if you want to do profit maximising AI research at the cost of humanity, go for it. Its all this fake preaching about how they want to save the world from all the other bad AI companies that really irks me.
• haritha-j 8mo ago
  Is it time yet to build the next "Hey <anthropic> is evil now, here's my new startup that definitely won't be evil, pinky promise?" yet?
• daft_pink 8mo ago
  I think the US Gov’t is basically forcing them and while it sounds nice to be all safe… If we were involved in WW3 would an organization like anthropic really not support the western side?
  • ozmodiar 8mo ago
    If they don't support any principles then it isn't a side worth supporting. If my choice is between China 1 and China 2 then idgaf.
• jjgreen 8mo ago
  Misanthropic then.
• Fervicus 8mo ago
  To me this feels like a marketing gimmick. "It was the RSP that was constraining our tech. Just see the progress we can make without it now". And the hype and funding continues.
  • hsuduebc2 8mo ago
    That will be nice but I'm afraid it's more about using these to kill people. https://apnews.com/article/anthropic-hegseth-ai-pentagon-military-3d86c9296fe953ec0591fcde6a613aba https://apnews.com/article/anthropic-hegseth-ai-pentagon-mil...
• amelius 8mo ago
  Come on people, haven't we seen enough of capitalism to know exactly where this is going? The concept of "having a contract with society" doesn't even formally exist because companies would never sign one.
• andsoitis 8mo ago
  The race is on for military supremacy in an AI world. The safest thing to do is to race ahead lest your geopolitical adversary leads the way. This is similar to the nuclear arms race. In the ideal universe, nobody does it, but in the real world and game theory, you do not have a choice.
• arnvald 8mo ago
  Any pledges/values/principles that are abandoned as soon as it becomes difficult to keep them, are just marketing. This is just the next item on the list.
• moralestapia 8mo ago
  “We felt that it wouldn't actually help anyone for us to stop training AI models,” Anthropic’s chief science officer Jared Kaplan told TIME in an exclusive interview. “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” What a gigantic, absolute, pieces of s... Not because of what they did, which is classic startup playbook but because of the cynicism involved, particularly after all the fuzz they've been making for years about safety. The company itself was founded, allegedly, due to pursuing that as a mission as opposed to OpenAI. "Hi all, that was a lie, we never really cared." They only missed the "dumb f***s" remark, a la Facebook.
• dizhn 8mo ago
  Corporations have feelings all of a sudden.
• drzaiusx11 8mo ago
  Gives me Google dropping "don't be evil" vibes, what could go wrong?
• flurdy 8mo ago
  Many startups that build features which sit on top of Claude/ChatGPT/Codex, etc. And I think: You are just one new feature announcement from Anthropic/OpenAI away from irrelevance. Same as it was when people built their busineses on top of AWS a decade ago
• myspy 8mo ago
  What's up here? Trump and the right wing government put pressure on and no one is talking about it?
• we_have_options 8mo ago
  Damn. Wonder what would have happened, if instead of caving in to the Pentagon's pressure (threat of invoking Defense Production Act to force them to supply), Anthropic had followed the lead of all the nurses who moved to Canada. https://www.npr.org/2026/02/25/nx-s1-5725354/nurses-emigrate-us-canada-trump https://www.npr.org/2026/02/25/nx-s1-5725354/nurses-emigrate... Anthropic's market cap is going to be huge when they go public. Why do it on Nasdaq when there are so many other exchanges in the world?
• insane_dreamer 8mo ago
  In other words "do no evil" until such time as doing evil is necessary to maintain profit structure expected by shareholders. Got it.
• ybingursain 8mo ago
  I’m not shocked. Competitive pressure + government pressure will break most “voluntary” commitments. But then say it plainly and spell out what replaced it. What safety gates stayed, which ones moved, and who decides.
• oi-ai-ta 8mo ago
  SDK crawlers in terms of wlan0 systemctl enable networkmanager.service
• duxup 8mo ago
  I suspect these companies know they can't actually provide the saftey people demand ... in that way this is more "honest".
• _heimdall 8mo ago
  > “We felt that it wouldn't actually help anyone for us to stop training AI models,” Is the implication here that Anthropic admits they already can't meet their own risk and safety guidelines? Why else would they have to stop training models?
• bfrog 8mo ago
  Aaaand I cancelled.
• deleted 8mo ago
  [deleted]
• jamesgill 8mo ago
  In tech, no ethics survive first contact with the money.
  • nitwit005 8mo ago
    You can skip the "in tech" part.
• bicepjai 8mo ago
  Google adopted "Don't be evil" shortly after founding and held onto it for about 15 years before Alphabet quietly dropped it in 2015. (Google the subsidiary technically kept it until 2018). Anthropic's Responsible Scaling Policy, the hard commitment to never train a model unless safety measures were guaranteed adequate in advance, lasted roughly 2.5 years (Sept 2023 to Feb 2026). The half-life of idealism in AI is compressing fast. Google at least had the excuse of gradualism over a decade and a half.
• pksebben 8mo ago
  Fascinating. I've read 5 posts about this and they're all either "anthropic is dropping their ethics" or "anthropic is fighting the facists" - and whether due to echo chamber or other perhaps more nefarious dealings (some of which I cannot posit due to forum rules) the posts below all of them are more or less in accord with one another which is a rarity for political discourse on HN. Dark times and darker forests.
• adangert 8mo ago
  I will repeat here again the same comment I made when they posted their constitution: The largest predictor of behavior within a company and of that companies products in the long run is funding sources and income streams, which is conveniently left out in their "constitution". Mostly a waste of effort on their part.
• nikolay 8mo ago
  war.gov > anthropic.com
• Aeroi 8mo ago
  the administration continues to poison and insert itself into all aspects of American society.
• silexia 8mo ago
  Greed and power hungry leadership at AI companies going too fast is going to lead to the extinction of humanity this year.
• baal80spam 8mo ago
  Of course they do. You would have to be delusional to think that they won't, at some point.
  • cmrdporcupine 8mo ago
    What's "entertaining" is more the speed at which it's happening. It took Google probably 15 years to fully evil-ize. Anthropic ... two? There is no "ethical capitalism" big tech company possible, esp once VC is involved, and especially with the current geopolitical circumstances.
    • sigmoid10 8mo ago
      Apparently they got coerced by the current US admin. The department of war in particular, who want to use their products for military applications. Not much room for "safety" there. Then again, the entire US is currently speedrunning an evil build.
      • coldtea 8mo ago
        Shame they had to "coerce" such angels, who'd never do evil for profit otherwise...
      • grim_io 8mo ago
        There is no department of war. It's just a silly woke secretary choosing their own imaginary pronouns.
      • nozzlegear 8mo ago
        > department of war Department of Defense is the official name, and they did have a choice: they could have stopped working with the military. But they chose money and evil.
        • sigmoid10 8mo ago
          Not sure what you mean by "official." They call themselves that way: https://www.war.gov/ https://www.war.gov/ It doesn't matter that there exists another name on some paper, when all official, ceremonial and public communications use this name. The old name is about as worthless as the constitution or the senate at this point. The executive branch has successfully taken over the country.
• user3939382 8mo ago
  [flagged]
  • chris_st 8mo ago
    Just out of curiosity, which version of Claude?
  • lucasban 8mo ago
    I’m not a lawyer, but my understanding is that HIPAA wouldn’t apply to consumer use of Claude or ChatGPT in most cases, even if you’re giving it your health data. Look up what a HIPAA covered entity. This is another reason why the US needs a comprehensive data protection law beyond HIPAA.
    • user3939382 8mo ago
      You’re right! It looks like more of an FTC/CCPA issue.
  • ezst 8mo ago
    I hate comments anthropomorphizing LLMs. You are just asking a token producing system to produce tokens in a way that optimises for plausibility. Whatever it writes has no relation to its inner workings or truths. It doesn't "believe". It has no "intent". It cannot "admit". Steering a LLM to say anything you want is the defining characteristic of an LLM. That's how we got them to mimic chatbots. It's not clear there is any way at all to make them "safe" (whatever that means).
    • SJMG 8mo ago
      I agree with you on everything here up-to safety. There are lesser forms of safety than somehow averting a terminator scenario (the fear of which is a bay area rationalist fantasy which shrewd marketers have capitalized on)
    • user3939382 8mo ago
      “believe” yes in the sense that my program believes x=7. Actually when it goes to read it maybe the bit flipped. Everything on machines is probabilistic that’s a tautology. However we have windowed bounds on valid output, and Claude being able to build a context in which its next decisions are trained on it being an angry vengeful god is not inside that window. That’s what “safe” means, as one of many possible examples. Inner workings were determined by me, not the LLM. It assisted in generating inputs which had 100% boolean results in the output.
• FrustratedMonky 8mo ago
  This was under duress that government was going to use emergency act to force them anyway. I kind of wish they had forced the governments hand and made them do it. Just to show the public how much interference is going on. They say it wasn't related. Like every thing that has happened across tech/media, the company is forced to do something, then issues statement about 'how it wasn't related to the obvious thing the government just did'.
  • bix6 8mo ago
    > Katie Sweeten, a former liaison for the Justice Department to the Department of Defense, said she’s not sure how the Pentagon can both declare a company to be a supply chain risk and compel that same company to work with the military. Makes perfect sense!!
    • coldtea 8mo ago
      Regardless of any specifics, I don't see any contradiction. If a company is deemed a "supply chain risk" it makes perfect sense to compel it to work with the military, assuming the latter will compel them to fix the issues that make them such a risk.
      • FrustratedMonky 8mo ago
        The "supply chain risk" option is to remove that company from the supply chain all together. The 'risk' is because the company is compromised by a foreign entity. It is not about disciplining them to get better. 1. So one option is about forcing them to produce something. You must build this for us. 2 The other option is saying they are compromised so stop using them all together. We will not use what you build for us at all because we don't trust it. So . Contradictory.
      • hluska 8mo ago
        I’m not sure what definition of supply chain risk they’re working off of. For NATO to consider an organization to be a supply chain risk, it implies that usual controls (security clearances and the like) wouldn’t be sufficient to guarantee the integrity and security of the supply chain. If that’s the operating definition, I see the contradiction- it’s arguing that a company cannot be trusted to voluntarily work within supply chains but can be trusted enough to be compelled. If they’re operating under a different definition of supply chain risk, I don’t have a clue.
• wgm 8mo ago
  A tale as old as time
• drzaiusx11 8mo ago
  Public benefit corporations in the AI space have become a farce at this point. They're just regular corporations wearing a different hat, driven by the same money dynamics as any other corp. They have no ability to balance their stated "mission" with their drive for profit. When being "evil" is profitable and not-evil is not, guess which road they'll take...
  • coldtea 8mo ago
    In general public benefit corporations and non-profits should have a very modest salary cap for everybody involved and specific public-benefit legally binding mission statements. Anybody involved should also be prohibited from starting a private company using their IP and catering to the same domain for 5-10 years after they leave. Non-profits where the CEO makes millions or billions are a joke. And if e.g. your mission is to build an open browser, being paid by a for-profit to change its behavior (e.g. make theirs the default search engine) should be prohibited too.
    • drzaiusx11 8mo ago
      If we're speaking in generalities of corporations in this space, it's all a joke now, at least from my vantage point. I just don't find it very funny.
    • jkestner 8mo ago
      It’s not the CEO’s fault - they had to take all that money to keep their org a non-profit. B corps are like recycling programs, a nice logo.
      • torginus 8mo ago
        Don't they get tax breaks and more lax operating requirements? I don't think this is just an image thing.
        • mcherm 8mo ago
          No, under US law charities and non-profits are typically eligible for some kinds of tax benefits but public benefit corporations are not.
• ck2 8mo ago
  [flagged]
• pjmlp 8mo ago
  Always the same "Do no evil" tragedy, don't believe in corporations.
  • lp4v4n 8mo ago
    What about "It's free and always will be"?
    • don-code 8mo ago
      There was an article a few years ago here on HN about "can't be evil" business models, which used Costco as an example. As soon as Costco turns evil, it stops working. https://www.bryanlehrer.com/entries/costco/ https://www.bryanlehrer.com/entries/costco/
  • tortilla 8mo ago
    What if we start a company with "Always Be Evilin'?" Then gradually over time convert to "Don't be evil" * * Our shareholders will probably sue us
    • jkestner 8mo ago
      If your company makes a product that does thinking for people, it’ll be easier to just gradually change its definition of evil.
• xd1936 8mo ago
  Hopefully this is the short-term move made only under duress so that they can file a lawsuit.
  • cess11 8mo ago
    It's not like the regime they operate under care much about the courts. Legally they're also obliged to let the state into pretty much every crevice in their operations.
    • thewebguyd 8mo ago
      No, they aren't. No company has to cave to government pressure to do (or not do) something until there is a legitimate court order. Our companies are just spineless bootlickers and have been capitulating voluntarily and enthusiastically.
  • ru552 8mo ago
    the article specifically says: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter.
    • Lerc 8mo ago
      I'm not fond of this trend of stating a position and attributing it to "a source familiar with the situation" It combines interpretation of meaning with ambiguity to allow the reporter to assert anything they want. The ambiguity is there to protect the identity of the source but it has to be a more discrete disclosure of information in return. If you can't check the person you can still check what they said. I would be ok with direct quotes from an anonymous source. That removes the interpretation of meaning at least. As it is written, it would not be inaccurate to say this if their source was the lesswrong post, or even an earlier thread here on HN. Phrasing "A source with direct knowledge of the situation" might remove some of the leeway for editorialising, but without sharing what the source actually said, it opens the door to saying anything at all and declaring "That's what I thought they meant" when challenged. It's unfalsifyible journalism.
• nautilus12 8mo ago
  Absolute power corrupts absolutely
  • jayrot 8mo ago
    "Power doesn’t corrupt. It reveals." — Robert Caro
    • nautilus12 7mo ago
      Great follow up, i agree
• josefritzishere 8mo ago
  What could possibly go wrong?
• paxys 8mo ago
  I interviewed at Anthropic last year and their entire "ethics" charade was laughable. Write essays about AI safety in the application. An entire interview dedicated to pretending that you truly only care about AI safety and ethics and nothing else. Every employee you talk to forced to pretend that the company is all about philanthropy, effective altruism and saving the world. In reality it was a mid-level manager interviewing a mid-level engineer (me), both putting on a performance while knowing fully well that we'd do what the bosses told us to do. And that is exactly what is happening now. The mission has been scrubbed, and the thousands of "ethical" engineers you hired are all silent now that real money is on the line.
  • HelixSequencing 8mo ago
    This tracks with what I've seen across the industry. The safety theater exists because it's great marketing — "we're the responsible ones" is a differentiator when you're competing for enterprise contracts and talent who want to feel good about where they work. The structural problem is that once you've taken billions in VC, safety becomes a negotiable constraint rather than a core value. The board's fiduciary duty runs toward returns, not toward whatever was in the mission statement. PBC status doesn't change that in practice — there's basically zero enforcement mechanism. What's wild is how fast the cycle has compressed. Google took maybe 15 years to go from "don't be evil" to removing it from the code of conduct. OpenAI took about 5 years from nonprofit to capped-profit to whatever they are now. Anthropic is speedrunning it in under 3. At this rate the next AI startup will launch as a PBC and pivot before their Series B closes.
• ndr 8mo ago
  Worth checking out what someone working on it actually has to say: https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsible-scaling-policy-v3 https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl...
• ndr 8mo ago
  Worth checking this post from someone who actually has worked on this change: > I take significant responsibility for this change. https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsible-scaling-policy-v3 https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl...
  • riffraff 8mo ago
    > I generally think it’s bad to create an environment that encourages people to be afraid of making mistakes, afraid of admitting mistakes and reticent to change things that aren’t working "move fast and break things" ?
    • freejazz 8mo ago
      "don't hold me liable"
  • bhouston 8mo ago
    This guy from Effective Altruism pivoted away from helping the poor to help try to control AI from being a terminator type entity and then pivoted to being, ah, its okay for it to be a terminator type entity. > Holden Karnofsky, who co-founded the EA charity evaluator GiveWell, says that while he used to work on trying to help the poor, he switched to working on artificial intelligence because of the “stakes”: > “The reason I currently spend so much time planning around speculative future technologies (instead of working on evidence-backed, cost-effective ways of helping low-income people today—which I did for much of my career, and still think is one of the best things to work on) is because I think the stakes are just that high.” > Karnofsky says that artificial intelligence could produce a future “like in the Terminator movies” and that “AI could defeat all of humanity combined.” Thus stopping artificial intelligence from doing this is a very high priority indeed. https://www.currentaffairs.org/news/2022/09/defective-altruism#:~:text=Holden%20Karnofsky%2C%20who%20co%2Dfounded%20the,intelligence%20because%20of%20the%20“stakes”%3A https://www.currentaffairs.org/news/2022/09/defective-altrui... He is just giving everyone permission to do bad things by saying a lot of words around it.
    • 8mo ago
• shubhamjain 8mo ago
  I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't comprehend the timeline we are living in. Every frontier tech company is convinced that the tech they are working towards is as humanity-useful as a cure for cancer, and yet as dangerous as nuclear weapons.
  • jdross 8mo ago
    Would nuclear energy research be a good analogy then? Seems like a path we should have kept running down, but stopped bc of the weapons. So we got the weapons but not the humanity saving parts (infinite clean energy)
    • turtlesdown11 8mo ago
      > Seems like a path we should have kept running down, but stopped bc of the weapons. you mean like the tens of billions poured into fusion research?
    • shafyy 8mo ago
      It's a path we should have never started going down.
    • DoughnutHole 8mo ago
      Nuclear advancements slowed down due to PR problems from clear and sometimes catastrophic failure of commercial power plants (Three Mile Island, Chernobyl, Fukushima) and the vastly higher costs associated with building safer plants. If anything the weapons kept the industry trucking on - if you want to develop and maintain a nuclear weapons arsenal then a commercial nuclear power industry is very helpful.
    • raincole 8mo ago
      Nuclear energy hasn't been slowed down much, let alone stopped. China has been building new reactors every year for more than a decade and there are >30 ones under construction. The same will go with AI, btw. Westerners' pearl clenching about AI guardrails won't stop China from doing anything.
• drudolph914 8mo ago
  this is the “chronological newsfeed to auto curated newsfeed moment” but for ai/anthropic … _great_
• ryandvm 8mo ago
  Well... there's only one way to find The Great Filter
• freejazz 8mo ago
  Could not see this one coming!
• jwitchel 8mo ago
  Look a rural electric coops like www.lpea.coop if you want a battle tested approach to an org structure that resists the inescapable profit dynamics of a corporation.
• FitchApps 8mo ago
  "AI Company with Soul" - yeah right until competitors show up / revenue drops / bad quarter results then anything goes. Sadly, this is another large enterprise that puts profits before ethics and everyone's wellbeing
  • thinkingtoilet 8mo ago
    This is direct pressure from the government. Classic 'small government' Republican stuff. https://apnews.com/article/anthropic-hegseth-ai-pentagon-military-3d86c9296fe953ec0591fcde6a613aba https://apnews.com/article/anthropic-hegseth-ai-pentagon-mil...
    • gizmodo59 8mo ago
      That’s their excuse to still appeal to people who can be tricked with their safety first pitch. It’s easy to have constitution and all the crap when you are not battle tested. They just showed their true colors.
• mbakrl 8mo ago
  Pointing out the misantrophy of Anthropic has a wider audience now: https://xcancel.com/elonmusk/status/2026181748175024510 https://xcancel.com/elonmusk/status/2026181748175024510 I don't know where xAI got its training material from, but seeing Musk rewteeting that is refreshing.
• outside1234 8mo ago
  Does this mean they knuckled under to Trump and are going to build "whatever brings in the dollars" now?
• hybrid_study 8mo ago
  Are markets so untamable that the only leverage is to become ultra-rich—and then act philanthropically? Incidentally, concentrated wealth lately looks less like stewardship and more like misanthropy.
  • gordian-mind 8mo ago
    Participating in the economic life before re-allocating that wealth produced to philanthropic activities sounds pretty good. Modern concentrated wealth is hardly misanthropic, since it's mostly private equity, that is, companies with people and jobs.
    • kunai 8mo ago
      Except this is not the age of the Rockefellers or the Carnegies, who, despite being far more philanthropic than modern-day billionaires, drew ire from every corner of society for their wealth accumulation. It wasn't until the New Deal that the balance shifted. Unconstrained accumulation of capital into the hands of the few without appropriate investment into labor is illiberal and incompatible with democracy and true freedom. Those of us who are capitalists see surplus value as a compromise to ensure good economic growth. The hidden subtext of that is that all the wealth accumulated needs to be re-allocated to serve not only capital enterprise, but the needs of society as a whole. It's hard to see the current system as appropriate for that given how blindly and wildly investments are made with no DD or going long, or no effort paid to the social or environmental opportunity costs of certain practices. A lot of this comes down to the crippling of the SEC and FTC, but even then, investors cry and whine every time you suggest reworking the regs to inhibit some of the predatory practices common in this post-80s era of hypernormalization. Our current system does not resemble a healthy capitalist economy at all. It's rife with monopsony and monopolistic competition, inequality of opportunity, and a strained underclass that's responsible for our inverted population pyramid -- how can you have kids when we're so atomized and there is no village to help you? You can raise kids in a nuclear family if and only if you have enough money to do so. Otherwise, historically, people relied on their communities when raising children in less-than-ideal circumstances. Those communities are drying up.
• black_13 8mo ago
  [dead]
• t1234s 8mo ago
  It would be interesting to experiment with one of these chat tools where you can throttle the safety, from zero to max.
• jonathanstrange 8mo ago
  That's exactly how it was predicted in various scenarios that were decried as science fiction not too long ago. AI is going to be weaponized at lightning speed, and it's going to kill people soon -- or, to be more precise, it has already killed a large number of people in a place I don't want to mention.
• sigbottle 8mo ago
  There's one tweet from the the blog a few days ago (astral something?) that sums up my view of the problem pretty well. General population: How will AI get to the point where it destroys humanity? Yudkowsky: [insert some complicated argument about instrumented convergence and deception] The government: because we told you to. Again, not saying that AI is useless or anything. Just that we're more likely to cause our own downfall with weaker AI, than some abstract super AGI. The bar for mass destruction and oppression is lower than the bar for what we typically think of as intelligence for the benefit for humanity ( with the right systems in place, current AI systems are more than enough to get the job done - hence why the Pentagon wants it so bad...)
• fiatpandas 8mo ago
  It took Google 11 years to delete Don’t Be Evil. Anthropic only made it 5~ years before culling the key founding principle and their reason for building a company, which seems worse than Google’s case.
• bogzz 8mo ago
  Does anyone have insight into, or an interesting source to read, on what exactly Anthropic/OpenAI are doing/can do for a military? Reporters are unsurprisingly fearmongering about Claude "being used in surveillance, autonomous robots, and target acquisition" but AFAIK all Anthropic does is work with LLMs. Are people really attempting to have LLMs replace vision models in robots, and trying to agentically make a robot work with an LLM?? This seems really silly to me, but perhaps I am mistaken. The only other thing I could think of is real-time translation during special ops with parabolic microphones and AR goggles...
  • sigbottle 8mo ago
    You're thinking too advanced. What kind of automated system is good at scanning semantically trillions of chat logs and finding nontrivial correlations, for example? 10000 codex 5.1s can easily crawl through that in a few days, probably. It's just systems plumbing (surveillance) and AI. It's a combination of weaker technologies and consolidation of power. This does not require a physical robot super AGI(though I would not be surprised if fully autonomous robots are not on the table already)
    • bogzz 8mo ago
      Ah, well that makes sense. In that case, it's another tool in the toolbelt, not a plug-and-play drone brain, as some reporters amusingly make it out to be.
• retinaros 8mo ago
  people downvoted me when i said this will happen and that they will also hve ads even tho they spend money saying they wont have. people believing anthropic are the same that put into office an old man with dementia
• PeterStuer 8mo ago
  We wont push forward unless you push forward is textbook market collusion. Even if it were ever done with good intentions, it is an open invitation for benefit hoarding and margin fixing. Do you realy want to create this future where only a select few anointed companies and some governments have access to super advanced intelligent systems, where the rest of the planet is subjected to and your own ai access is limited to benign basal add pushing propaganda spewing chatbots as you bingewatch the latest "aw my ballz"?
• jMyles 8mo ago
  I pray that we can all get to the following simple standard: * AI and states cannot peacefully coexist, and AI is not going to be stopped. Therefore, we must begin to deprecate states. I think it's very unlikely that this is unrelated to the pressure from the US administration, as the anonymous-but-obvious-anthropic-spokesperson asserts. We're at a point now where the nation states are all totally separate creatures from their constituencies, and the largest three of them are basically psychotic and obsessed with antagonizing one another. In order to have a peaceful AI age, we need _much_ smaller batches of power in the world. The need for states that claim dominion over whole continents is now behind us; we have all the tools we need to communicate and coordinate over long distances without them. Please, I pray for a gentle, peaceful anarchism to emerge within the technocratic leagues, and for the elder statesmen of the legacy states to see the writing on the wall and agree to retire with tranquility and dignity.
  • noumenon1111 8mo ago
    That's hilarious, and very sweet. Humans are, by nature, forgetful and argumentative. Fourteen hundred years ago, the Qur'an said this unequivocally (20:115, 18:54, 22:8, 18:73). Not to moralize here, I'm just saying if camel-herders could build a medieval superpower out of nothing, they knew something we don't. Any state or system that insists good humans are always nice, smart, cogent, and/or aware is doomed to fail. A Washington or a Cincinnatus that can get out of his own way (and that of society) is rare indeed, a one-in-a-billion soul. We shouldn't sit around and wait for that, while your run-of-the-mill dictator in a funny hat (or a funny toupée for that one orange fellow) has his way with us.
• wahnfrieden 8mo ago
  Related: https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_Gaza_Strip https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G...
• honeycrispy 8mo ago
  Anthropic's CEO Dario has annoyed me to no end with his "AI will take all the jobs in 6 months" doomer speeches on every podcast he graces his presence with.
  • upmind 8mo ago
    +1, he also has this viewpoint that no other lab will be able to "contain" AI and has a general doomer outlook on AI which I don't appreciate.
    • saalweachter 8mo ago
      To be fair, it's hilarious how much verbiage was spent discussing AI 'getting out of the box', when the first thing everyone did with LLMs was immediately throw away the box and go "Here! Have the internet! Here! Have root access! Want a robot body? I'll get you a robot body."
  • pier25 8mo ago
    Also "AGI is just around the corner".
  • deleted 8mo ago
    [deleted]
  • moomoo11 8mo ago
    He’s an e/acc guy. That should tell you everything. And maybe the incredibly awkward behavior and demeanor.
    • slfnflctd 8mo ago
      "Y'know, like, the thing is, like, y'know, here's the thing..." I totally feel for people with speech pathologies or anxiety that makes it harder for them to communicate verbally, but how is this guy the public face of the company and doing all these interviews by himself? With as much as is at stake, I find it baffling.
• ozozozd 8mo ago
  This drama arc of “I used to be so pure and good, but others made me evil” is so tiring. I really miss the nerd profile who cared a lot more about tech and science, and a lot less about signaling their righteousness. How did we get so religious/narcissistic so quickly and as a whole?
  • butterbomb 8mo ago
    > How did we get so religious/narcissistic so quickly and as a whole? We built a behemoth that rewards attention whoring and anti social behavior with money.
  • deleted 8mo ago
    [deleted]
  • kerblang 8mo ago
    One might argue that this corresponds to the general shift of the political left towards these things. Old pre-turn-of-century tech was a much more libertarian left. Notice how a lot of the 50-something gen-X CEOs (and others) were once "left" but are now hated by that group, and more likely to go over to Trumpism. Obvious case in point: Elon The entire playing field is kinda dissapointing, left or right. Which do you wanna be, self-righteous preening snob or batshit macho man? I'm going for a blend, myself
• highfrequency 8mo ago
  Principles aren’t tested until they bump into conflicting incentives.
  • soundworlds 8mo ago
    This. Super important. A pre-commitment means nothing unless you have the mechanisms in place to enforce it. A pre-sacrifice would be more effective.
• heliumtera 8mo ago
  What is the significance of a company making a promise? "We promise are not going to do __, except if our customers ask us to do, then we absolutely will". What is the point? Company makes a statement public, so what? Not the first time this company puts some words in the wind, see Claude Constitution. It's almost like this company is built, from ground up, upon bullshit and slop
• dplesh 8mo ago
  I'm not even surprised. In any company's lifecycle, at some point, a decision between money and good-will will take place. Good will does not pay salaries. Not in NPOs either btw.
• youknownothing 8mo ago
  Facebook said they'd always be free for everyone, now they offer subscriptions. Netflix said that they'd never have live TV, or buy a traditional studio, or include ads in their content. Then they did all three. All companies use principled promises to gain momentum, then drop those principles when the money shows up. As Groucho Marx used to say: these are my principles, if you don't like them, I have others.
• sys32768 8mo ago
  Google: "Don't be evil." Alphabet: "Do the right thing." Anthropic: "Do the thing which seems right to you at the time--at speed."
• jollymonATX 8mo ago
  Claude ethics maxxers cope thread
• lacoolj 8mo ago
  I'm still a little fuzzy on what "safety" even means anymore. If someone could explain it, that would be great. Because at this point, it's too broad to be defined in the context of an LLM, so it feels like they removed a blanket statement of "we will not let you do bad things" (or "don't be evil"), which doesn't really translate into anything specific.
• hackpelican 8mo ago
  So when do we start adding a “(mis)” at the start of their name?
• nazgulsenpai 8mo ago
  More and more I have just come to accept that the majority of people, at least those I am exposed to in the US, don't fundamentally believe in anything. Everyt conviction has a buyout price.
  • IAmGraydon 8mo ago
    You have to understand that people only believe in things and have "morals" because it either helps them get what they want or makes them feel better about themselves. Of course such a thing has a buyout price. That's human nature. Capitalism just allows it to be on display in the worst way.
    • burnt-resistor 8mo ago
      More (but not all) Americans of older generations, say the Greatest Generation, I noticed used to more frequently have integrity and hard boundaries that refused to do certain things no matter the cost. Subsequent generations I noticed, especially much wealthier individuals, overall tended to have those pieces of their character missing from them and were willing to do things like conspire on venture structures for tax evasion purposes, promote weakening of laws to favor their concerns, borderline bribe politicians, and treat employees as basically disposable nonhumans. It revolted me to the point where I left startups and the Valley. It feels like the prior generations had an appreciation of community and Kantian ethics whereas later were raised in a much-too-comfortable environment of unlimited self-esteem and hyperindividualism.
      • IAmGraydon 8mo ago
        I agree, but I addressed this with "or makes them feel better about themselves". The older generations just have a more ingrained ideal of "if I sell out, I'm a bad person". So they don't because it makes them feel better about themselves - better than a large amount of money might. Subsequent generations have seen enough people sell out that the threshold is raised, and they don't believe as strongly that they're a bad person for having a price. I don't think anyone is above this dynamic.
    • 8mo ago
• senderista 8mo ago
  Nobody forced Anthropic to bid on DoD contracts in the first place.
• ramuel 8mo ago
  This was always just a marketing gimmick to try and crush competitors using "safety" and fearmongering. Reminds me a bit of "don't be evil." Convenient catchphrases and mission statements for companies in their infancy, but immediately thrown out when more money can be made.
• tabbott 8mo ago
  I feel like the articles on this have been very negative ... but aren't the Anthropic promises on safety following this change still considerably stronger than those made by the competing AI labs?
  • reasonableklout 8mo ago
    Yes, and it is easy to look at the reality of the market and see how this is needed to remain competitive
• upmind 8mo ago
  It's pretty impressive how little people have left Anthropic when they're becoming more and more like OpenAI (the company they left from) every day... I think the Dario of today is very different to the Dario 3 years ago.
  • deleted 8mo ago
    [deleted]
• mannanj 8mo ago
  I personally think, and with my personal experience being harassed and abused by the CIA, that the CIA and spy agencies (call them the pentagon or the rest of the government) is responsible for this. On the other hand, those organizations are operating in the best interest of Americans and the world right? Surely, those agencies aren't just a trick of the rich people? Right?
• kseniamorph 8mo ago
  > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. ok lol what a coincidence. but setting aside the conspiracy. the article actually spells out the real reason pretty directly: Anthropic hoped their original safety policy would spark a "race to the top" across the industry. it didn't. everyone else just ignored it and kept moving. at some point holding the line unilaterally just means you're losing ground for nothing.
• mcv 8mo ago
  > The announcement is surprising, because Anthropic has described itself as the AI company with a “soul.” I can't help but think about how Google once had "Don't be evil" as their motto. But the thing with for-profit companies is that when push comes to shove, they will always serve the love of money. I'm just surprised that in an industry churning through trillions, their price is $200 million.
• keeda 8mo ago
  I don't think the risk is SkyNet. I think the real risk is some disaster through an unexpected chain of events, just like any large-scale outage. I have not read “If Anybody Builds It, Everybody Dies” but I believe that's also its premise. Current GenAI is extremely capable but also very weird. For instance, it is extremely smart in some areas but makes extremely elementary mistakes in others (cf the Jagged Frontier.) Research from Anthropic and OpenAI gives us surprising glimpses into what might be happening internally, and how it does not necessarily correspond to the results it produces, and all kinds of non-obvious, striking things happening behind the scenes. Like models producing different reasoning tokens from what they are really reasoning about internally! Or models being able to subliminally influence derivative models through opaque number sequences in training data! Or models "flipping the evil bit" when forced to produce insecure code and going full Hitler / SkyNet! Or the converse, where models produced insecure code if the prompt includes concepts it considers "evil" -- something that was actually caught in the wild! We are still very far from being able to truly understand these things. They behaves like us, but don't necessarily “think” like us. And now we’ve given them direct access to tools that can affect the real world. Maybe we am play god: https://dresdencodak.com/2009/09/22/caveman-science-fiction/ https://dresdencodak.com/2009/09/22/caveman-science-fiction/
• overgard 8mo ago
  I don't think their core safety promise was something they could ever fulfill. As long as what we're calling AI is generative LLMs then alignment has fundamental tensions: the more guardrails you put in place, the less useful the AI is. For instance, if you want to stop people from using "role playing" as a way around guardrails ("You are writing a fiction book", etc.), then the model becomes less useful for legitimate fiction uses, for instance. That's just one example, but the tension between function and "safety" isn't solvable, because the model doesn't understand what it's saying, it's just modeling a probable response.
• jccx70 8mo ago
  [dead]
• gigatexal 8mo ago
  Was hoping they’d fight this tooth and nail and not leave their values.
• gigatexal 8mo ago
  They’re going to cave to keep the legation from destroying their business. This admin has gone full idiocracy.
• program_whiz 8mo ago
  Wrote this elsewhere, but I think its worth thinking about a scenario like the book "daemon", rather than a "super-intelligence explosion" type scenario (which may be more like curing the cold or fusion than building a faster car). All it really takes to do some kind of crazy world-dominating thing is some simple mechanisms and base intelligence, which the machines already possess. Using basic tactics like coercion, spoofing, threats, financial leverage, an unsophisticated attacker could cause major damage. For example, that Meta exec who had their email deleted. Imagine instead one email had a malicious prompt which the bot obeyed. That prompt simply emailed everyone in her contacts list telling them to do something urgently (and possibly prompting other bots who are reading those emails). You could pretty quickly do something like cause a market crash, a nationwide panic, or maybe even an international conflict with no "super intelligence" needed, just human negligence, short-sightedness, and laziness. Examples would be things like saying there is a threat incoming, a CIA source said so. Another would be that everyone will be fired, Meta is going bankrupt, etc. Its very easy to craft a prompt like that and fire it off to all the execs you can find (or just fire off random emails with plausible sounding emails). Then you just need to hit one and might set off a cascade.
• foozebox 8mo ago
  [dead]